changed app to use json config for pipeline steps
readme command line usage - to specify pipeline file name readme updated to include reasoning behind the project use native golang sqlite RunScriptCommand named in functionMap removed unused functions removed unused functions run script and pipeline example renamed functions to drop the word script and add pipeline verb
This commit is contained in:
jon brookes
parent
bd7cee720a
commit
924954d0ff
49 changed files with 2059 additions and 101 deletions
219
README.md
219
README.md
|
|
@ -1,11 +1,12 @@
|
|||
# INFCTL CLI
|
||||
|
||||
A command-line tool for automated deployment and management of an MVK (Minimal Viable Kubernetes) infrastrucure. The CLI orchestrates Kubernetes deployments by executing shell scripts and applying Kubernetes manifests through a pipeline-based approach.
|
||||
A command-line tool for automated deployment and management of an MVK (Minimal Viable Kubernetes) infrastrucure. The CLI orchestrates Kubernetes deployments by executing shell scripts and applying Kubernetes manifests through a JSON-defined pipeline approach.
|
||||
|
||||
## Table of Contents
|
||||
|
||||
- [Overview](#overview)
|
||||
- [Features](#features)
|
||||
- [Design Philosophy](#design-philosophy)
|
||||
- [Prerequisites](#prerequisites)
|
||||
- [Installation](#installation)
|
||||
- [Configuration](#configuration)
|
||||
|
|
@ -20,27 +21,40 @@ A command-line tool for automated deployment and management of an MVK (Minimal V
|
|||
|
||||
## Overview
|
||||
|
||||
INFCTL CLI is a Go-based deployment orchestrator that automates the setup and deployment of INFCTL applications in Kubernetes environments. The tool executes a series of predefined scripts from the `scripts/` directory and applies Kubernetes manifests from the `k8s-manifests/` directory using kubectl and kustomize.
|
||||
INFCTL CLI is a Go-based deployment orchestrator that automates the setup and deployment of INFCTL applications in Kubernetes environments. The tool executes a series of predefined scripts from the `scripts/` directory and applies Kubernetes manifests from the `k8s-manifests/` directory using kubectl and kustomize, all defined in a JSON pipeline file.
|
||||
|
||||
## Features
|
||||
|
||||
- **Automated Pipeline Execution**: Runs deployment scripts and manifests in a specific order
|
||||
- **JSON-Defined Pipeline Execution**: Runs deployment scripts and manifests in an order specified in a JSON pipeline file
|
||||
- **Script Orchestration**: Executes shell scripts from the `scripts/` directory for various setup tasks
|
||||
- **Kustomize Integration**: Applies Kubernetes manifests using kubectl kustomize
|
||||
- **Multi-Component Deployment**: Supports INFCTL, Galene, and SL application deployments
|
||||
- **Namespace Management**: Automatically creates required Kubernetes namespaces
|
||||
- **Secret Management**: Automated creation of secrets for databases, SMTP, AWS, etc.
|
||||
- **ConfigMap Management**: Creates and manages application configuration maps
|
||||
- **Infrastructure Setup**: Installs and configures cert-manager, Traefik, Longhorn, and PostgreSQL operators
|
||||
- **Retry Logic**: Built-in retry mechanism for failed operations
|
||||
- **Structured Logging**: JSON-based logging with debug support
|
||||
- **Integrated Testing**: Includes smoke tests using k3d for validation
|
||||
|
||||
## Design Philosophy
|
||||
|
||||
infctl-cli is built on principles derived from over 20 years of experience tackling deployment and orchestration challenges. The design is inspired by a "plugin" mentality, where each plugin is essentially a script. This approach emphasizes simplicity and modularity, allowing each script to act as an independent unit of execution.
|
||||
|
||||
Key principles include:
|
||||
|
||||
- **Script-Based Orchestration**: Each script or program, when executed, returns an exit code that indicates success or failure. This exit code is used to determine the next steps in the pipeline, enabling robust and predictable orchestration.
|
||||
- **Structured Logging**: Scripts produce structured logs that can be consumed by web interfaces or stored in a database. This ensures transparency and traceability, making it easier to debug and monitor deployments.
|
||||
- **Modularity and Reusability**: By treating scripts as plugins, the system encourages reusability and flexibility. New functionality can be added by simply introducing new scripts without altering the core logic.
|
||||
- **UNIX Philosophy**: The design adheres to the UNIX philosophy of building small, composable tools that do one thing well. Each script is a self-contained unit that performs a specific task.
|
||||
|
||||
This philosophy underpins infctl's ability to orchestrate complex deployments while remaining simple and extensible.
|
||||
|
||||
## Prerequisites
|
||||
|
||||
- Go 1.23.3 or later
|
||||
- `DEPLOYMENT_TYPE` environment variable set to one of: `customer_k8s_setup`, `infctl_k8s_setup`, or `sl_k8s_setup`
|
||||
- Kubernetes cluster with kubectl configured (for K8s deployment types)
|
||||
- Bash shell environment
|
||||
- For running tests: k3d installed
|
||||
- Kubernetes cluster with kubectl configured (for actual deployments)
|
||||
- Required Kubernetes operators (installed by the tool for K8s deployments):
|
||||
- cert-manager
|
||||
- Traefik ingress controller
|
||||
|
|
@ -49,6 +63,37 @@ INFCTL CLI is a Go-based deployment orchestrator that automates the setup and de
|
|||
|
||||
## Installation
|
||||
|
||||
### Option 1: Download Pre-built Binary
|
||||
|
||||
You can run an install from `curl -L https://codeberg.org/headshed/infctl-cli/raw/branch/chore/code-refactor/install.sh | bash `
|
||||
|
||||
.or.
|
||||
|
||||
manually download the pre-built binary for your platform from the [releases page](https://codeberg.org/headshed/infctl-cli/releases).
|
||||
|
||||
1. Download the binary for your platform:
|
||||
- **Linux**:
|
||||
```bash
|
||||
wget https://codeberg.org/headshed/infctl-cli/releases/download/v0.0.1/infctl-linux-amd64 -O /usr/local/bin/infctl
|
||||
```
|
||||
- **Windows**:
|
||||
Download the `.exe` file from the [releases page](https://codeberg.org/headshed/infctl-cli/releases) and place it in a directory included in your `PATH`.
|
||||
|
||||
- **macOS (Intel)**:
|
||||
```bash
|
||||
wget https://codeberg.org/headshed/infctl-cli/releases/download/v0.0.1/infctl-darwin-amd64 -O /usr/local/bin/infctl
|
||||
```
|
||||
- **macOS (Apple Silicon)**:
|
||||
```bash
|
||||
wget https://codeberg.org/headshed/infctl-cli/releases/download/v0.0.1/infctl-darwin-arm64 -O /usr/local/bin/infctl
|
||||
```
|
||||
|
||||
2. Make the binary executable:
|
||||
```bash
|
||||
chmod +x /usr/local/bin/infctl
|
||||
```
|
||||
### Option 2: Clone Repository and Build Binary
|
||||
|
||||
1. Clone the repository:
|
||||
```bash
|
||||
git clone <repository-url>
|
||||
|
|
@ -61,28 +106,27 @@ go mod download
|
|||
go build -o infctl-cli .
|
||||
```
|
||||
|
||||
3. (Optional) Install globally:
|
||||
```bash
|
||||
sudo mv infctl-cli /usr/local/bin/
|
||||
```
|
||||
### Quick start example
|
||||
|
||||
4. Quick start example:
|
||||
```bash
|
||||
# Copy configuration examples
|
||||
cp base.json.example base.json
|
||||
cp config.json.example config.json
|
||||
cp pipeline.json.example pipeline.json
|
||||
|
||||
# Edit configuration files as needed
|
||||
# nano base.json
|
||||
# nano config.json
|
||||
# Edit* configuration files as needed
|
||||
# vim base.json
|
||||
# vim config.json
|
||||
# vim pipeline.json
|
||||
# - where vim may be your default or chosen editor be it nano, vi, ed, emacs, etc ...
|
||||
|
||||
# Run with desired deployment type
|
||||
DEPLOYMENT_TYPE=customer_k8s_setup ./infctl-cli
|
||||
# Run with pipeline file
|
||||
./infctl-cli --deployment-file pipeline.json
|
||||
```
|
||||
|
||||
## Configuration
|
||||
|
||||
The infctl-cli requires two configuration files:
|
||||
The infctl-cli requires three configuration files:
|
||||
|
||||
### Base Configuration (`base.json`)
|
||||
|
||||
|
|
@ -114,77 +158,59 @@ Key configuration options:
|
|||
- `static_url`: Static content URL
|
||||
- `port`: Service port
|
||||
|
||||
### Pipeline Configuration (`pipeline.json`)
|
||||
|
||||
Copy and customize the pipeline definition:
|
||||
|
||||
```bash
|
||||
cp pipeline.json.example pipeline.json
|
||||
```
|
||||
|
||||
This file defines the sequence of operations to be executed, including:
|
||||
- Scripts to run
|
||||
- Kubernetes manifests to apply
|
||||
- Order of operations
|
||||
- Specific deployment type configuration
|
||||
|
||||
## Usage
|
||||
|
||||
The infctl-cli requires the `DEPLOYMENT_TYPE` environment variable to be set to specify the type of deployment. Run the CLI from a directory containing your configuration files.
|
||||
|
||||
### Available Deployment Types
|
||||
|
||||
#### 1. Customer Kubernetes Setup
|
||||
For customer-specific Kubernetes deployments:
|
||||
Run the CLI by providing a path to your pipeline JSON file:
|
||||
|
||||
```bash
|
||||
DEPLOYMENT_TYPE=customer_k8s_setup ./infctl-cli
|
||||
```
|
||||
|
||||
This deployment type:
|
||||
- Creates customer-specific Kubernetes resources
|
||||
- Sets up namespaces, secrets, and config maps
|
||||
- Deploys customer applications with custom configurations
|
||||
|
||||
#### 2. INFCTL Kubernetes Setup
|
||||
For INFCTL infrastructure and application deployment:
|
||||
|
||||
```bash
|
||||
DEPLOYMENT_TYPE=inf_k8s_setup ./infctl-cli
|
||||
```
|
||||
|
||||
This deployment type:
|
||||
- Sets up core INFCTL infrastructure
|
||||
- Installs required operators (cert-manager, Traefik, PostgreSQL)
|
||||
- Deploys INFCTL applications and services
|
||||
|
||||
#### 3. SL Kubernetes Setup
|
||||
For SL (Share Lite) application deployment:
|
||||
|
||||
```bash
|
||||
DEPLOYMENT_TYPE=sl_k8s_setup ./infctl-cli
|
||||
```
|
||||
|
||||
This deployment type:
|
||||
- Deploys SL applications using Helm charts
|
||||
- Configures SL-specific resources and networking
|
||||
- Sets up SL instance management
|
||||
|
||||
### Running from Source
|
||||
|
||||
You can also run directly with Go:
|
||||
|
||||
```bash
|
||||
# For Docker Compose setup
|
||||
DEPLOYMENT_TYPE=customer_compose_setup go run main.go
|
||||
|
||||
# For customer Kubernetes setup
|
||||
DEPLOYMENT_TYPE=customer_k8s_setup go run main.go
|
||||
|
||||
# For INFCTL Kubernetes setup
|
||||
DEPLOYMENT_TYPE=inf_k8s_setup go run main.go
|
||||
|
||||
# For SL Kubernetes setup
|
||||
DEPLOYMENT_TYPE=sl_k8s_setup go run main.go
|
||||
./infctl-cli --deployment-file /path/to/pipeline.json
|
||||
```
|
||||
|
||||
The tool will automatically:
|
||||
|
||||
1. Load base and customer configurations
|
||||
2. Initialize SQLite database for state management
|
||||
3. Execute the appropriate deployment pipeline based on `DEPLOYMENT_TYPE`
|
||||
3. Execute the deployment pipeline defined in your JSON file
|
||||
4. Run scripts from the `scripts/` directory
|
||||
5. Apply Kubernetes manifests using kustomize (for K8s deployments)
|
||||
|
||||
### Running from Source
|
||||
|
||||
You can also run directly with Go:
|
||||
|
||||
```bash
|
||||
go run main.go -pipeline /path/to/pipeline.json
|
||||
```
|
||||
|
||||
### Running Tests
|
||||
|
||||
The project includes smoke tests using k3d for validation:
|
||||
|
||||
```bash
|
||||
# Run all tests
|
||||
go test ./... -v
|
||||
|
||||
# Run specific test
|
||||
go test ./app -run TestRunPipeline
|
||||
```
|
||||
|
||||
## Pipeline Execution
|
||||
|
||||
The CLI executes deployment tasks in two main pipelines:
|
||||
The CLI executes deployment tasks defined in your pipeline.json file, which typically includes:
|
||||
|
||||
### Infrastructure Setup Pipeline
|
||||
|
||||
|
|
@ -248,15 +274,6 @@ The `k8s-manifests/` directory contains Kubernetes resources applied via kustomi
|
|||
- `issuer.yaml` - Certificate issuer
|
||||
- `kustomization.yaml` - Kustomize configuration
|
||||
|
||||
### Galene (`k8s-manifests/galene/`)
|
||||
- WebRTC conferencing application manifests
|
||||
- TCP/UDP ingress configurations
|
||||
- Certificate management
|
||||
|
||||
### SL Applications (`k8s-manifests/sl/`)
|
||||
- Helm chart templates for SL instances
|
||||
- `Chart.yaml` and `values.yaml` for Helm deployments
|
||||
|
||||
## Project Structure
|
||||
|
||||
```
|
||||
|
|
@ -266,25 +283,21 @@ infctl-cli/
|
|||
├── base.json.example # Base configuration template
|
||||
├── config.json.example # Customer configuration template
|
||||
├── app/ # Core application logic
|
||||
│ ├── app.go # Pipeline orchestration and state management
|
||||
│ └── k8s.go # Kubernetes operations (kubectl, kustomize)
|
||||
│ ├── app.go # Pipeline orchestration and state management
|
||||
│ └── k8s.go # Kubernetes operations (kubectl, kustomize)
|
||||
├── config/ # Configuration management
|
||||
│ ├── base.go # Base configuration handling
|
||||
│ └── customer.go # Customer configuration handling
|
||||
│ ├── base.go # Base configuration handling
|
||||
│ └── customer.go # Customer configuration handling
|
||||
├── database/ # SQLite database operations
|
||||
├── scripts/ # Shell scripts executed by the CLI
|
||||
│ ├── install_*.sh # Infrastructure installation scripts
|
||||
│ ├── create_*_secrets.sh # Secret creation scripts
|
||||
│ ├── create_*_configmap_*.sh # ConfigMap creation scripts
|
||||
│ └── galene/ # Galene-specific scripts
|
||||
├── k8s-manifests/ # Kubernetes manifests applied via kustomize
|
||||
│ ├── inf/ # INFCTL application manifests
|
||||
│ ├── inf-ingress/ # INFCTL ingress configuration
|
||||
│ ├── galene/ # Galene application manifests
|
||||
│ ├── sl/ # SL application Helm charts
|
||||
│ └── sl-certificate/ # SSL certificate management
|
||||
│ ├── install_*.sh # Infrastructure installation scripts
|
||||
│ ├── create_*_secrets.sh # Secret creation scripts
|
||||
│ └── create_*_configmap_*.sh # ConfigMap creation scripts
|
||||
├── k8s-manifests/ # Kubernetes manifests applied via kustomize
|
||||
│ ├── ctl/ # INFCTL application manifests
|
||||
│ └── ctl-ingress/ # INFCTL ingress configuration
|
||||
├── templates/ # Template files for configuration generation
|
||||
└── files/ # Static configuration files
|
||||
└── files/ # Static configuration files
|
||||
```
|
||||
|
||||
## Development
|
||||
|
|
@ -302,15 +315,16 @@ The CLI uses structured JSON logging. Debug logs are enabled by default and incl
|
|||
|
||||
### Adding New Scripts
|
||||
|
||||
1. Place shell scripts in the `scripts/` directory
|
||||
2. Add script execution to the appropriate pipeline in `app/app.go`
|
||||
3. Use the `k8sRunSriptCommand()` function to execute scripts
|
||||
1. Place shell scripts / executables in the `scripts/` directory
|
||||
2. Add confiiguration as appropriate into `pipeline.json`
|
||||
3. Re-run `cnfctl --deployment-file pipeline.json`
|
||||
|
||||
### Adding New Manifests
|
||||
|
||||
1. Create Kubernetes YAML files in the appropriate `k8s-manifests/` subdirectory
|
||||
2. Include a `kustomization.yaml` file for kustomize processing
|
||||
3. Add manifest application to the pipeline using `k8sRunKustomizeCommand()`
|
||||
3. Add confiiguration as appropriate into `pipeline.json`
|
||||
4. Re-run `cnfctl --deployment-file pipeline.json`
|
||||
|
||||
## Contributing
|
||||
|
||||
|
|
@ -323,3 +337,6 @@ The CLI uses structured JSON logging. Debug logs are enabled by default and incl
|
|||
## License
|
||||
|
||||
This project is licensed under the GNU General Public License v3.0. See the [LICENSE](./LICENSE) file for details.
|
||||
## License
|
||||
|
||||
This project is licensed under the GNU General Public License v3.0. See the [LICENSE](./LICENSE) file for details.
|
||||
|
|
|
|||
Loading…
Add table
Add a link
Reference in a new issue