jesus/provisioning
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
config-driven
provisioning/core/nulib/main_provisioning/ops.nu
Jesús Pérez 6c538b62c8
feat: Complete config-driven architecture migration v2.0.0 
Transform provisioning system from ENV-based to hierarchical config-driven architecture.
This represents a complete system redesign with breaking changes requiring migration.

## Migration Summary
- 65+ files migrated across entire codebase
- 200+ ENV variables replaced with 476 config accessors
- 29 syntax errors fixed across 17 files
- 92% token efficiency maintained during migration

## Core Features Added

### Hierarchical Configuration System
- 6-layer precedence: defaults → user → project → infra → env → runtime
- Deep merge strategy with intelligent precedence rules
- Multi-environment support (dev/test/prod) with auto-detection
- Configuration templates for all environments

### Enhanced Interpolation Engine
- Dynamic variables: {{paths.base}}, {{env.HOME}}, {{now.date}}
- Git context: {{git.branch}}, {{git.commit}}, {{git.remote}}
- SOPS integration: {{sops.decrypt()}} for secrets management
- Path operations: {{path.join()}} for dynamic construction
- Security: circular dependency detection, injection prevention

### Comprehensive Validation
- Structure, path, type, semantic, and security validation
- Code injection and path traversal detection
- Detailed error reporting with actionable messages
- Configuration health checks and warnings

## Architecture Changes

### Configuration Management (core/nulib/lib_provisioning/config/)
- loader.nu: 1600+ line hierarchical config loader with validation
- accessor.nu: 476 config accessor functions replacing ENV vars

### Provider System (providers/)
- AWS, UpCloud, Local providers fully config-driven
- Unified middleware system with standardized interfaces

### Task Services (core/nulib/taskservs/)
- Kubernetes, storage, networking, registry services migrated
- Template-driven configuration generation

### Cluster Management (core/nulib/clusters/)
- Complete lifecycle management through configuration
- Environment-specific cluster templates

## New Configuration Files
- config.defaults.toml: System defaults (84 lines)
- config.*.toml.example: Environment templates (400+ lines each)
- Enhanced CLI: validate, env, multi-environment support

## Security Enhancements
- Type-safe configuration access through validated functions
- SOPS integration for encrypted secrets management
- Input validation preventing injection attacks
- Environment isolation and access controls

## Breaking Changes
⚠️  ENV variables no longer supported as primary configuration
⚠️  Function signatures require --config parameter
⚠️  CLI arguments and return types modified
⚠️  Provider authentication now config-driven

## Migration Path
1. Backup current environment variables
2. Copy config.user.toml.example → config.user.toml
3. Migrate ENV vars to TOML format
4. Validate: ./core/nulib/provisioning validate config
5. Test functionality with new configuration

## Validation Results
✅ Structure valid
✅ Paths valid
✅ Types valid
✅ Semantic rules valid
✅ File references valid

System ready for production use with config-driven architecture.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
2025-09-23 03:36:50 +01:00

216 lines
16 KiB
Plaintext
Raw Permalink Blame History

use ../lib_provisioning/config/accessor.nu *
export def provisioning_options [
]: nothing -> string {
let target_items = $"(_ansi blue)server(_ansi reset) | (_ansi yellow)tasks(_ansi reset) | (_ansi purple)cluster(_ansi reset)"
(
$"(_ansi green_bold)Options(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) sed - to edit content from a SOPS file \n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) ssh - to config and get SSH settings for servers\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) list [items] - to list items: " +
$"[ (_ansi green)providers(_ansi reset) p | (_ansi green)tasks(_ansi reset) t | (_ansi green)nfra(_ansi reset) k ]\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) nu - to run a nushell in ((get-base-path)) path\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) qr - to get ((get-provisioning-url)) QR code\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) context - to change (_ansi blue)context(_ansi reset) settings. " +
$"(_ansi default_dimmed)use context -h for help(_ansi reset)\n" +
$"\n(_ansi green_bold)Targets(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) generate - to generate (_ansi blue)((get-provisioning-name))(_ansi reset) use one option: (_ansi green)provision(_ansi reset) " +
$"| ($target_items)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) create - to create use one option: ($target_items)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) delete - to delete use one option: ($target_items)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) cst - to create (_ansi blue)Servers(_ansi reset) and (_ansi yellow)Tasks(_ansi reset). " +
$"Alias from (_ansi blue_bold)create-servers-tasks(_ansi reset)\n" +
$"\n(_ansi blue)((get-provisioning-name))(_ansi reset) deploy-sel - to sel (_ansi blue)((get-provisioning-name))(_ansi reset) " +
$"(_ansi cyan_bold)deployments info(_ansi reset) --onsel [ (_ansi yellow_bold)e(_ansi reset)dit | " +
$"(_ansi yellow_bold)v(_ansi reset)iew | (_ansi yellow_bold)l(_ansi reset)ist | (_ansi yellow_bold)t(_ansi reset)ree " +
$"(_ansi yellow_bold)c(_ansi reset)ode | (_ansi yellow_bold)s(_ansi reset)hell | (_ansi yellow_bold)n(_ansi reset)u ]\n" +
$"\n(_ansi blue)((get-provisioning-name))(_ansi reset) deploy-rm - to remove (_ansi blue)((get-provisioning-name))(_ansi reset) " +
$"(_ansi cyan_bold)deployments infos(_ansi reset)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) destroy - to remove (_ansi blue)((get-provisioning-name))(_ansi reset) " +
$"(_ansi cyan_bold)deployments infos(_ansi reset) and (_ansi green_bold)servers(_ansi reset) with confirmation or add '--yes'\n" +
$"\n(_ansi green_bold)Targets(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) server - On Servers or instances \n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) taskserv - On Task Services for servers: settings, services\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) cluster - On Cluster for provisioning\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) infra - On Infrastructures for provisioning\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) validate - Infrastructure validation and review tool\n" +
$"\n(_ansi green_bold)Others(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) show - To show (_ansi blue)((get-provisioning-name))(_ansi reset) settings and data \n" +
$"(_ansi default_dimmed)Options:(_ansi reset) (_ansi blue)((get-provisioning-name))(_ansi reset) show [ settings | defsettings | servers | serverdefs | costs | alldata | data ] \n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) new - To create a new (_ansi blue)((get-provisioning-name))(_ansi reset) Infrastructure \n" +
$"\n(_ansi default_dimmed)To get help on Targets use:(_ansi reset) (_ansi blue)((get-provisioning-name))(_ansi reset) [target-name] help\n" +
$"\n(_ansi default_dimmed)NOTICE: Most of Options and Targets have a shortcut by using a single dash and a letter(_ansi reset)\n" +
$"(_ansi default_dimmed)example(_ansi reset) -h (_ansi default_dimmed)for(_ansi reset)" +
$" --helpinfo (_ansi default_dimmed)or(_ansi reset) help" +
$" (_ansi default_dimmed)even it can simply be used as(_ansi reset) h \n"
)
}
export def provisioning_context_options [
]: nothing -> string {
(
$"(_ansi green_bold)Context options(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) install - to install (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)context(_ansi reset) \n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) view - to view (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)context(_ansi reset)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) default [name] - to set default as [name] \n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) remove [name] - to remove [name] from (_ansi yellow)context(_ansi reset)\n" +
$"\n(_ansi blue)((get-provisioning-name))(_ansi reset) set [name] -k [key] -v [value] - to set (_ansi green)[key] = [value](_ansi reset) in [name] (_ansi yellow)context(_ansi reset)"
)
}
export def provisioning_setup_options [
]: nothing -> string {
(
$"(_ansi green_bold)Setup options(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) providers - to view (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)context(_ansi reset) use 'check' or 'help'\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) tools - to install (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)tools(_ansi reset) use 'check' or 'help'\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) versions - to generate (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)tools versions file (_ansi reset)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) midddleware - to generate (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)providers middleware library(_ansi reset)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) context - to create (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)context file(_ansi reset)\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) defaults - to create (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)defaults file(_ansi reset)"
)
}
export def provisioning_infra_options [
]: nothing -> string {
(
$"(_ansi green_bold)Cloud options(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) view - to view (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)context(_ansi reset)"
)
}
export def provisioning_tools_options [
]: nothing -> string {
(
$"(_ansi green_bold)Tools options(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name)) tools(_ansi reset) - to check (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)tools(_ansi reset) and versions\n" +
$"(_ansi blue)((get-provisioning-name)) tools(_ansi reset) check - to check (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)tools(_ansi reset) and versions\n" +
$"(_ansi blue)((get-provisioning-name)) tools(_ansi reset) install - to install(_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)tools(_ansi reset)\n" +
$"(_ansi blue)((get-provisioning-name)) tools(_ansi reset) show - to show (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)tools(_ansi reset) info \n" +
$"(_ansi blue)((get-provisioning-name)) tools(_ansi reset) show providers - to show (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)providers (_ansi reset) info \n" +
$"(_ansi blue)((get-provisioning-name)) tools(_ansi reset) show all - to show (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)tools and providers (_ansi reset) info \n" +
$"(_ansi blue)((get-provisioning-name)) tools(_ansi reset) info - alias (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi cyan)tools show(_ansi reset) \n" +
$"\n(_ansi blue)((get-provisioning-name)) tools(_ansi reset) (_ansi cyan)[install | check | show](_ansi reset) commmands support to add specifict (_ansi green)'tool-name'(_ansi reset) at the end, " +
$"\n(_ansi blue)((get-provisioning-name)) tools(_ansi reset) (_ansi cyan)show or info(_ansi reset) commmands support to add specifict (_ansi green)'provider-name'(_ansi reset) at the end, " +
$"by default uses (_ansi green)'all'(_ansi reset)" +
$"\n(_ansi blue)((get-provisioning-name)) tools(_ansi reset) (_ansi green)'tool-name'(_ansi reset) to check tool installation and version"
)
}
export def provisioning_generate_options [
]: nothing -> string {
(
$"(_ansi green_bold)Generate options(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)generate new [name-or-path](_ansi reset) - to create a new (_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)directory(_ansi reset)" +
$"\nif '[name-or-path]' is not relative or full path it will be created in (_ansi blue)((get-provisioning-infra-path))(_ansi reset) " +
$"\nadd (_ansi blue)--template [name](_ansi reset) to (_ansi cyan)copy(_ansi reset) from existing (_ansi green)template 'name'(_ansi reset) " +
$"\ndefault (_ansi blue)template(_ansi reset) to use (_ansi cyan)((get-base-path) | path join (get-provisioning-generate-dirpath) | path join "default")(_ansi reset)"
)
}
export def provisioning_show_options [
]: nothing -> string {
(
$"(_ansi green_bold)Show options(_ansi reset):\n" +
$"(_ansi blue)((get-provisioning-name))(_ansi reset) (_ansi yellow)show [options](_ansi reset) - To show (_ansi blue)((get-provisioning-name))(_ansi reset) settings and data (_ansi yellow)(_ansi reset)" +
$"\n(_ansi blue)settings (_ansi reset) to (_ansi cyan)get(_ansi reset) (_ansi green)settings(_ansi reset) " +
$"\n(_ansi blue)defsettings (_ansi reset) to (_ansi cyan)get(_ansi reset) (_ansi green)def settings content (_ansi reset) " +
$"\n(_ansi blue)servers (_ansi reset) to (_ansi cyan)get(_ansi reset) (_ansi green)servers(_ansi reset) " +
$"\n(_ansi blue)defservers (_ansi reset) to (_ansi cyan)get(_ansi reset) (_ansi green)def servers content (_ansi reset) " +
$"\n(_ansi blue)costs (_ansi reset) to (_ansi cyan)get(_ansi reset) (_ansi green) prices or costs (_ansi reset) " +
$"\n(_ansi blue)alldata (_ansi reset) to (_ansi cyan)get(_ansi reset) (_ansi green) all data settings and prices or costs (_ansi reset) " +
$"\n(_ansi blue)data (_ansi reset) to (_ansi cyan)get(_ansi reset) (_ansi green) data settings (_ansi reset) " +
$"\nby adding (_ansi blue)--out(_ansi reset) (_ansi cyan)[ json | yaml | toml ](_ansi reset) it can be used ad a kind of API source" +
$"\nby adding (_ansi blue)--view(_ansi reset) with (_ansi blue)--out(_ansi reset) option, content is formated with highlight"
)
}
export def provisioning_validate_options [
]: nothing -> string {
print "Infrastructure Validation & Review Tool"
print "========================================"
print ""
print "Validates KCL/YAML configurations, checks best practices, and generates reports"
print ""
print "USAGE:"
print $" ((get-provisioning-name)) validate [SUBCOMMAND] [INFRA_PATH] [OPTIONS]"
print ""
print "SUBCOMMANDS:"
print " config Configuration validation - checks TOML config files"
print " (none) Full validation with customizable options"
print " quick Quick validation focusing on errors and critical issues"
print " ci CI/CD optimized validation with structured output"
print " full Comprehensive validation including info-level checks"
print " agent Agent/automation interface with JSON output"
print " rules List all available validation rules"
print " test Run validation system self-tests"
print " help Show detailed help information"
print ""
print "ARGUMENTS:"
print " INFRA_PATH Path to infrastructure configuration (default: current directory)"
print ""
print "OPTIONS:"
print " -f, --fix Auto-fix issues where possible"
print " -r, --report FORMAT Report format: md, yaml, json, all (default: md)"
print " -o, --output DIR Output directory (default: ./validation_results)"
print " -s, --severity LEVEL Minimum severity: info, warning, error, critical (default: warning)"
print " --ci CI/CD mode (exit codes, no colors, minimal output)"
print " -d, --dry-run Show what would be fixed without actually fixing"
print " --rules RULES Comma-separated list of specific rules to run"
print " --exclude RULES Comma-separated list of rules to exclude"
print " -v, --verbose Verbose output (show all details)"
print " -h, --help Show detailed help"
print ""
print "EXIT CODES:"
print " 0 All validations passed"
print " 1 Critical errors found (blocks deployment)"
print " 2 Errors found (should be fixed)"
print " 3 Only warnings found"
print " 4 Validation system error"
print ""
print "VALIDATION RULES:"
print " VAL001 YAML Syntax Validation (critical)"
print " VAL002 KCL Compilation Check (critical)"
print " VAL003 Unquoted Variable References (error, auto-fixable)"
print " VAL004 Required Fields Validation (error)"
print " VAL005 Resource Naming Conventions (warning, auto-fixable)"
print " VAL006 Basic Security Checks (error)"
print " VAL007 Version Compatibility Check (warning)"
print " VAL008 Network Configuration Validation (error)"
print ""
print "Legend: Auto-fixable rules can be automatically corrected with --fix"
print ""
print "EXAMPLES:"
print ""
print " # Validate configuration files"
print $" ((get-provisioning-name)) validate config"
print ""
print " # Validate configuration with strict mode (warnings as errors)"
print $" ((get-provisioning-name)) validate config --strict"
print ""
print " # Validate current directory"
print $" ((get-provisioning-name)) validate"
print ""
print " # Quick validation with auto-fix"
print $" ((get-provisioning-name)) validate quick klab/sgoyol --fix"
print ""
print " # CI/CD validation"
print $" ((get-provisioning-name)) validate ci klab/sgoyol --report yaml"
print ""
print " # Dry run to see what would be fixed"
print $" ((get-provisioning-name)) validate klab/sgoyol --fix --dry-run"
print ""
print " # Generate all report formats"
print $" ((get-provisioning-name)) validate klab/sgoyol --report all --output ./reports"
print ""
print " # List available rules"
print $" ((get-provisioning-name)) validate rules"
print ""
print " # Test the validation system"
print $" ((get-provisioning-name)) validate test"
print ""
""
}
Reference in New Issue View Git Blame Copy Permalink