diff --git a/CLAUDE.md b/CLAUDE.md new file mode 100644 index 0000000..13a7018 --- /dev/null +++ b/CLAUDE.md @@ -0,0 +1,71 @@ +# CLAUDE.md + +This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository. + +## Project Overview + +YABE (YAml Base Extractor) is a GitOps YAML organizer tool written in Rust. It computes common base configurations among multiple YAML files and generates differences for each file, reducing duplication in GitOps workflows. The tool supports quorum-based diffing, YAML sorting, and both configuration file and command-line interfaces. + +## Common Development Commands + +### Building and Testing +```bash +# Build the project +cargo build + +# Run tests +cargo test + +# Build and install locally +cargo install --path . + +# Run with debug logging +cargo run -- --debug [other-args] +``` + +### Running the Tool +```bash +# Basic usage +cargo run -- file1.yaml file2.yaml file3.yaml + +# With configuration file +cargo run -- --config config.yaml + +# In-place modification +cargo run -- -i -r helm_values.yaml file1.yaml file2.yaml + +# Using path patterns +cargo run -- -p "*.yaml" -p "configs/*.yaml" +``` + +## Architecture + +### Core Modules +- **`main.rs`**: CLI interface, argument parsing, and main workflow orchestration +- **`lib.rs`**: Library entry point exposing core functionality +- **`diff.rs`**: Core diffing logic including `compute_diff()` and `diff_and_common_multiple()` +- **`merge.rs`**: YAML merging functionality for combining base files with overrides +- **`sorter.rs`**: YAML content sorting based on user-defined configurations +- **`deep_equal.rs`**: Deep comparison utility for YAML values + +### Key Data Flow +1. **Input Processing**: Parse CLI args and config files, expand glob patterns +2. **YAML Loading**: Read and parse all input YAML files +3. **Base Merging**: If existing base provided, merge with each input file +4. **Diff Computation**: Compute diffs against read-only base (if provided) +5. **Quorum Processing**: Extract common base from diffs using quorum percentage +6. **Output Generation**: Write base file and per-file diffs (in-place or to output folder) + +### Configuration System +The tool supports both command-line arguments and YAML configuration files. Config file values are overridden by command-line arguments. Key configuration options: +- `read_only_base`: Reference YAML for diff computation +- `base`: Existing base to merge with inputs +- `quorum`: Percentage threshold for common base extraction +- `sort_config_path`: YAML sorting rules configuration + +### Testing Structure +Tests are organized by module: +- `test_diff.rs`: Tests for diff computation and multi-file diffing +- `test_deep_equal.rs`: Tests for YAML comparison utility +- `test_sorter.rs`: Tests for YAML sorting functionality +- `test_common.rs`: Shared test utilities and common test cases \ No newline at end of file diff --git a/README.md b/README.md index de2ae8d..fa0c3d9 100644 --- a/README.md +++ b/README.md @@ -8,6 +8,7 @@ YABE is a tool designed to help manage large amounts of YAML files in a GitOps e - **Merge YAML files:** Combine YAML files with a base YAML, either from an existing file or dynamically computed. - **Quorum-based diffing:** Extract common base YAML based on a quorum percentage. - **Sort YAML content:** Sort keys in YAML files based on user-defined configuration. +- **Sort-only mode:** Sort YAML files without performing any diffing operations. - **Helm Values Integration:** Merge input YAML files with Helm values files. - **In-place modification or output to new files.** - **Configuration File Support:** Run the tool using a configuration file to simplify usage in automated workflows. @@ -36,6 +37,7 @@ Options: -q, --quorum Quorum percentage (0-100) [default: 51] --base-out-path (Optional) Base file output path [default: ./base.yaml] --sort-config-path (Optional) Sort configuration file path [default: ./sort-config.yaml], if not provided, will not sort + --sort-only Sort only mode - only sort files without diffing --config (Optional) Configuration file -h, --help Print help -V, --version Print version @@ -62,9 +64,9 @@ This will compute the differences among the override files and generate: * base.yaml: The common base configuration. * file1_diff.yaml, file2_diff.yaml, file3_diff.yaml: The differences for each file. -### Inplace Modification +### In-place Modification -Use the -i or --inplace flag to modify the original override files with their differences: +Use the -i or --in-place flag to modify the original override files with their differences: ```bash ./yabe -i -r helm_values.yaml file1.yaml file2.yaml file3.yaml ``` @@ -76,6 +78,24 @@ Use the --debug flag to enable detailed debug logging: ./yabe --debug -r helm_values.yaml file1.yaml file2.yaml file3.yaml ``` +### Sort Only Mode + +Use the --sort-only flag to only sort YAML files without performing any diffing operations: + +```bash +# Sort files and output to ./out directory +./yabe --sort-only --sort-config-path sort-config.yaml file1.yaml file2.yaml + +# Sort files in-place (modify original files) +./yabe --sort-only --sort-config-path sort-config.yaml -i file1.yaml file2.yaml + +# Sort files using path patterns +./yabe --sort-only --sort-config-path sort-config.yaml -p "*.yaml" -p "configs/*.yaml" + +# Sort files to a specific output directory +./yabe --sort-only --sort-config-path sort-config.yaml -o ./sorted-files *.yaml +``` + ### Using Configuration File You can also use a configuration file to specify options: @@ -100,6 +120,7 @@ debug: false quorum: 60 base_out_path: "./base_output.yaml" sort_config_path: "./sort_config.yaml" +sort_only: false # Set to true for sort-only mode ``` Then run the tool with: @@ -185,7 +206,7 @@ settings: level: 7 ``` -### Inplace Modification Example +### In-place Modification Example Running with the -i flag: ```bash ./yabe -i -r helm_values.yaml file1.yaml file2.yaml file3.yaml diff --git a/src/main.rs b/src/main.rs index 90bf10b..acc538e 100644 --- a/src/main.rs +++ b/src/main.rs @@ -59,6 +59,10 @@ struct Args { /// Sort configuration file path #[arg(long = "sort-config-path", default_value = "./sort-config.yaml")] sort_config_path: String, + + /// Sort only mode - only sort files without diffing + #[arg(long = "sort-only")] + sort_only: bool, } #[derive(Deserialize)] @@ -74,6 +78,110 @@ struct Config { quorum: Option, base_out_path: Option, sort_config_path: Option, + sort_only: Option, +} + +fn sort_only_workflow(args: &Args) -> Result<(), Box> { + info!("Running in sort-only mode"); + + // Process path patterns and add matching files to input_files + let mut expanded_input_files = args.input_files.clone(); + + for pattern in &args.path_patterns { + info!("Expanding path pattern: {}", pattern); + match glob(pattern) { + Ok(paths) => { + for entry in paths { + match entry { + Ok(path) => { + if path.is_file() { + if let Some(path_str) = path.to_str() { + info!("Found matching file: {}", path_str); + expanded_input_files.push(path_str.to_string()); + } + } + } + Err(e) => warn!("Error matching path: {}", e), + } + } + } + Err(e) => warn!("Invalid glob pattern '{}': {}", pattern, e), + } + } + + // Remove duplicates from expanded_input_files + expanded_input_files.sort(); + expanded_input_files.dedup(); + + // Validate that we have files to process + if expanded_input_files.is_empty() { + eprintln!("Error: No input files found for sort-only mode. Please specify either input files or path patterns."); + std::process::exit(1); + } + + // Read sort configuration + let sort_config = if !args.sort_config_path.is_empty() && Path::new(&args.sort_config_path).exists() { + info!("Reading sort configuration file: {}", args.sort_config_path); + let content = fs::read_to_string(&args.sort_config_path)?; + YamlLoader::load_from_str(&content)?.into_iter().next().unwrap_or(Yaml::Null) + } else { + eprintln!("Error: Sort configuration file is required for sort-only mode but not found: {}", args.sort_config_path); + std::process::exit(1); + }; + + if sort_config == Yaml::Null { + eprintln!("Error: Sort configuration file is empty or invalid: {}", args.sort_config_path); + std::process::exit(1); + } + + // Ensure output directory exists if not in-place mode + if !args.inplace { + if !Path::new(&args.out_folder).exists() { + info!("Creating output directory: {}", args.out_folder); + fs::create_dir_all(&args.out_folder)?; + } + } + + // Process each input file + for filename in &expanded_input_files { + info!("Sorting file: {}", filename); + + // Read and parse the YAML file + let content = fs::read_to_string(filename)?; + if let Some(doc) = YamlLoader::load_from_str(&content)?.into_iter().next() { + // Sort the YAML document + let sorted_yaml = sort_yaml(&doc, &sort_config); + + // Convert to string + let mut out_str = String::new(); + { + let mut emitter = YamlEmitter::new(&mut out_str); + emitter.dump(&sorted_yaml)?; + } + out_str = out_str.trim_start_matches("---\n").to_string(); + out_str.push('\n'); + + // Write the sorted content + if args.inplace { + info!("Writing sorted content back to: {}", filename); + fs::write(filename, out_str)?; + } else { + let input_path = Path::new(filename); + let file_stem = input_path + .file_stem() + .and_then(|s| s.to_str()) + .unwrap_or("sorted"); + let sorted_filename = format!("{}/{}_sorted.yaml", args.out_folder, file_stem); + info!("Writing sorted content to: {}", sorted_filename); + fs::write(&sorted_filename, out_str)?; + } + } else { + warn!("No YAML documents found in {}", filename); + } + } + + info!("Sort-only workflow completed successfully."); + Ok(()) } fn main() -> Result<(), Box> { @@ -142,6 +250,12 @@ fn main() -> Result<(), Box> { args.sort_config_path = sort_config_path; } } + + if !args.sort_only { + if let Some(sort_only) = config.sort_only { + args.sort_only = sort_only; + } + } } // Initialize logger with appropriate level @@ -151,6 +265,11 @@ fn main() -> Result<(), Box> { env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init(); } + // Handle sort-only mode + if args.sort_only { + return sort_only_workflow(&args); + } + // Process path patterns and add matching files to input_files let mut expanded_input_files = args.input_files.clone();