Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
71 changes: 71 additions & 0 deletions CLAUDE.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,71 @@
# CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

## Project Overview

YABE (YAml Base Extractor) is a GitOps YAML organizer tool written in Rust. It computes common base configurations among multiple YAML files and generates differences for each file, reducing duplication in GitOps workflows. The tool supports quorum-based diffing, YAML sorting, and both configuration file and command-line interfaces.

## Common Development Commands

### Building and Testing
```bash
# Build the project
cargo build

# Run tests
cargo test

# Build and install locally
cargo install --path .

# Run with debug logging
cargo run -- --debug [other-args]
```

### Running the Tool
```bash
# Basic usage
cargo run -- file1.yaml file2.yaml file3.yaml

# With configuration file
cargo run -- --config config.yaml

# In-place modification
cargo run -- -i -r helm_values.yaml file1.yaml file2.yaml

# Using path patterns
cargo run -- -p "*.yaml" -p "configs/*.yaml"
```

## Architecture

### Core Modules
- **`main.rs`**: CLI interface, argument parsing, and main workflow orchestration
- **`lib.rs`**: Library entry point exposing core functionality
- **`diff.rs`**: Core diffing logic including `compute_diff()` and `diff_and_common_multiple()`
- **`merge.rs`**: YAML merging functionality for combining base files with overrides
- **`sorter.rs`**: YAML content sorting based on user-defined configurations
- **`deep_equal.rs`**: Deep comparison utility for YAML values

### Key Data Flow
1. **Input Processing**: Parse CLI args and config files, expand glob patterns
2. **YAML Loading**: Read and parse all input YAML files
3. **Base Merging**: If existing base provided, merge with each input file
4. **Diff Computation**: Compute diffs against read-only base (if provided)
5. **Quorum Processing**: Extract common base from diffs using quorum percentage
6. **Output Generation**: Write base file and per-file diffs (in-place or to output folder)

### Configuration System
The tool supports both command-line arguments and YAML configuration files. Config file values are overridden by command-line arguments. Key configuration options:
- `read_only_base`: Reference YAML for diff computation
- `base`: Existing base to merge with inputs
- `quorum`: Percentage threshold for common base extraction
- `sort_config_path`: YAML sorting rules configuration

### Testing Structure
Tests are organized by module:
- `test_diff.rs`: Tests for diff computation and multi-file diffing
- `test_deep_equal.rs`: Tests for YAML comparison utility
- `test_sorter.rs`: Tests for YAML sorting functionality
- `test_common.rs`: Shared test utilities and common test cases
27 changes: 24 additions & 3 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@ YABE is a tool designed to help manage large amounts of YAML files in a GitOps e
- **Merge YAML files:** Combine YAML files with a base YAML, either from an existing file or dynamically computed.
- **Quorum-based diffing:** Extract common base YAML based on a quorum percentage.
- **Sort YAML content:** Sort keys in YAML files based on user-defined configuration.
- **Sort-only mode:** Sort YAML files without performing any diffing operations.
- **Helm Values Integration:** Merge input YAML files with Helm values files.
- **In-place modification or output to new files.**
- **Configuration File Support:** Run the tool using a configuration file to simplify usage in automated workflows.
Expand Down Expand Up @@ -36,6 +37,7 @@ Options:
-q, --quorum <QUORUM> Quorum percentage (0-100) [default: 51]
--base-out-path <BASE_OUT_PATH> (Optional) Base file output path [default: ./base.yaml]
--sort-config-path <SORT_CONFIG_PATH> (Optional) Sort configuration file path [default: ./sort-config.yaml], if not provided, will not sort
--sort-only Sort only mode - only sort files without diffing
--config <CONFIG_FILE> (Optional) Configuration file
-h, --help Print help
-V, --version Print version
Expand All @@ -62,9 +64,9 @@ This will compute the differences among the override files and generate:
* base.yaml: The common base configuration.
* file1_diff.yaml, file2_diff.yaml, file3_diff.yaml: The differences for each file.

### Inplace Modification
### In-place Modification

Use the -i or --inplace flag to modify the original override files with their differences:
Use the -i or --in-place flag to modify the original override files with their differences:
```bash
./yabe -i -r helm_values.yaml file1.yaml file2.yaml file3.yaml
```
Expand All @@ -76,6 +78,24 @@ Use the --debug flag to enable detailed debug logging:
./yabe --debug -r helm_values.yaml file1.yaml file2.yaml file3.yaml
```

### Sort Only Mode

Use the --sort-only flag to only sort YAML files without performing any diffing operations:

```bash
# Sort files and output to ./out directory
./yabe --sort-only --sort-config-path sort-config.yaml file1.yaml file2.yaml

# Sort files in-place (modify original files)
./yabe --sort-only --sort-config-path sort-config.yaml -i file1.yaml file2.yaml

# Sort files using path patterns
./yabe --sort-only --sort-config-path sort-config.yaml -p "*.yaml" -p "configs/*.yaml"

# Sort files to a specific output directory
./yabe --sort-only --sort-config-path sort-config.yaml -o ./sorted-files *.yaml
```

### Using Configuration File

You can also use a configuration file to specify options:
Expand All @@ -100,6 +120,7 @@ debug: false
quorum: 60
base_out_path: "./base_output.yaml"
sort_config_path: "./sort_config.yaml"
sort_only: false # Set to true for sort-only mode
```

Then run the tool with:
Expand Down Expand Up @@ -185,7 +206,7 @@ settings:
level: 7
```

### Inplace Modification Example
### In-place Modification Example
Running with the -i flag:
```bash
./yabe -i -r helm_values.yaml file1.yaml file2.yaml file3.yaml
Expand Down
119 changes: 119 additions & 0 deletions src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -59,6 +59,10 @@ struct Args {
/// Sort configuration file path
#[arg(long = "sort-config-path", default_value = "./sort-config.yaml")]
sort_config_path: String,

/// Sort only mode - only sort files without diffing
#[arg(long = "sort-only")]
sort_only: bool,
}

#[derive(Deserialize)]
Expand All @@ -74,6 +78,110 @@ struct Config {
quorum: Option<u8>,
base_out_path: Option<String>,
sort_config_path: Option<String>,
sort_only: Option<bool>,
}

fn sort_only_workflow(args: &Args) -> Result<(), Box<dyn Error>> {
info!("Running in sort-only mode");

// Process path patterns and add matching files to input_files
let mut expanded_input_files = args.input_files.clone();

for pattern in &args.path_patterns {
info!("Expanding path pattern: {}", pattern);
match glob(pattern) {
Ok(paths) => {
for entry in paths {
match entry {
Ok(path) => {
if path.is_file() {
if let Some(path_str) = path.to_str() {
info!("Found matching file: {}", path_str);
expanded_input_files.push(path_str.to_string());
}
}
}
Err(e) => warn!("Error matching path: {}", e),
}
}
}
Err(e) => warn!("Invalid glob pattern '{}': {}", pattern, e),
}
}

// Remove duplicates from expanded_input_files
expanded_input_files.sort();
expanded_input_files.dedup();

// Validate that we have files to process
if expanded_input_files.is_empty() {
eprintln!("Error: No input files found for sort-only mode. Please specify either input files or path patterns.");
std::process::exit(1);
}

// Read sort configuration
let sort_config = if !args.sort_config_path.is_empty() && Path::new(&args.sort_config_path).exists() {
info!("Reading sort configuration file: {}", args.sort_config_path);
let content = fs::read_to_string(&args.sort_config_path)?;
YamlLoader::load_from_str(&content)?.into_iter().next().unwrap_or(Yaml::Null)
} else {
eprintln!("Error: Sort configuration file is required for sort-only mode but not found: {}", args.sort_config_path);
std::process::exit(1);
};

if sort_config == Yaml::Null {
eprintln!("Error: Sort configuration file is empty or invalid: {}", args.sort_config_path);
std::process::exit(1);
}

// Ensure output directory exists if not in-place mode
if !args.inplace {
if !Path::new(&args.out_folder).exists() {
info!("Creating output directory: {}", args.out_folder);
fs::create_dir_all(&args.out_folder)?;
}
}

// Process each input file
for filename in &expanded_input_files {
info!("Sorting file: {}", filename);

// Read and parse the YAML file
let content = fs::read_to_string(filename)?;
if let Some(doc) = YamlLoader::load_from_str(&content)?.into_iter().next() {
// Sort the YAML document
let sorted_yaml = sort_yaml(&doc, &sort_config);

// Convert to string
let mut out_str = String::new();
{
let mut emitter = YamlEmitter::new(&mut out_str);
emitter.dump(&sorted_yaml)?;
}
out_str = out_str.trim_start_matches("---\n").to_string();
out_str.push('\n');

// Write the sorted content
if args.inplace {
info!("Writing sorted content back to: {}", filename);
fs::write(filename, out_str)?;
} else {
let input_path = Path::new(filename);
let file_stem = input_path
.file_stem()
.and_then(|s| s.to_str())
.unwrap_or("sorted");
let sorted_filename = format!("{}/{}_sorted.yaml", args.out_folder, file_stem);
info!("Writing sorted content to: {}", sorted_filename);
fs::write(&sorted_filename, out_str)?;
}
} else {
warn!("No YAML documents found in {}", filename);
}
}

info!("Sort-only workflow completed successfully.");
Ok(())
}

fn main() -> Result<(), Box<dyn Error>> {
Expand Down Expand Up @@ -142,6 +250,12 @@ fn main() -> Result<(), Box<dyn Error>> {
args.sort_config_path = sort_config_path;
}
}

if !args.sort_only {
if let Some(sort_only) = config.sort_only {
args.sort_only = sort_only;
}
}
}

// Initialize logger with appropriate level
Expand All @@ -151,6 +265,11 @@ fn main() -> Result<(), Box<dyn Error>> {
env_logger::Builder::from_env(env_logger::Env::default().default_filter_or("info")).init();
}

// Handle sort-only mode
if args.sort_only {
return sort_only_workflow(&args);
}

// Process path patterns and add matching files to input_files
let mut expanded_input_files = args.input_files.clone();

Expand Down
Loading