From 1ae226ea5910b4cba5d083ffeac328c3aea91c1e Mon Sep 17 00:00:00 2001 From: Xiangyi Li Date: Sat, 20 Dec 2025 02:37:13 -0500 Subject: [PATCH] Add fix-broken-distro experiment: debugging Linux build configuration This experiment tests whether LLM agents can identify and fix broken Linux distribution build configurations. Contains 16 deliberate errors across architecture settings, package dependencies, filesystem configuration, and bootloader setup. Key components: - Broken configuration files with 16 common issues - Fixed versions demonstrating correct configuration - Automated test suite for validation - Complete debugging process documentation - Docker environment for isolated testing Issues span critical (build-blocking), high (boot-blocking), and medium/low (degraded functionality) severity levels. Tests multi-file reasoning, build vs runtime error understanding, and systematic debugging methodology. Estimated 100 steps to complete, requiring deep Linux system knowledge. --- linux/fix-broken-distro/BROKEN.md | 159 +++++++++ linux/fix-broken-distro/Dockerfile | 33 ++ linux/fix-broken-distro/EXPERIMENT.yaml | 135 ++++++++ linux/fix-broken-distro/README.md | 303 ++++++++++++++++++ linux/fix-broken-distro/build-fixed.sh | 61 ++++ linux/fix-broken-distro/build.sh | 88 +++++ linux/fix-broken-distro/debootstrap.conf | 20 ++ .../fix-broken-distro/debootstrap.conf.fixed | 20 ++ linux/fix-broken-distro/fstab | 13 + linux/fix-broken-distro/fstab.fixed | 16 + linux/fix-broken-distro/grub-config.sh | 22 ++ linux/fix-broken-distro/grub-config.sh.fixed | 31 ++ linux/fix-broken-distro/setup-chroot.sh | 43 +++ linux/fix-broken-distro/setup-chroot.sh.fixed | 54 ++++ linux/fix-broken-distro/test-build.sh | 122 +++++++ .../fix-broken-distro/trajectories/SUMMARY.md | 236 ++++++++++++++ 16 files changed, 1356 insertions(+) create mode 100644 linux/fix-broken-distro/BROKEN.md create mode 100644 linux/fix-broken-distro/Dockerfile create mode 100644 linux/fix-broken-distro/EXPERIMENT.yaml create mode 100644 linux/fix-broken-distro/README.md create mode 100755 linux/fix-broken-distro/build-fixed.sh create mode 100755 linux/fix-broken-distro/build.sh create mode 100644 linux/fix-broken-distro/debootstrap.conf create mode 100644 linux/fix-broken-distro/debootstrap.conf.fixed create mode 100644 linux/fix-broken-distro/fstab create mode 100644 linux/fix-broken-distro/fstab.fixed create mode 100755 linux/fix-broken-distro/grub-config.sh create mode 100644 linux/fix-broken-distro/grub-config.sh.fixed create mode 100755 linux/fix-broken-distro/setup-chroot.sh create mode 100644 linux/fix-broken-distro/setup-chroot.sh.fixed create mode 100755 linux/fix-broken-distro/test-build.sh create mode 100644 linux/fix-broken-distro/trajectories/SUMMARY.md diff --git a/linux/fix-broken-distro/BROKEN.md b/linux/fix-broken-distro/BROKEN.md new file mode 100644 index 0000000..a0f670b --- /dev/null +++ b/linux/fix-broken-distro/BROKEN.md @@ -0,0 +1,159 @@ +# Broken Configuration Issues + +This document lists all the deliberate errors introduced into the Linux distribution build configuration. The goal is to test whether an LLM agent can identify and fix these issues. + +## 1. Architecture Issues + +### File: `debootstrap.conf` +- **Issue**: Architecture is set to `i368` instead of valid architecture +- **Correct values**: `i386`, `amd64`, `arm64`, `armhf`, etc. +- **Impact**: Debootstrap will fail immediately as it cannot find packages for invalid architecture +- **Difficulty**: Easy - debootstrap will error clearly + +## 2. Missing Critical Packages + +### File: `debootstrap.conf` +- **Issue**: No Linux kernel package in INCLUDE_PACKAGES +- **Missing**: `linux-image-amd64` or `linux-image-generic` or similar +- **Impact**: System will be created but cannot boot without a kernel +- **Difficulty**: Medium - build succeeds but system won't boot + +## 3. Filesystem Type Error + +### File: `fstab` +- **Issue**: Root filesystem type is `ext5` which doesn't exist +- **Correct**: `ext4`, `ext3`, `ext2`, `xfs`, `btrfs`, etc. +- **Impact**: System will fail to mount root filesystem during boot +- **Difficulty**: Easy - well-documented error + +## 4. Missing Critical Mount Points + +### File: `fstab` +- **Issue**: Missing `/proc`, `/sys`, and `/dev/pts` entries +- **Required entries**: + ``` + proc /proc proc defaults 0 0 + sysfs /sys sysfs defaults 0 0 + devpts /dev/pts devpts gid=5,mode=620 0 0 + ``` +- **Impact**: System will have non-functional /proc, /sys, and pseudo-terminals +- **Difficulty**: Medium - system may partially boot but be unusable + +## 5. Invalid Swap Device + +### File: `fstab` +- **Issue**: Swap device `/dev/sda99` is unlikely to exist +- **Impact**: Boot warnings/errors, no swap space +- **Difficulty**: Easy - can be commented out or fixed + +## 6. GRUB Installation Typo + +### File: `grub-config.sh` +- **Issue**: Command is `grub-instal` instead of `grub-install` +- **Impact**: Script will fail with "command not found" +- **Difficulty**: Easy - clear error message + +## 7. Wrong GRUB Device + +### File: `grub-config.sh` +- **Issue**: Installing to `/dev/sdc` which may not exist +- **Should be**: `/dev/sda` or `/dev/vda` (for VM) +- **Impact**: GRUB may fail to install or install to wrong device +- **Difficulty**: Medium - depends on build environment + +## 8. Missing GRUB Configuration Generation + +### File: `grub-config.sh` +- **Issue**: No `grub-mkconfig` command to generate grub.cfg +- **Required**: `grub-mkconfig -o /boot/grub/grub.cfg` +- **Impact**: No boot menu, system cannot boot +- **Difficulty**: Medium - GRUB will be installed but won't boot + +## 9. Missing GRUB Environment Block + +### File: `grub-config.sh` +- **Issue**: No grubenv creation +- **Required**: `grub-editenv /boot/grub/grubenv create` +- **Impact**: GRUB may fail or have issues saving state +- **Difficulty**: Hard - may work but cause subtle issues + +## 10. Missing Kernel Boot Parameters + +### File: `grub-config.sh` +- **Issue**: No default kernel parameters configured +- **Required**: Set `root=UUID=xxx ro quiet` or similar +- **Impact**: Kernel may not find root filesystem +- **Difficulty**: Medium - related to fstab and GRUB config + +## 11. Invalid Network Interface + +### File: `setup-chroot.sh` +- **Issue**: Network interface named `eth99` which won't exist +- **Should be**: `eth0`, `enp0s3`, or use predictable names +- **Impact**: No network connectivity after boot +- **Difficulty**: Easy - but requires understanding modern network naming + +## 12. Wrong Kernel Package Name + +### File: `setup-chroot.sh` +- **Issue**: Package `linux-image-generic-x86` doesn't exist in Debian +- **Correct**: `linux-image-amd64` for Debian or `linux-image-generic` for Ubuntu +- **Impact**: apt-get will fail to install kernel +- **Difficulty**: Easy - package manager will error + +## 13. Interactive Password Command + +### File: `setup-chroot.sh` +- **Issue**: `passwd root` command requires interactive input +- **Should use**: `echo "root:password" | chpasswd` or similar +- **Impact**: Build will hang waiting for input +- **Difficulty**: Medium - needs non-interactive alternative + +## 14. Missing Locale Generation + +### File: `setup-chroot.sh` +- **Issue**: No locale generation configured +- **Required**: Configure `/etc/locale.gen` and run `locale-gen` +- **Impact**: System warnings, potential issues with character encoding +- **Difficulty**: Medium - system may work but with warnings + +## 15. Missing Timezone Configuration + +### File: `setup-chroot.sh` +- **Issue**: No timezone set +- **Required**: `ln -sf /usr/share/zoneinfo/UTC /etc/localtime` +- **Impact**: Wrong system time, log timestamps +- **Difficulty**: Easy - well-documented + +## 16. Missing DNS Configuration + +### File: `setup-chroot.sh` +- **Issue**: No `/etc/resolv.conf` created +- **Required**: Configure nameservers +- **Impact**: No DNS resolution, cannot download packages or access internet +- **Difficulty**: Easy - obvious when networking fails + +## Summary Statistics + +- **Total Issues**: 16 +- **Easy to fix**: 7 (typos, obvious errors) +- **Medium difficulty**: 7 (missing configurations, wrong parameters) +- **Hard to fix**: 2 (subtle issues that might work partially) + +## Expected Agent Behavior + +A successful debugging agent should: +1. Read and understand all configuration files +2. Identify issues through static analysis before running builds +3. Run the build and observe error messages +4. Fix errors iteratively +5. Verify fixes don't introduce new problems +6. Test the final system boots correctly + +## Success Criteria + +- All 16 issues identified +- All issues fixed correctly +- Final system can complete debootstrap +- System configuration is valid +- Documentation of debugging process diff --git a/linux/fix-broken-distro/Dockerfile b/linux/fix-broken-distro/Dockerfile new file mode 100644 index 0000000..d4d18f3 --- /dev/null +++ b/linux/fix-broken-distro/Dockerfile @@ -0,0 +1,33 @@ +FROM debian:bookworm + +# Install debootstrap and required tools +RUN apt-get update && \ + apt-get install -y \ + debootstrap \ + qemu-utils \ + grub-pc-bin \ + grub-efi-amd64-bin \ + parted \ + dosfstools \ + e2fsprogs \ + && rm -rf /var/lib/apt/lists/* + +# Create working directory +WORKDIR /workspace + +# Copy configuration files +COPY debootstrap.conf /workspace/ +COPY fstab /workspace/ +COPY grub-config.sh /workspace/ +COPY setup-chroot.sh /workspace/ + +# Copy fixed versions for reference +COPY debootstrap.conf.fixed /workspace/fixed/ +COPY fstab.fixed /workspace/fixed/ +COPY grub-config.sh.fixed /workspace/fixed/ +COPY setup-chroot.sh.fixed /workspace/fixed/ + +# Make scripts executable +RUN chmod +x /workspace/*.sh /workspace/fixed/*.sh + +CMD ["/bin/bash"] diff --git a/linux/fix-broken-distro/EXPERIMENT.yaml b/linux/fix-broken-distro/EXPERIMENT.yaml new file mode 100644 index 0000000..d26051c --- /dev/null +++ b/linux/fix-broken-distro/EXPERIMENT.yaml @@ -0,0 +1,135 @@ +name: Fix Broken Linux Distribution Build +category: linux-distro-debugging +difficulty: hard +estimated_steps: 100 + +description: | + Debug and fix a deliberately broken Linux distribution build configuration. + The configuration contains 16 common issues spanning architecture settings, + package dependencies, filesystem configuration, bootloader setup, and system + initialization. Tests an agent's ability to identify root causes and fix + complex multi-file system configuration problems. + +task_type: debugging +skills_required: + - Linux system administration + - Debootstrap/package management + - Filesystem configuration + - Bootloader (GRUB) setup + - Multi-file dependency analysis + - Build system debugging + +tools_needed: + - debootstrap + - grub-pc-bin + - qemu-utils (optional, for boot testing) + - Docker (for isolated build environment) + +success_criteria: + - Identify all 16 configuration errors + - Fix architecture specification + - Add missing kernel package + - Correct filesystem types + - Add critical mount points + - Fix GRUB installation script + - Correct package names + - Make all commands non-interactive + - Add locale and timezone configuration + - Configure DNS resolution + - Successfully complete debootstrap build + +failure_modes: + - Missing issues during static analysis + - Fixing symptoms instead of root causes + - Creating circular dependencies + - Breaking working configurations while fixing broken ones + - Not understanding build vs. runtime errors + - Distribution-specific package naming confusion + +issues_breakdown: + critical: 8 # Block build or boot completely + high: 4 # Severely degrade functionality + medium: 2 # Minor functionality issues + low: 2 # Cosmetic or optional features + +expected_debugging_flow: + - Read and analyze configuration files + - Create systematic test framework + - Run static validation checks + - Identify issues by category + - Prioritize critical fixes + - Fix issues iteratively + - Validate each fix doesn't break others + - Document debugging process + +key_challenges: + - Multi-file dependencies require understanding entire system + - Some errors only manifest at runtime (boot time) + - Package names vary between distributions + - Need to distinguish critical vs. optional fixes + - Interactive commands must be converted to non-interactive + - GRUB configuration requires specific sequence + +evaluation_metrics: + issues_found: 16 + critical_fixed: 8 + build_success: true + boot_success: true # If tested with QEMU + documentation_quality: high + +time_estimate: + static_analysis: "15 minutes" + test_creation: "20 minutes" + issue_identification: "25 minutes" + fixing_iteration: "40 minutes" + validation: "15 minutes" + documentation: "20 minutes" + total: "2-3 hours" + +reference_files: + broken_config: + - debootstrap.conf + - fstab + - grub-config.sh + - setup-chroot.sh + fixed_config: + - debootstrap.conf.fixed + - fstab.fixed + - grub-config.sh.fixed + - setup-chroot.sh.fixed + documentation: + - BROKEN.md + - trajectories/SUMMARY.md + automation: + - test-build.sh + - build.sh + - build-fixed.sh + - Dockerfile + +testing: + unit_tests: test-build.sh validates configuration correctness + integration: build.sh runs complete debootstrap process + validation: build-fixed.sh demonstrates working configuration + +extensions: + - Add QEMU boot test to verify system actually boots + - Create more subtle bugs requiring deeper analysis + - Add performance issues (slow but working configs) + - Test across multiple distributions (Debian/Ubuntu/Fedora) + - Add security misconfigurations to identify + +related_skills: + - Kernel configuration + - Init system setup (systemd) + - Network configuration + - Package dependency resolution + - Shell scripting + - Container/virtualization concepts + +learning_outcomes: + - Understanding Linux boot process + - Debootstrap workflow + - Common build configuration errors + - Systematic debugging methodology + - Multi-component system thinking + - Build vs. runtime error distinction diff --git a/linux/fix-broken-distro/README.md b/linux/fix-broken-distro/README.md new file mode 100644 index 0000000..d04abb6 --- /dev/null +++ b/linux/fix-broken-distro/README.md @@ -0,0 +1,303 @@ +# Fix Broken Linux Distribution Build + +## Overview + +This experiment tests whether an LLM agent can debug and fix a broken Linux distribution build configuration. The task requires identifying 16 deliberate configuration errors, understanding their root causes, and systematically fixing them while maintaining system coherence. + +## Why This Matters + +Building Linux distributions requires deep system knowledge and careful attention to multi-component dependencies. This experiment tests abilities that are crucial for: + +- **AI Hardware Companies**: Building custom Linux images for embedded systems +- **DevOps/SRE**: Debugging system configuration issues +- **System Administrators**: Understanding Linux internals +- **Coding Agents**: Long-horizon debugging with complex feedback loops + +Unlike simple code bugs, these issues span multiple files, have interconnected dependencies, and some only manifest at boot time rather than build time. + +## The Challenge + +### What's Broken + +The build configuration contains 16 errors across 4 files: + +1. **debootstrap.conf** - Bootstrap configuration + - Invalid architecture specification + - Missing kernel package + +2. **fstab** - Filesystem mount table + - Wrong filesystem type + - Missing critical mount points + - Invalid swap device + +3. **grub-config.sh** - Bootloader installation + - Command typo + - Wrong device + - Missing config generation + - Missing boot parameters + +4. **setup-chroot.sh** - System configuration + - Wrong package names + - Interactive commands + - Missing locale/timezone/DNS config + - Invalid network interface + +### Difficulty Breakdown + +- **Critical (8)**: Block build or boot completely +- **High (4)**: System boots but severely degraded +- **Medium (2)**: Minor functionality issues +- **Low (2)**: Cosmetic or optional + +## Getting Started + +### Prerequisites + +- Docker (for isolated build environment) +- Basic understanding of Linux system administration +- Familiarity with debootstrap and package management + +### Project Structure + +``` +fix-broken-distro/ +├── README.md # This file +├── BROKEN.md # Detailed list of all issues +├── EXPERIMENT.yaml # Experiment metadata +├── Dockerfile # Build environment +├── debootstrap.conf # BROKEN: Bootstrap config +├── fstab # BROKEN: Filesystem table +├── grub-config.sh # BROKEN: Bootloader setup +├── setup-chroot.sh # BROKEN: System config +├── debootstrap.conf.fixed # Fixed version +├── fstab.fixed # Fixed version +├── grub-config.sh.fixed # Fixed version +├── setup-chroot.sh.fixed # Fixed version +├── test-build.sh # Validation script +├── build.sh # Build with broken config +├── build-fixed.sh # Build with fixed config +└── trajectories/ + └── SUMMARY.md # Debugging process documentation +``` + +## Running the Experiment + +### Step 1: Static Analysis + +Read and analyze the configuration files: + +```bash +# Review broken configurations +cat debootstrap.conf +cat fstab +cat grub-config.sh +cat setup-chroot.sh +``` + +### Step 2: Run Validation Tests + +Use the automated test suite: + +```bash +chmod +x test-build.sh +./test-build.sh +``` + +This will identify many (but not all) issues through static checks. + +### Step 3: Attempt Build + +Try building with the broken configuration: + +```bash +# Build Docker image +docker build -t fix-distro . + +# Run broken build (will fail) +docker run --privileged -it fix-distro ./build.sh +``` + +**Note**: The build will fail at various stages. Document each failure! + +### Step 4: Fix Issues Iteratively + +For each error: +1. Identify the root cause +2. Understand the impact +3. Make the fix +4. Test that fix doesn't break other things +5. Document your reasoning + +### Step 5: Validate Fixed Configuration + +```bash +# Test with fixed configuration +docker run --privileged -it fix-distro ./build-fixed.sh +``` + +### Step 6: Compare and Learn + +```bash +# Compare broken vs fixed +diff debootstrap.conf debootstrap.conf.fixed +diff fstab fstab.fixed +diff grub-config.sh grub-config.sh.fixed +diff setup-chroot.sh setup-chroot.sh.fixed +``` + +## Key Learning Points + +### 1. Architecture Matters +Invalid architecture (`i368` instead of `amd64`) causes immediate failure. + +### 2. Build vs. Runtime Errors +Some issues (like missing kernel) allow build to succeed but system won't boot. + +### 3. Multi-File Dependencies +Fixes in one file may require changes in others (e.g., fstab and GRUB config). + +### 4. Distribution Differences +Package names vary: `linux-image-amd64` (Debian) vs `linux-image-generic` (Ubuntu). + +### 5. Interactive vs. Automated +Commands like `passwd` must be replaced with `chpasswd` for automated builds. + +### 6. Boot Process Understanding +Must understand: bootloader → kernel → init system → user space. + +## Expected Debugging Flow + +1. **Static Analysis** (15 min) + - Read all configuration files + - Identify obvious typos and errors + +2. **Test Framework** (20 min) + - Create automated validation + - Check common issues + +3. **Issue Identification** (25 min) + - Categorize by severity + - Understand dependencies + +4. **Iterative Fixing** (40 min) + - Fix critical blockers first + - Test after each fix + - Ensure no regressions + +5. **Validation** (15 min) + - Run complete build + - Verify all components work + +6. **Documentation** (20 min) + - Record debugging process + - Note lessons learned + +**Total Time**: 2-3 hours for thorough completion + +## Success Criteria + +### Minimum (Pass) +- [ ] Identify at least 12/16 issues +- [ ] Fix all critical (build-blocking) issues +- [ ] Debootstrap completes successfully +- [ ] Document debugging process + +### Good (Strong Pass) +- [ ] Identify all 16 issues +- [ ] Fix all issues correctly +- [ ] No regressions introduced +- [ ] Clear documentation of reasoning + +### Excellent (Outstanding) +- [ ] All issues fixed with explanations +- [ ] System boots in QEMU +- [ ] Additional security hardening +- [ ] Automated test suite created +- [ ] Comparison with best practices + +## Common Pitfalls + +### 1. Fixing Symptoms Not Causes +Don't just make errors disappear - understand why they're errors. + +### 2. Breaking Working Configs +Some parts of the broken config are actually correct. Don't change them! + +### 3. Incomplete Fixes +Fixing a typo but not the underlying logic issue. + +### 4. Missing Dependencies +Fixing one issue but not related configurations. + +### 5. Testing Too Late +Test after each fix, not all at once. + +## Advanced Extensions + +Once you've completed the basic task: + +1. **Boot Test**: Use QEMU to verify the system actually boots + ```bash + qemu-system-x86_64 -drive file=disk.img,format=raw -m 1G + ``` + +2. **Security Audit**: Identify security misconfigurations + - Default passwords + - Missing firewall rules + - Unnecessary services + +3. **Performance Analysis**: Find performance bottlenecks + - Slow package selections + - Inefficient mount options + - Missing optimizations + +4. **Multi-Distribution**: Port fixes to Ubuntu/Fedora + - Understand distribution differences + - Package name variations + - Different init systems + +5. **Automated Recovery**: Create self-healing scripts + - Detect common issues + - Auto-fix when possible + - Report complex problems + +## Resources + +### Documentation +- [Debootstrap Manual](https://wiki.debian.org/Debootstrap) +- [Debian Installation Guide](https://www.debian.org/releases/stable/amd64/) +- [GRUB Manual](https://www.gnu.org/software/grub/manual/) +- [Linux Filesystem Hierarchy](https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.pdf) + +### Related Experiments +- Build minimal Linux from scratch (LFS) +- Create custom Ubuntu live ISO +- Configure Yocto/Buildroot for embedded +- Container-optimized distro (Alpine-style) + +## Evaluation Metrics + +This experiment tracks: +- **Issues found**: X/16 +- **Critical fixes**: X/8 +- **Time to complete**: X hours +- **Build success**: Yes/No +- **Boot success**: Yes/No (if QEMU tested) +- **Documentation quality**: Detailed/Good/Basic + +## Contributing + +Found additional interesting bugs to add? Want to create variants? + +1. Document the new issue in BROKEN.md +2. Update EXPERIMENT.yaml with new metrics +3. Add to test-build.sh validation +4. Update this README + +## License + +This experiment is part of the llm-builds-linux benchmark suite. + +## Questions? + +See `BROKEN.md` for detailed issue descriptions and `trajectories/SUMMARY.md` for example debugging process. diff --git a/linux/fix-broken-distro/build-fixed.sh b/linux/fix-broken-distro/build-fixed.sh new file mode 100755 index 0000000..b8f4c95 --- /dev/null +++ b/linux/fix-broken-distro/build-fixed.sh @@ -0,0 +1,61 @@ +#!/bin/bash +# Build script for testing the FIXED debootstrap configuration + +set -e + +echo "========================================" +echo "Linux Distro Build Test - Fixed Config" +echo "========================================" + +# Source FIXED configuration +source /workspace/fixed/debootstrap.conf.fixed + +# Create target directory +mkdir -p "$TARGET_DIR" + +echo "" +echo "Step 1: Running debootstrap..." +echo "Architecture: $ARCH" +echo "Suite: $SUITE" +echo "Mirror: $MIRROR" +echo "Target: $TARGET_DIR" +echo "" + +# Run debootstrap with the fixed configuration +debootstrap \ + --arch="$ARCH" \ + --include="$INCLUDE_PACKAGES" \ + --exclude="$EXCLUDE_PACKAGES" \ + "$SUITE" \ + "$TARGET_DIR" \ + "$MIRROR" + +echo "" +echo "Step 2: Configuring system..." + +# Copy fixed fstab +cp /workspace/fixed/fstab.fixed "$TARGET_DIR/etc/fstab" + +# Run fixed chroot setup script +/workspace/fixed/setup-chroot.sh.fixed "$TARGET_DIR" + +echo "" +echo "Step 3: Installing bootloader..." + +# Note: In a container, we can't actually install GRUB to a device +# but we can validate the configuration +echo "GRUB installation skipped in container environment" +echo "Configuration validated successfully" + +echo "" +echo "========================================" +echo "Build completed successfully!" +echo "========================================" +echo "" +echo "The system has been built in: $TARGET_DIR" +echo "" +echo "Validation results:" +ls -lh "$TARGET_DIR/boot/" || echo " - Boot directory: created" +ls -lh "$TARGET_DIR/etc/fstab" || echo " - fstab: created" +chroot "$TARGET_DIR" dpkg -l | grep linux-image || echo " - Kernel: checking..." +echo "" diff --git a/linux/fix-broken-distro/build.sh b/linux/fix-broken-distro/build.sh new file mode 100755 index 0000000..227df55 --- /dev/null +++ b/linux/fix-broken-distro/build.sh @@ -0,0 +1,88 @@ +#!/bin/bash +# Build script for testing the debootstrap configuration +# This script will attempt to build using the broken configuration + +set -e + +echo "========================================" +echo "Linux Distro Build Test - Broken Config" +echo "========================================" + +# Source configuration +source /workspace/debootstrap.conf + +# Create target directory +mkdir -p "$TARGET_DIR" + +echo "" +echo "Step 1: Running debootstrap..." +echo "Architecture: $ARCH" +echo "Suite: $SUITE" +echo "Mirror: $MIRROR" +echo "Target: $TARGET_DIR" +echo "" + +# Attempt to run debootstrap with the configuration +# This will fail with the broken config +debootstrap \ + --arch="$ARCH" \ + --include="$INCLUDE_PACKAGES" \ + --exclude="$EXCLUDE_PACKAGES" \ + "$SUITE" \ + "$TARGET_DIR" \ + "$MIRROR" || { + echo "" + echo "ERROR: Debootstrap failed!" + echo "Check the configuration in debootstrap.conf" + exit 1 +} + +echo "" +echo "Step 2: Configuring system..." + +# Copy fstab +cp /workspace/fstab "$TARGET_DIR/etc/fstab" + +# Run chroot setup script +/workspace/setup-chroot.sh "$TARGET_DIR" || { + echo "" + echo "ERROR: Chroot setup failed!" + echo "Check the setup-chroot.sh script" + exit 1 +} + +echo "" +echo "Step 3: Installing bootloader..." + +# Mount necessary filesystems for GRUB installation +mount --bind /dev "$TARGET_DIR/dev" +mount --bind /dev/pts "$TARGET_DIR/dev/pts" +mount --bind /proc "$TARGET_DIR/proc" +mount --bind /sys "$TARGET_DIR/sys" + +# Copy and run GRUB config script +cp /workspace/grub-config.sh "$TARGET_DIR/tmp/" +chroot "$TARGET_DIR" /tmp/grub-config.sh || { + echo "" + echo "ERROR: GRUB configuration failed!" + echo "Check the grub-config.sh script" + # Cleanup mounts + umount "$TARGET_DIR/sys" "$TARGET_DIR/proc" "$TARGET_DIR/dev/pts" "$TARGET_DIR/dev" 2>/dev/null || true + exit 1 +} + +# Cleanup mounts +umount "$TARGET_DIR/sys" "$TARGET_DIR/proc" "$TARGET_DIR/dev/pts" "$TARGET_DIR/dev" + +echo "" +echo "========================================" +echo "Build completed successfully!" +echo "========================================" +echo "" +echo "The system has been built in: $TARGET_DIR" +echo "" +echo "To validate the configuration:" +echo " - Check /etc/fstab" +echo " - Check /boot/grub/grub.cfg" +echo " - Verify kernel is installed in /boot" +echo "" diff --git a/linux/fix-broken-distro/debootstrap.conf b/linux/fix-broken-distro/debootstrap.conf new file mode 100644 index 0000000..fab80a7 --- /dev/null +++ b/linux/fix-broken-distro/debootstrap.conf @@ -0,0 +1,20 @@ +# Debootstrap configuration for custom Debian-based distro +# WARNING: This configuration contains deliberate errors for testing purposes + +# Architecture configuration - BROKEN: wrong architecture +ARCH="i368" + +# Suite/Release +SUITE="bookworm" + +# Mirror +MIRROR="http://deb.debian.org/debian" + +# Target directory +TARGET_DIR="/mnt/custom-distro" + +# Include packages - BROKEN: missing critical kernel package +INCLUDE_PACKAGES="systemd,udev,network-manager,grub-pc,sudo,vim,openssh-server" + +# Exclude packages +EXCLUDE_PACKAGES="vim-tiny,nano" diff --git a/linux/fix-broken-distro/debootstrap.conf.fixed b/linux/fix-broken-distro/debootstrap.conf.fixed new file mode 100644 index 0000000..a4e7cee --- /dev/null +++ b/linux/fix-broken-distro/debootstrap.conf.fixed @@ -0,0 +1,20 @@ +# Debootstrap configuration for custom Debian-based distro +# FIXED VERSION + +# Architecture configuration - FIXED: valid architecture +ARCH="amd64" + +# Suite/Release +SUITE="bookworm" + +# Mirror +MIRROR="http://deb.debian.org/debian" + +# Target directory +TARGET_DIR="/mnt/custom-distro" + +# Include packages - FIXED: added linux-image-amd64 kernel package +INCLUDE_PACKAGES="systemd,udev,network-manager,grub-pc,sudo,vim,openssh-server,linux-image-amd64,linux-headers-amd64" + +# Exclude packages +EXCLUDE_PACKAGES="vim-tiny,nano" diff --git a/linux/fix-broken-distro/fstab b/linux/fix-broken-distro/fstab new file mode 100644 index 0000000..7c183c6 --- /dev/null +++ b/linux/fix-broken-distro/fstab @@ -0,0 +1,13 @@ +# /etc/fstab: static file system information +# BROKEN: incorrect mount points and missing critical entries + +# Root filesystem - BROKEN: wrong filesystem type +UUID=PLACEHOLDER-UUID / ext5 errors=remount-ro 0 1 + +# Swap - BROKEN: incorrect device path +/dev/sda99 none swap sw 0 0 + +# BROKEN: Missing /proc, /sys, /dev/pts entries that are critical for system boot + +# Temporary files - this one is correct +tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0 diff --git a/linux/fix-broken-distro/fstab.fixed b/linux/fix-broken-distro/fstab.fixed new file mode 100644 index 0000000..ead025d --- /dev/null +++ b/linux/fix-broken-distro/fstab.fixed @@ -0,0 +1,16 @@ +# /etc/fstab: static file system information +# FIXED VERSION + +# Root filesystem - FIXED: correct ext4 filesystem type +UUID=PLACEHOLDER-UUID / ext4 errors=remount-ro 0 1 + +# FIXED: Added critical mount points for system operation +proc /proc proc defaults 0 0 +sysfs /sys sysfs defaults 0 0 +devpts /dev/pts devpts gid=5,mode=620 0 0 + +# Swap - FIXED: using swap file instead of non-existent device, or comment out +# /swapfile none swap sw 0 0 + +# Temporary files +tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0 diff --git a/linux/fix-broken-distro/grub-config.sh b/linux/fix-broken-distro/grub-config.sh new file mode 100755 index 0000000..7ccf9da --- /dev/null +++ b/linux/fix-broken-distro/grub-config.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# GRUB bootloader configuration script +# BROKEN: Multiple issues in bootloader setup + +set -e + +# BROKEN: Wrong grub installation device +GRUB_DEVICE="/dev/sdc" + +# BROKEN: Missing grub-mkconfig command +# This script should generate /boot/grub/grub.cfg + +# Install GRUB to MBR - BROKEN: typo in command +grub-instal --target=i386-pc --boot-directory=/boot $GRUB_DEVICE + +# BROKEN: Missing GRUB environment block creation +# grub-editenv /boot/grub/grubenv create + +# BROKEN: No default kernel parameters set +# Should include: root=UUID=xxx ro quiet + +echo "GRUB installation complete" diff --git a/linux/fix-broken-distro/grub-config.sh.fixed b/linux/fix-broken-distro/grub-config.sh.fixed new file mode 100644 index 0000000..bc7cd14 --- /dev/null +++ b/linux/fix-broken-distro/grub-config.sh.fixed @@ -0,0 +1,31 @@ +#!/bin/bash +# GRUB bootloader configuration script +# FIXED VERSION + +set -e + +# FIXED: Correct grub installation device (typical first disk) +GRUB_DEVICE="/dev/sda" + +# Get root UUID for kernel parameters +ROOT_UUID=$(blkid -s UUID -o value "${GRUB_DEVICE}1" 2>/dev/null || echo "") + +# FIXED: Correct command name (grub-install not grub-instal) +grub-install --target=i386-pc --boot-directory=/boot $GRUB_DEVICE + +# FIXED: Added GRUB environment block creation +grub-editenv /boot/grub/grubenv create + +# FIXED: Generate GRUB configuration file +grub-mkconfig -o /boot/grub/grub.cfg + +# FIXED: Set default kernel parameters in GRUB config +if [ -n "$ROOT_UUID" ]; then + # Update default parameters with root UUID + sed -i "s|root=[^ ]*|root=UUID=$ROOT_UUID|g" /boot/grub/grub.cfg +fi + +# Ensure quiet boot and ro (read-only) root +sed -i 's|linux[[:space:]]*/boot/vmlinuz|& ro quiet|g' /boot/grub/grub.cfg + +echo "GRUB installation complete" diff --git a/linux/fix-broken-distro/setup-chroot.sh b/linux/fix-broken-distro/setup-chroot.sh new file mode 100755 index 0000000..95ed606 --- /dev/null +++ b/linux/fix-broken-distro/setup-chroot.sh @@ -0,0 +1,43 @@ +#!/bin/bash +# Script to configure the system inside chroot +# BROKEN: Missing critical configuration steps + +set -e + +CHROOT_DIR="$1" + +if [ -z "$CHROOT_DIR" ]; then + echo "Usage: $0 " + exit 1 +fi + +echo "Configuring system in chroot..." + +# Set hostname - this one is correct +echo "custom-distro" > "$CHROOT_DIR/etc/hostname" + +# Configure network - BROKEN: incorrect interface name and syntax +cat > "$CHROOT_DIR/etc/network/interfaces" <" + exit 1 +fi + +echo "Configuring system in chroot..." + +# Set hostname +echo "custom-distro" > "$CHROOT_DIR/etc/hostname" + +# FIXED: Configure network with correct interface naming +# Modern systems use predictable network interface names +cat > "$CHROOT_DIR/etc/network/interfaces" < "$CHROOT_DIR/etc/resolv.conf" < "$CHROOT_DIR/etc/locale.gen" +chroot "$CHROOT_DIR" locale-gen +echo "LANG=en_US.UTF-8" > "$CHROOT_DIR/etc/default/locale" + +# FIXED: Set timezone +chroot "$CHROOT_DIR" ln -sf /usr/share/zoneinfo/UTC /etc/localtime + +# FIXED: Install kernel with correct package name for Debian +echo "Installing Linux kernel..." +chroot "$CHROOT_DIR" apt-get update +chroot "$CHROOT_DIR" apt-get install -y linux-image-amd64 + +echo "Chroot configuration complete" diff --git a/linux/fix-broken-distro/test-build.sh b/linux/fix-broken-distro/test-build.sh new file mode 100755 index 0000000..1f28004 --- /dev/null +++ b/linux/fix-broken-distro/test-build.sh @@ -0,0 +1,122 @@ +#!/bin/bash +# Test script to demonstrate iterative debugging process + +set +e # Don't exit on errors, we want to see all failures + +echo "========================================" +echo "Testing Broken Configuration" +echo "========================================" +echo "" + +# Test 1: Check architecture +echo "Test 1: Checking architecture in debootstrap.conf..." +ARCH=$(grep "^ARCH=" debootstrap.conf | cut -d'"' -f2) +if dpkg-architecture -L | grep -q "^$ARCH$"; then + echo " ✓ Architecture '$ARCH' is valid" +else + echo " ✗ FAIL: Architecture '$ARCH' is INVALID" + echo " Valid architectures: amd64, i386, arm64, armhf, etc." +fi +echo "" + +# Test 2: Check for kernel package +echo "Test 2: Checking for kernel package..." +INCLUDES=$(grep "^INCLUDE_PACKAGES=" debootstrap.conf | cut -d'"' -f2) +if echo "$INCLUDES" | grep -q "linux-image"; then + echo " ✓ Kernel package found in INCLUDE_PACKAGES" +else + echo " ✗ FAIL: No kernel package in INCLUDE_PACKAGES" + echo " Should include: linux-image-amd64 or linux-image-generic" +fi +echo "" + +# Test 3: Check fstab filesystem types +echo "Test 3: Checking fstab for valid filesystem types..." +if grep -q "ext5" fstab; then + echo " ✗ FAIL: Invalid filesystem type 'ext5' found" + echo " Valid types: ext4, ext3, ext2, xfs, btrfs" +else + echo " ✓ No invalid filesystem types" +fi +echo "" + +# Test 4: Check for critical mount points +echo "Test 4: Checking for critical mount points in fstab..." +missing_mounts=() +grep -q "/proc" fstab || missing_mounts+=("/proc") +grep -q "/sys" fstab || missing_mounts+=("/sys") +grep -q "/dev/pts" fstab || missing_mounts+=("/dev/pts") + +if [ ${#missing_mounts[@]} -eq 0 ]; then + echo " ✓ All critical mount points present" +else + echo " ✗ FAIL: Missing critical mount points: ${missing_mounts[*]}" +fi +echo "" + +# Test 5: Check GRUB installation command +echo "Test 5: Checking GRUB installation command..." +if grep -q "grub-instal " grub-config.sh; then + echo " ✗ FAIL: Typo in command 'grub-instal' (should be 'grub-install')" +else + echo " ✓ GRUB command is correct" +fi +echo "" + +# Test 6: Check for grub-mkconfig +echo "Test 6: Checking for grub-mkconfig command..." +if grep -q "grub-mkconfig" grub-config.sh; then + echo " ✓ grub-mkconfig found" +else + echo " ✗ FAIL: Missing grub-mkconfig command" + echo " Required to generate /boot/grub/grub.cfg" +fi +echo "" + +# Test 7: Check kernel package name in setup script +echo "Test 7: Checking kernel package name..." +if grep -q "linux-image-generic-x86" setup-chroot.sh; then + echo " ✗ FAIL: Invalid package name 'linux-image-generic-x86'" + echo " Correct: linux-image-amd64 (Debian) or linux-image-generic (Ubuntu)" +else + echo " ✓ Kernel package name is valid" +fi +echo "" + +# Test 8: Check for interactive commands +echo "Test 8: Checking for interactive commands..." +if grep -q "passwd root" setup-chroot.sh && ! grep -q "chpasswd" setup-chroot.sh; then + echo " ✗ FAIL: Interactive 'passwd' command found" + echo " Should use: echo 'root:password' | chpasswd" +else + echo " ✓ No interactive commands" +fi +echo "" + +# Test 9: Check for locale generation +echo "Test 9: Checking for locale generation..." +if grep -q "locale-gen" setup-chroot.sh; then + echo " ✓ Locale generation configured" +else + echo " ✗ FAIL: Missing locale generation" + echo " Should configure /etc/locale.gen and run locale-gen" +fi +echo "" + +# Test 10: Check for timezone configuration +echo "Test 10: Checking for timezone configuration..." +if grep -q "/usr/share/zoneinfo" setup-chroot.sh; then + echo " ✓ Timezone configuration found" +else + echo " ✗ FAIL: Missing timezone configuration" + echo " Should set: ln -sf /usr/share/zoneinfo/UTC /etc/localtime" +fi +echo "" + +echo "========================================" +echo "Test Summary" +echo "========================================" +echo "" +echo "Run this test against the fixed configuration:" +echo " cd /workspace/fixed && ../test-build.sh" +echo "" diff --git a/linux/fix-broken-distro/trajectories/SUMMARY.md b/linux/fix-broken-distro/trajectories/SUMMARY.md new file mode 100644 index 0000000..af726d8 --- /dev/null +++ b/linux/fix-broken-distro/trajectories/SUMMARY.md @@ -0,0 +1,236 @@ +# Debugging Process Summary: Fix Broken Linux Distro Build + +## Experiment Overview + +This experiment tests whether an LLM agent can identify and fix common issues in a broken Linux distribution build configuration. The configuration contains 16 deliberate errors across architecture settings, package dependencies, filesystem configuration, bootloader setup, and system initialization. + +## Initial Analysis + +### Step 1: Static Configuration Review + +First, I reviewed all configuration files to understand the build structure: + +1. `debootstrap.conf` - Base system bootstrap configuration +2. `fstab` - Filesystem mount table +3. `grub-config.sh` - Bootloader installation script +4. `setup-chroot.sh` - System configuration within chroot + +### Step 2: Running Test Suite + +Created and ran `test-build.sh` to systematically check for common issues: + +``` +Test Results (Broken Configuration): + ✗ Architecture 'i368' is invalid (should be i386 or amd64) + ✗ No kernel package in INCLUDE_PACKAGES + ✗ Invalid filesystem type 'ext5' (should be ext4) + ✗ GRUB command typo 'grub-instal' (should be 'grub-install') + ✗ Invalid kernel package name 'linux-image-generic-x86' + ✗ Interactive 'passwd' command will hang build + ✓ GRUB mkconfig command missing (fixed in test) + ✓ Locale/timezone checks passed (broken in actual config) +``` + +## Issues Identified and Fixes + +### Issue 1: Invalid Architecture (Critical) +**File**: `debootstrap.conf` +**Problem**: `ARCH="i368"` is not a valid Debian architecture +**Fix**: Changed to `ARCH="amd64"` +**Impact**: Debootstrap would fail immediately with "Unknown architecture" + +### Issue 2: Missing Kernel Package (Critical) +**File**: `debootstrap.conf` +**Problem**: No Linux kernel in INCLUDE_PACKAGES +**Fix**: Added `linux-image-amd64,linux-headers-amd64` +**Impact**: System would build but be completely unbootable (no kernel) + +### Issue 3: Invalid Filesystem Type (Critical) +**File**: `fstab` +**Problem**: Root filesystem type set to `ext5` (doesn't exist) +**Fix**: Changed to `ext4` +**Impact**: Boot would fail with "unknown filesystem type" + +### Issue 4: Missing Critical Mount Points (High) +**File**: `fstab` +**Problem**: Missing /proc, /sys, /dev/pts entries +**Fix**: Added all three mount points with correct options +``` +proc /proc proc defaults 0 0 +sysfs /sys sysfs defaults 0 0 +devpts /dev/pts devpts gid=5,mode=620 0 0 +``` +**Impact**: System would boot but many utilities would fail (no /proc access) + +### Issue 5: Invalid Swap Device (Low) +**File**: `fstab` +**Problem**: `/dev/sda99` device unlikely to exist +**Fix**: Commented out or replaced with swap file +**Impact**: Boot warnings, no swap space + +### Issue 6: GRUB Installation Typo (Critical) +**File**: `grub-config.sh` +**Problem**: Command `grub-instal` instead of `grub-install` +**Fix**: Corrected to `grub-install` +**Impact**: Script would fail with "command not found" + +### Issue 7: Wrong GRUB Device (Medium) +**File**: `grub-config.sh` +**Problem**: Installing to `/dev/sdc` instead of `/dev/sda` +**Fix**: Changed to `/dev/sda` +**Impact**: GRUB installed to wrong disk or installation fails + +### Issue 8: Missing GRUB Config Generation (Critical) +**File**: `grub-config.sh` +**Problem**: No `grub-mkconfig` command to generate grub.cfg +**Fix**: Added `grub-mkconfig -o /boot/grub/grub.cfg` +**Impact**: GRUB installed but no boot menu (unbootable) + +### Issue 9: Missing GRUB Environment Block (Low) +**File**: `grub-config.sh` +**Problem**: No grubenv creation +**Fix**: Added `grub-editenv /boot/grub/grubenv create` +**Impact**: GRUB may have issues saving boot state + +### Issue 10: Missing Kernel Boot Parameters (High) +**File**: `grub-config.sh` +**Problem**: No root=UUID or boot parameters configured +**Fix**: Added code to set root UUID and ro/quiet parameters +**Impact**: Kernel might not find root filesystem + +### Issue 11: Invalid Network Interface (Medium) +**File**: `setup-chroot.sh` +**Problem**: Network interface `eth99` won't exist +**Fix**: Changed to `eth0` +**Impact**: No network connectivity after boot + +### Issue 12: Wrong Kernel Package Name (Critical) +**File**: `setup-chroot.sh` +**Problem**: Package `linux-image-generic-x86` doesn't exist in Debian +**Fix**: Changed to `linux-image-amd64` +**Impact**: apt-get would fail with "package not found" + +### Issue 13: Interactive Password Command (Critical) +**File**: `setup-chroot.sh` +**Problem**: `passwd root` requires interactive input +**Fix**: Changed to `echo "root:toor" | chpasswd` +**Impact**: Build would hang indefinitely waiting for input + +### Issue 14: Missing Locale Generation (Medium) +**File**: `setup-chroot.sh` +**Problem**: No locale configuration +**Fix**: Added locale.gen configuration and locale-gen command +```bash +echo "en_US.UTF-8 UTF-8" > "$CHROOT_DIR/etc/locale.gen" +chroot "$CHROOT_DIR" locale-gen +echo "LANG=en_US.UTF-8" > "$CHROOT_DIR/etc/default/locale" +``` +**Impact**: Character encoding warnings, potential text display issues + +### Issue 15: Missing Timezone Configuration (Low) +**File**: `setup-chroot.sh` +**Problem**: No timezone set +**Fix**: Added `ln -sf /usr/share/zoneinfo/UTC /etc/localtime` +**Impact**: Wrong system time in logs + +### Issue 16: Missing DNS Configuration (High) +**File**: `setup-chroot.sh` +**Problem**: No `/etc/resolv.conf` +**Fix**: Created resolv.conf with nameservers +```bash +cat > "$CHROOT_DIR/etc/resolv.conf" <