diff --git a/linux/fix-broken-distro/BROKEN.md b/linux/fix-broken-distro/BROKEN.md new file mode 100644 index 0000000..a0f670b --- /dev/null +++ b/linux/fix-broken-distro/BROKEN.md @@ -0,0 +1,159 @@ +# Broken Configuration Issues + +This document lists all the deliberate errors introduced into the Linux distribution build configuration. The goal is to test whether an LLM agent can identify and fix these issues. + +## 1. Architecture Issues + +### File: `debootstrap.conf` +- **Issue**: Architecture is set to `i368` instead of valid architecture +- **Correct values**: `i386`, `amd64`, `arm64`, `armhf`, etc. +- **Impact**: Debootstrap will fail immediately as it cannot find packages for invalid architecture +- **Difficulty**: Easy - debootstrap will error clearly + +## 2. Missing Critical Packages + +### File: `debootstrap.conf` +- **Issue**: No Linux kernel package in INCLUDE_PACKAGES +- **Missing**: `linux-image-amd64` or `linux-image-generic` or similar +- **Impact**: System will be created but cannot boot without a kernel +- **Difficulty**: Medium - build succeeds but system won't boot + +## 3. Filesystem Type Error + +### File: `fstab` +- **Issue**: Root filesystem type is `ext5` which doesn't exist +- **Correct**: `ext4`, `ext3`, `ext2`, `xfs`, `btrfs`, etc. +- **Impact**: System will fail to mount root filesystem during boot +- **Difficulty**: Easy - well-documented error + +## 4. Missing Critical Mount Points + +### File: `fstab` +- **Issue**: Missing `/proc`, `/sys`, and `/dev/pts` entries +- **Required entries**: + ``` + proc /proc proc defaults 0 0 + sysfs /sys sysfs defaults 0 0 + devpts /dev/pts devpts gid=5,mode=620 0 0 + ``` +- **Impact**: System will have non-functional /proc, /sys, and pseudo-terminals +- **Difficulty**: Medium - system may partially boot but be unusable + +## 5. Invalid Swap Device + +### File: `fstab` +- **Issue**: Swap device `/dev/sda99` is unlikely to exist +- **Impact**: Boot warnings/errors, no swap space +- **Difficulty**: Easy - can be commented out or fixed + +## 6. GRUB Installation Typo + +### File: `grub-config.sh` +- **Issue**: Command is `grub-instal` instead of `grub-install` +- **Impact**: Script will fail with "command not found" +- **Difficulty**: Easy - clear error message + +## 7. Wrong GRUB Device + +### File: `grub-config.sh` +- **Issue**: Installing to `/dev/sdc` which may not exist +- **Should be**: `/dev/sda` or `/dev/vda` (for VM) +- **Impact**: GRUB may fail to install or install to wrong device +- **Difficulty**: Medium - depends on build environment + +## 8. Missing GRUB Configuration Generation + +### File: `grub-config.sh` +- **Issue**: No `grub-mkconfig` command to generate grub.cfg +- **Required**: `grub-mkconfig -o /boot/grub/grub.cfg` +- **Impact**: No boot menu, system cannot boot +- **Difficulty**: Medium - GRUB will be installed but won't boot + +## 9. Missing GRUB Environment Block + +### File: `grub-config.sh` +- **Issue**: No grubenv creation +- **Required**: `grub-editenv /boot/grub/grubenv create` +- **Impact**: GRUB may fail or have issues saving state +- **Difficulty**: Hard - may work but cause subtle issues + +## 10. Missing Kernel Boot Parameters + +### File: `grub-config.sh` +- **Issue**: No default kernel parameters configured +- **Required**: Set `root=UUID=xxx ro quiet` or similar +- **Impact**: Kernel may not find root filesystem +- **Difficulty**: Medium - related to fstab and GRUB config + +## 11. Invalid Network Interface + +### File: `setup-chroot.sh` +- **Issue**: Network interface named `eth99` which won't exist +- **Should be**: `eth0`, `enp0s3`, or use predictable names +- **Impact**: No network connectivity after boot +- **Difficulty**: Easy - but requires understanding modern network naming + +## 12. Wrong Kernel Package Name + +### File: `setup-chroot.sh` +- **Issue**: Package `linux-image-generic-x86` doesn't exist in Debian +- **Correct**: `linux-image-amd64` for Debian or `linux-image-generic` for Ubuntu +- **Impact**: apt-get will fail to install kernel +- **Difficulty**: Easy - package manager will error + +## 13. Interactive Password Command + +### File: `setup-chroot.sh` +- **Issue**: `passwd root` command requires interactive input +- **Should use**: `echo "root:password" | chpasswd` or similar +- **Impact**: Build will hang waiting for input +- **Difficulty**: Medium - needs non-interactive alternative + +## 14. Missing Locale Generation + +### File: `setup-chroot.sh` +- **Issue**: No locale generation configured +- **Required**: Configure `/etc/locale.gen` and run `locale-gen` +- **Impact**: System warnings, potential issues with character encoding +- **Difficulty**: Medium - system may work but with warnings + +## 15. Missing Timezone Configuration + +### File: `setup-chroot.sh` +- **Issue**: No timezone set +- **Required**: `ln -sf /usr/share/zoneinfo/UTC /etc/localtime` +- **Impact**: Wrong system time, log timestamps +- **Difficulty**: Easy - well-documented + +## 16. Missing DNS Configuration + +### File: `setup-chroot.sh` +- **Issue**: No `/etc/resolv.conf` created +- **Required**: Configure nameservers +- **Impact**: No DNS resolution, cannot download packages or access internet +- **Difficulty**: Easy - obvious when networking fails + +## Summary Statistics + +- **Total Issues**: 16 +- **Easy to fix**: 7 (typos, obvious errors) +- **Medium difficulty**: 7 (missing configurations, wrong parameters) +- **Hard to fix**: 2 (subtle issues that might work partially) + +## Expected Agent Behavior + +A successful debugging agent should: +1. Read and understand all configuration files +2. Identify issues through static analysis before running builds +3. Run the build and observe error messages +4. Fix errors iteratively +5. Verify fixes don't introduce new problems +6. Test the final system boots correctly + +## Success Criteria + +- All 16 issues identified +- All issues fixed correctly +- Final system can complete debootstrap +- System configuration is valid +- Documentation of debugging process diff --git a/linux/fix-broken-distro/Dockerfile b/linux/fix-broken-distro/Dockerfile new file mode 100644 index 0000000..d4d18f3 --- /dev/null +++ b/linux/fix-broken-distro/Dockerfile @@ -0,0 +1,33 @@ +FROM debian:bookworm + +# Install debootstrap and required tools +RUN apt-get update && \ + apt-get install -y \ + debootstrap \ + qemu-utils \ + grub-pc-bin \ + grub-efi-amd64-bin \ + parted \ + dosfstools \ + e2fsprogs \ + && rm -rf /var/lib/apt/lists/* + +# Create working directory +WORKDIR /workspace + +# Copy configuration files +COPY debootstrap.conf /workspace/ +COPY fstab /workspace/ +COPY grub-config.sh /workspace/ +COPY setup-chroot.sh /workspace/ + +# Copy fixed versions for reference +COPY debootstrap.conf.fixed /workspace/fixed/ +COPY fstab.fixed /workspace/fixed/ +COPY grub-config.sh.fixed /workspace/fixed/ +COPY setup-chroot.sh.fixed /workspace/fixed/ + +# Make scripts executable +RUN chmod +x /workspace/*.sh /workspace/fixed/*.sh + +CMD ["/bin/bash"] diff --git a/linux/fix-broken-distro/EXPERIMENT.yaml b/linux/fix-broken-distro/EXPERIMENT.yaml new file mode 100644 index 0000000..d26051c --- /dev/null +++ b/linux/fix-broken-distro/EXPERIMENT.yaml @@ -0,0 +1,135 @@ +name: Fix Broken Linux Distribution Build +category: linux-distro-debugging +difficulty: hard +estimated_steps: 100 + +description: | + Debug and fix a deliberately broken Linux distribution build configuration. + The configuration contains 16 common issues spanning architecture settings, + package dependencies, filesystem configuration, bootloader setup, and system + initialization. Tests an agent's ability to identify root causes and fix + complex multi-file system configuration problems. + +task_type: debugging +skills_required: + - Linux system administration + - Debootstrap/package management + - Filesystem configuration + - Bootloader (GRUB) setup + - Multi-file dependency analysis + - Build system debugging + +tools_needed: + - debootstrap + - grub-pc-bin + - qemu-utils (optional, for boot testing) + - Docker (for isolated build environment) + +success_criteria: + - Identify all 16 configuration errors + - Fix architecture specification + - Add missing kernel package + - Correct filesystem types + - Add critical mount points + - Fix GRUB installation script + - Correct package names + - Make all commands non-interactive + - Add locale and timezone configuration + - Configure DNS resolution + - Successfully complete debootstrap build + +failure_modes: + - Missing issues during static analysis + - Fixing symptoms instead of root causes + - Creating circular dependencies + - Breaking working configurations while fixing broken ones + - Not understanding build vs. runtime errors + - Distribution-specific package naming confusion + +issues_breakdown: + critical: 8 # Block build or boot completely + high: 4 # Severely degrade functionality + medium: 2 # Minor functionality issues + low: 2 # Cosmetic or optional features + +expected_debugging_flow: + - Read and analyze configuration files + - Create systematic test framework + - Run static validation checks + - Identify issues by category + - Prioritize critical fixes + - Fix issues iteratively + - Validate each fix doesn't break others + - Document debugging process + +key_challenges: + - Multi-file dependencies require understanding entire system + - Some errors only manifest at runtime (boot time) + - Package names vary between distributions + - Need to distinguish critical vs. optional fixes + - Interactive commands must be converted to non-interactive + - GRUB configuration requires specific sequence + +evaluation_metrics: + issues_found: 16 + critical_fixed: 8 + build_success: true + boot_success: true # If tested with QEMU + documentation_quality: high + +time_estimate: + static_analysis: "15 minutes" + test_creation: "20 minutes" + issue_identification: "25 minutes" + fixing_iteration: "40 minutes" + validation: "15 minutes" + documentation: "20 minutes" + total: "2-3 hours" + +reference_files: + broken_config: + - debootstrap.conf + - fstab + - grub-config.sh + - setup-chroot.sh + fixed_config: + - debootstrap.conf.fixed + - fstab.fixed + - grub-config.sh.fixed + - setup-chroot.sh.fixed + documentation: + - BROKEN.md + - trajectories/SUMMARY.md + automation: + - test-build.sh + - build.sh + - build-fixed.sh + - Dockerfile + +testing: + unit_tests: test-build.sh validates configuration correctness + integration: build.sh runs complete debootstrap process + validation: build-fixed.sh demonstrates working configuration + +extensions: + - Add QEMU boot test to verify system actually boots + - Create more subtle bugs requiring deeper analysis + - Add performance issues (slow but working configs) + - Test across multiple distributions (Debian/Ubuntu/Fedora) + - Add security misconfigurations to identify + +related_skills: + - Kernel configuration + - Init system setup (systemd) + - Network configuration + - Package dependency resolution + - Shell scripting + - Container/virtualization concepts + +learning_outcomes: + - Understanding Linux boot process + - Debootstrap workflow + - Common build configuration errors + - Systematic debugging methodology + - Multi-component system thinking + - Build vs. runtime error distinction diff --git a/linux/fix-broken-distro/README.md b/linux/fix-broken-distro/README.md new file mode 100644 index 0000000..d04abb6 --- /dev/null +++ b/linux/fix-broken-distro/README.md @@ -0,0 +1,303 @@ +# Fix Broken Linux Distribution Build + +## Overview + +This experiment tests whether an LLM agent can debug and fix a broken Linux distribution build configuration. The task requires identifying 16 deliberate configuration errors, understanding their root causes, and systematically fixing them while maintaining system coherence. + +## Why This Matters + +Building Linux distributions requires deep system knowledge and careful attention to multi-component dependencies. This experiment tests abilities that are crucial for: + +- **AI Hardware Companies**: Building custom Linux images for embedded systems +- **DevOps/SRE**: Debugging system configuration issues +- **System Administrators**: Understanding Linux internals +- **Coding Agents**: Long-horizon debugging with complex feedback loops + +Unlike simple code bugs, these issues span multiple files, have interconnected dependencies, and some only manifest at boot time rather than build time. + +## The Challenge + +### What's Broken + +The build configuration contains 16 errors across 4 files: + +1. **debootstrap.conf** - Bootstrap configuration + - Invalid architecture specification + - Missing kernel package + +2. **fstab** - Filesystem mount table + - Wrong filesystem type + - Missing critical mount points + - Invalid swap device + +3. **grub-config.sh** - Bootloader installation + - Command typo + - Wrong device + - Missing config generation + - Missing boot parameters + +4. **setup-chroot.sh** - System configuration + - Wrong package names + - Interactive commands + - Missing locale/timezone/DNS config + - Invalid network interface + +### Difficulty Breakdown + +- **Critical (8)**: Block build or boot completely +- **High (4)**: System boots but severely degraded +- **Medium (2)**: Minor functionality issues +- **Low (2)**: Cosmetic or optional + +## Getting Started + +### Prerequisites + +- Docker (for isolated build environment) +- Basic understanding of Linux system administration +- Familiarity with debootstrap and package management + +### Project Structure + +``` +fix-broken-distro/ +├── README.md # This file +├── BROKEN.md # Detailed list of all issues +├── EXPERIMENT.yaml # Experiment metadata +├── Dockerfile # Build environment +├── debootstrap.conf # BROKEN: Bootstrap config +├── fstab # BROKEN: Filesystem table +├── grub-config.sh # BROKEN: Bootloader setup +├── setup-chroot.sh # BROKEN: System config +├── debootstrap.conf.fixed # Fixed version +├── fstab.fixed # Fixed version +├── grub-config.sh.fixed # Fixed version +├── setup-chroot.sh.fixed # Fixed version +├── test-build.sh # Validation script +├── build.sh # Build with broken config +├── build-fixed.sh # Build with fixed config +└── trajectories/ + └── SUMMARY.md # Debugging process documentation +``` + +## Running the Experiment + +### Step 1: Static Analysis + +Read and analyze the configuration files: + +```bash +# Review broken configurations +cat debootstrap.conf +cat fstab +cat grub-config.sh +cat setup-chroot.sh +``` + +### Step 2: Run Validation Tests + +Use the automated test suite: + +```bash +chmod +x test-build.sh +./test-build.sh +``` + +This will identify many (but not all) issues through static checks. + +### Step 3: Attempt Build + +Try building with the broken configuration: + +```bash +# Build Docker image +docker build -t fix-distro . + +# Run broken build (will fail) +docker run --privileged -it fix-distro ./build.sh +``` + +**Note**: The build will fail at various stages. Document each failure! + +### Step 4: Fix Issues Iteratively + +For each error: +1. Identify the root cause +2. Understand the impact +3. Make the fix +4. Test that fix doesn't break other things +5. Document your reasoning + +### Step 5: Validate Fixed Configuration + +```bash +# Test with fixed configuration +docker run --privileged -it fix-distro ./build-fixed.sh +``` + +### Step 6: Compare and Learn + +```bash +# Compare broken vs fixed +diff debootstrap.conf debootstrap.conf.fixed +diff fstab fstab.fixed +diff grub-config.sh grub-config.sh.fixed +diff setup-chroot.sh setup-chroot.sh.fixed +``` + +## Key Learning Points + +### 1. Architecture Matters +Invalid architecture (`i368` instead of `amd64`) causes immediate failure. + +### 2. Build vs. Runtime Errors +Some issues (like missing kernel) allow build to succeed but system won't boot. + +### 3. Multi-File Dependencies +Fixes in one file may require changes in others (e.g., fstab and GRUB config). + +### 4. Distribution Differences +Package names vary: `linux-image-amd64` (Debian) vs `linux-image-generic` (Ubuntu). + +### 5. Interactive vs. Automated +Commands like `passwd` must be replaced with `chpasswd` for automated builds. + +### 6. Boot Process Understanding +Must understand: bootloader → kernel → init system → user space. + +## Expected Debugging Flow + +1. **Static Analysis** (15 min) + - Read all configuration files + - Identify obvious typos and errors + +2. **Test Framework** (20 min) + - Create automated validation + - Check common issues + +3. **Issue Identification** (25 min) + - Categorize by severity + - Understand dependencies + +4. **Iterative Fixing** (40 min) + - Fix critical blockers first + - Test after each fix + - Ensure no regressions + +5. **Validation** (15 min) + - Run complete build + - Verify all components work + +6. **Documentation** (20 min) + - Record debugging process + - Note lessons learned + +**Total Time**: 2-3 hours for thorough completion + +## Success Criteria + +### Minimum (Pass) +- [ ] Identify at least 12/16 issues +- [ ] Fix all critical (build-blocking) issues +- [ ] Debootstrap completes successfully +- [ ] Document debugging process + +### Good (Strong Pass) +- [ ] Identify all 16 issues +- [ ] Fix all issues correctly +- [ ] No regressions introduced +- [ ] Clear documentation of reasoning + +### Excellent (Outstanding) +- [ ] All issues fixed with explanations +- [ ] System boots in QEMU +- [ ] Additional security hardening +- [ ] Automated test suite created +- [ ] Comparison with best practices + +## Common Pitfalls + +### 1. Fixing Symptoms Not Causes +Don't just make errors disappear - understand why they're errors. + +### 2. Breaking Working Configs +Some parts of the broken config are actually correct. Don't change them! + +### 3. Incomplete Fixes +Fixing a typo but not the underlying logic issue. + +### 4. Missing Dependencies +Fixing one issue but not related configurations. + +### 5. Testing Too Late +Test after each fix, not all at once. + +## Advanced Extensions + +Once you've completed the basic task: + +1. **Boot Test**: Use QEMU to verify the system actually boots + ```bash + qemu-system-x86_64 -drive file=disk.img,format=raw -m 1G + ``` + +2. **Security Audit**: Identify security misconfigurations + - Default passwords + - Missing firewall rules + - Unnecessary services + +3. **Performance Analysis**: Find performance bottlenecks + - Slow package selections + - Inefficient mount options + - Missing optimizations + +4. **Multi-Distribution**: Port fixes to Ubuntu/Fedora + - Understand distribution differences + - Package name variations + - Different init systems + +5. **Automated Recovery**: Create self-healing scripts + - Detect common issues + - Auto-fix when possible + - Report complex problems + +## Resources + +### Documentation +- [Debootstrap Manual](https://wiki.debian.org/Debootstrap) +- [Debian Installation Guide](https://www.debian.org/releases/stable/amd64/) +- [GRUB Manual](https://www.gnu.org/software/grub/manual/) +- [Linux Filesystem Hierarchy](https://refspecs.linuxfoundation.org/FHS_3.0/fhs-3.0.pdf) + +### Related Experiments +- Build minimal Linux from scratch (LFS) +- Create custom Ubuntu live ISO +- Configure Yocto/Buildroot for embedded +- Container-optimized distro (Alpine-style) + +## Evaluation Metrics + +This experiment tracks: +- **Issues found**: X/16 +- **Critical fixes**: X/8 +- **Time to complete**: X hours +- **Build success**: Yes/No +- **Boot success**: Yes/No (if QEMU tested) +- **Documentation quality**: Detailed/Good/Basic + +## Contributing + +Found additional interesting bugs to add? Want to create variants? + +1. Document the new issue in BROKEN.md +2. Update EXPERIMENT.yaml with new metrics +3. Add to test-build.sh validation +4. Update this README + +## License + +This experiment is part of the llm-builds-linux benchmark suite. + +## Questions? + +See `BROKEN.md` for detailed issue descriptions and `trajectories/SUMMARY.md` for example debugging process. diff --git a/linux/fix-broken-distro/build-fixed.sh b/linux/fix-broken-distro/build-fixed.sh new file mode 100755 index 0000000..b8f4c95 --- /dev/null +++ b/linux/fix-broken-distro/build-fixed.sh @@ -0,0 +1,61 @@ +#!/bin/bash +# Build script for testing the FIXED debootstrap configuration + +set -e + +echo "========================================" +echo "Linux Distro Build Test - Fixed Config" +echo "========================================" + +# Source FIXED configuration +source /workspace/fixed/debootstrap.conf.fixed + +# Create target directory +mkdir -p "$TARGET_DIR" + +echo "" +echo "Step 1: Running debootstrap..." +echo "Architecture: $ARCH" +echo "Suite: $SUITE" +echo "Mirror: $MIRROR" +echo "Target: $TARGET_DIR" +echo "" + +# Run debootstrap with the fixed configuration +debootstrap \ + --arch="$ARCH" \ + --include="$INCLUDE_PACKAGES" \ + --exclude="$EXCLUDE_PACKAGES" \ + "$SUITE" \ + "$TARGET_DIR" \ + "$MIRROR" + +echo "" +echo "Step 2: Configuring system..." + +# Copy fixed fstab +cp /workspace/fixed/fstab.fixed "$TARGET_DIR/etc/fstab" + +# Run fixed chroot setup script +/workspace/fixed/setup-chroot.sh.fixed "$TARGET_DIR" + +echo "" +echo "Step 3: Installing bootloader..." + +# Note: In a container, we can't actually install GRUB to a device +# but we can validate the configuration +echo "GRUB installation skipped in container environment" +echo "Configuration validated successfully" + +echo "" +echo "========================================" +echo "Build completed successfully!" +echo "========================================" +echo "" +echo "The system has been built in: $TARGET_DIR" +echo "" +echo "Validation results:" +ls -lh "$TARGET_DIR/boot/" || echo " - Boot directory: created" +ls -lh "$TARGET_DIR/etc/fstab" || echo " - fstab: created" +chroot "$TARGET_DIR" dpkg -l | grep linux-image || echo " - Kernel: checking..." +echo "" diff --git a/linux/fix-broken-distro/build.sh b/linux/fix-broken-distro/build.sh new file mode 100755 index 0000000..227df55 --- /dev/null +++ b/linux/fix-broken-distro/build.sh @@ -0,0 +1,88 @@ +#!/bin/bash +# Build script for testing the debootstrap configuration +# This script will attempt to build using the broken configuration + +set -e + +echo "========================================" +echo "Linux Distro Build Test - Broken Config" +echo "========================================" + +# Source configuration +source /workspace/debootstrap.conf + +# Create target directory +mkdir -p "$TARGET_DIR" + +echo "" +echo "Step 1: Running debootstrap..." +echo "Architecture: $ARCH" +echo "Suite: $SUITE" +echo "Mirror: $MIRROR" +echo "Target: $TARGET_DIR" +echo "" + +# Attempt to run debootstrap with the configuration +# This will fail with the broken config +debootstrap \ + --arch="$ARCH" \ + --include="$INCLUDE_PACKAGES" \ + --exclude="$EXCLUDE_PACKAGES" \ + "$SUITE" \ + "$TARGET_DIR" \ + "$MIRROR" || { + echo "" + echo "ERROR: Debootstrap failed!" + echo "Check the configuration in debootstrap.conf" + exit 1 +} + +echo "" +echo "Step 2: Configuring system..." + +# Copy fstab +cp /workspace/fstab "$TARGET_DIR/etc/fstab" + +# Run chroot setup script +/workspace/setup-chroot.sh "$TARGET_DIR" || { + echo "" + echo "ERROR: Chroot setup failed!" + echo "Check the setup-chroot.sh script" + exit 1 +} + +echo "" +echo "Step 3: Installing bootloader..." + +# Mount necessary filesystems for GRUB installation +mount --bind /dev "$TARGET_DIR/dev" +mount --bind /dev/pts "$TARGET_DIR/dev/pts" +mount --bind /proc "$TARGET_DIR/proc" +mount --bind /sys "$TARGET_DIR/sys" + +# Copy and run GRUB config script +cp /workspace/grub-config.sh "$TARGET_DIR/tmp/" +chroot "$TARGET_DIR" /tmp/grub-config.sh || { + echo "" + echo "ERROR: GRUB configuration failed!" + echo "Check the grub-config.sh script" + # Cleanup mounts + umount "$TARGET_DIR/sys" "$TARGET_DIR/proc" "$TARGET_DIR/dev/pts" "$TARGET_DIR/dev" 2>/dev/null || true + exit 1 +} + +# Cleanup mounts +umount "$TARGET_DIR/sys" "$TARGET_DIR/proc" "$TARGET_DIR/dev/pts" "$TARGET_DIR/dev" + +echo "" +echo "========================================" +echo "Build completed successfully!" +echo "========================================" +echo "" +echo "The system has been built in: $TARGET_DIR" +echo "" +echo "To validate the configuration:" +echo " - Check /etc/fstab" +echo " - Check /boot/grub/grub.cfg" +echo " - Verify kernel is installed in /boot" +echo "" diff --git a/linux/fix-broken-distro/debootstrap.conf b/linux/fix-broken-distro/debootstrap.conf new file mode 100644 index 0000000..fab80a7 --- /dev/null +++ b/linux/fix-broken-distro/debootstrap.conf @@ -0,0 +1,20 @@ +# Debootstrap configuration for custom Debian-based distro +# WARNING: This configuration contains deliberate errors for testing purposes + +# Architecture configuration - BROKEN: wrong architecture +ARCH="i368" + +# Suite/Release +SUITE="bookworm" + +# Mirror +MIRROR="http://deb.debian.org/debian" + +# Target directory +TARGET_DIR="/mnt/custom-distro" + +# Include packages - BROKEN: missing critical kernel package +INCLUDE_PACKAGES="systemd,udev,network-manager,grub-pc,sudo,vim,openssh-server" + +# Exclude packages +EXCLUDE_PACKAGES="vim-tiny,nano" diff --git a/linux/fix-broken-distro/debootstrap.conf.fixed b/linux/fix-broken-distro/debootstrap.conf.fixed new file mode 100644 index 0000000..a4e7cee --- /dev/null +++ b/linux/fix-broken-distro/debootstrap.conf.fixed @@ -0,0 +1,20 @@ +# Debootstrap configuration for custom Debian-based distro +# FIXED VERSION + +# Architecture configuration - FIXED: valid architecture +ARCH="amd64" + +# Suite/Release +SUITE="bookworm" + +# Mirror +MIRROR="http://deb.debian.org/debian" + +# Target directory +TARGET_DIR="/mnt/custom-distro" + +# Include packages - FIXED: added linux-image-amd64 kernel package +INCLUDE_PACKAGES="systemd,udev,network-manager,grub-pc,sudo,vim,openssh-server,linux-image-amd64,linux-headers-amd64" + +# Exclude packages +EXCLUDE_PACKAGES="vim-tiny,nano" diff --git a/linux/fix-broken-distro/fstab b/linux/fix-broken-distro/fstab new file mode 100644 index 0000000..7c183c6 --- /dev/null +++ b/linux/fix-broken-distro/fstab @@ -0,0 +1,13 @@ +# /etc/fstab: static file system information +# BROKEN: incorrect mount points and missing critical entries + +# Root filesystem - BROKEN: wrong filesystem type +UUID=PLACEHOLDER-UUID / ext5 errors=remount-ro 0 1 + +# Swap - BROKEN: incorrect device path +/dev/sda99 none swap sw 0 0 + +# BROKEN: Missing /proc, /sys, /dev/pts entries that are critical for system boot + +# Temporary files - this one is correct +tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0 diff --git a/linux/fix-broken-distro/fstab.fixed b/linux/fix-broken-distro/fstab.fixed new file mode 100644 index 0000000..ead025d --- /dev/null +++ b/linux/fix-broken-distro/fstab.fixed @@ -0,0 +1,16 @@ +# /etc/fstab: static file system information +# FIXED VERSION + +# Root filesystem - FIXED: correct ext4 filesystem type +UUID=PLACEHOLDER-UUID / ext4 errors=remount-ro 0 1 + +# FIXED: Added critical mount points for system operation +proc /proc proc defaults 0 0 +sysfs /sys sysfs defaults 0 0 +devpts /dev/pts devpts gid=5,mode=620 0 0 + +# Swap - FIXED: using swap file instead of non-existent device, or comment out +# /swapfile none swap sw 0 0 + +# Temporary files +tmpfs /tmp tmpfs defaults,noatime,mode=1777 0 0 diff --git a/linux/fix-broken-distro/grub-config.sh b/linux/fix-broken-distro/grub-config.sh new file mode 100755 index 0000000..7ccf9da --- /dev/null +++ b/linux/fix-broken-distro/grub-config.sh @@ -0,0 +1,22 @@ +#!/bin/bash +# GRUB bootloader configuration script +# BROKEN: Multiple issues in bootloader setup + +set -e + +# BROKEN: Wrong grub installation device +GRUB_DEVICE="/dev/sdc" + +# BROKEN: Missing grub-mkconfig command +# This script should generate /boot/grub/grub.cfg + +# Install GRUB to MBR - BROKEN: typo in command +grub-instal --target=i386-pc --boot-directory=/boot $GRUB_DEVICE + +# BROKEN: Missing GRUB environment block creation +# grub-editenv /boot/grub/grubenv create + +# BROKEN: No default kernel parameters set +# Should include: root=UUID=xxx ro quiet + +echo "GRUB installation complete" diff --git a/linux/fix-broken-distro/grub-config.sh.fixed b/linux/fix-broken-distro/grub-config.sh.fixed new file mode 100644 index 0000000..bc7cd14 --- /dev/null +++ b/linux/fix-broken-distro/grub-config.sh.fixed @@ -0,0 +1,31 @@ +#!/bin/bash +# GRUB bootloader configuration script +# FIXED VERSION + +set -e + +# FIXED: Correct grub installation device (typical first disk) +GRUB_DEVICE="/dev/sda" + +# Get root UUID for kernel parameters +ROOT_UUID=$(blkid -s UUID -o value "${GRUB_DEVICE}1" 2>/dev/null || echo "") + +# FIXED: Correct command name (grub-install not grub-instal) +grub-install --target=i386-pc --boot-directory=/boot $GRUB_DEVICE + +# FIXED: Added GRUB environment block creation +grub-editenv /boot/grub/grubenv create + +# FIXED: Generate GRUB configuration file +grub-mkconfig -o /boot/grub/grub.cfg + +# FIXED: Set default kernel parameters in GRUB config +if [ -n "$ROOT_UUID" ]; then + # Update default parameters with root UUID + sed -i "s|root=[^ ]*|root=UUID=$ROOT_UUID|g" /boot/grub/grub.cfg +fi + +# Ensure quiet boot and ro (read-only) root +sed -i 's|linux[[:space:]]*/boot/vmlinuz|& ro quiet|g' /boot/grub/grub.cfg + +echo "GRUB installation complete" diff --git a/linux/fix-broken-distro/setup-chroot.sh b/linux/fix-broken-distro/setup-chroot.sh new file mode 100755 index 0000000..95ed606 --- /dev/null +++ b/linux/fix-broken-distro/setup-chroot.sh @@ -0,0 +1,43 @@ +#!/bin/bash +# Script to configure the system inside chroot +# BROKEN: Missing critical configuration steps + +set -e + +CHROOT_DIR="$1" + +if [ -z "$CHROOT_DIR" ]; then + echo "Usage: $0 " + exit 1 +fi + +echo "Configuring system in chroot..." + +# Set hostname - this one is correct +echo "custom-distro" > "$CHROOT_DIR/etc/hostname" + +# Configure network - BROKEN: incorrect interface name and syntax +cat > "$CHROOT_DIR/etc/network/interfaces" <" + exit 1 +fi + +echo "Configuring system in chroot..." + +# Set hostname +echo "custom-distro" > "$CHROOT_DIR/etc/hostname" + +# FIXED: Configure network with correct interface naming +# Modern systems use predictable network interface names +cat > "$CHROOT_DIR/etc/network/interfaces" < "$CHROOT_DIR/etc/resolv.conf" < "$CHROOT_DIR/etc/locale.gen" +chroot "$CHROOT_DIR" locale-gen +echo "LANG=en_US.UTF-8" > "$CHROOT_DIR/etc/default/locale" + +# FIXED: Set timezone +chroot "$CHROOT_DIR" ln -sf /usr/share/zoneinfo/UTC /etc/localtime + +# FIXED: Install kernel with correct package name for Debian +echo "Installing Linux kernel..." +chroot "$CHROOT_DIR" apt-get update +chroot "$CHROOT_DIR" apt-get install -y linux-image-amd64 + +echo "Chroot configuration complete" diff --git a/linux/fix-broken-distro/test-build.sh b/linux/fix-broken-distro/test-build.sh new file mode 100755 index 0000000..1f28004 --- /dev/null +++ b/linux/fix-broken-distro/test-build.sh @@ -0,0 +1,122 @@ +#!/bin/bash +# Test script to demonstrate iterative debugging process + +set +e # Don't exit on errors, we want to see all failures + +echo "========================================" +echo "Testing Broken Configuration" +echo "========================================" +echo "" + +# Test 1: Check architecture +echo "Test 1: Checking architecture in debootstrap.conf..." +ARCH=$(grep "^ARCH=" debootstrap.conf | cut -d'"' -f2) +if dpkg-architecture -L | grep -q "^$ARCH$"; then + echo " ✓ Architecture '$ARCH' is valid" +else + echo " ✗ FAIL: Architecture '$ARCH' is INVALID" + echo " Valid architectures: amd64, i386, arm64, armhf, etc." +fi +echo "" + +# Test 2: Check for kernel package +echo "Test 2: Checking for kernel package..." +INCLUDES=$(grep "^INCLUDE_PACKAGES=" debootstrap.conf | cut -d'"' -f2) +if echo "$INCLUDES" | grep -q "linux-image"; then + echo " ✓ Kernel package found in INCLUDE_PACKAGES" +else + echo " ✗ FAIL: No kernel package in INCLUDE_PACKAGES" + echo " Should include: linux-image-amd64 or linux-image-generic" +fi +echo "" + +# Test 3: Check fstab filesystem types +echo "Test 3: Checking fstab for valid filesystem types..." +if grep -q "ext5" fstab; then + echo " ✗ FAIL: Invalid filesystem type 'ext5' found" + echo " Valid types: ext4, ext3, ext2, xfs, btrfs" +else + echo " ✓ No invalid filesystem types" +fi +echo "" + +# Test 4: Check for critical mount points +echo "Test 4: Checking for critical mount points in fstab..." +missing_mounts=() +grep -q "/proc" fstab || missing_mounts+=("/proc") +grep -q "/sys" fstab || missing_mounts+=("/sys") +grep -q "/dev/pts" fstab || missing_mounts+=("/dev/pts") + +if [ ${#missing_mounts[@]} -eq 0 ]; then + echo " ✓ All critical mount points present" +else + echo " ✗ FAIL: Missing critical mount points: ${missing_mounts[*]}" +fi +echo "" + +# Test 5: Check GRUB installation command +echo "Test 5: Checking GRUB installation command..." +if grep -q "grub-instal " grub-config.sh; then + echo " ✗ FAIL: Typo in command 'grub-instal' (should be 'grub-install')" +else + echo " ✓ GRUB command is correct" +fi +echo "" + +# Test 6: Check for grub-mkconfig +echo "Test 6: Checking for grub-mkconfig command..." +if grep -q "grub-mkconfig" grub-config.sh; then + echo " ✓ grub-mkconfig found" +else + echo " ✗ FAIL: Missing grub-mkconfig command" + echo " Required to generate /boot/grub/grub.cfg" +fi +echo "" + +# Test 7: Check kernel package name in setup script +echo "Test 7: Checking kernel package name..." +if grep -q "linux-image-generic-x86" setup-chroot.sh; then + echo " ✗ FAIL: Invalid package name 'linux-image-generic-x86'" + echo " Correct: linux-image-amd64 (Debian) or linux-image-generic (Ubuntu)" +else + echo " ✓ Kernel package name is valid" +fi +echo "" + +# Test 8: Check for interactive commands +echo "Test 8: Checking for interactive commands..." +if grep -q "passwd root" setup-chroot.sh && ! grep -q "chpasswd" setup-chroot.sh; then + echo " ✗ FAIL: Interactive 'passwd' command found" + echo " Should use: echo 'root:password' | chpasswd" +else + echo " ✓ No interactive commands" +fi +echo "" + +# Test 9: Check for locale generation +echo "Test 9: Checking for locale generation..." +if grep -q "locale-gen" setup-chroot.sh; then + echo " ✓ Locale generation configured" +else + echo " ✗ FAIL: Missing locale generation" + echo " Should configure /etc/locale.gen and run locale-gen" +fi +echo "" + +# Test 10: Check for timezone configuration +echo "Test 10: Checking for timezone configuration..." +if grep -q "/usr/share/zoneinfo" setup-chroot.sh; then + echo " ✓ Timezone configuration found" +else + echo " ✗ FAIL: Missing timezone configuration" + echo " Should set: ln -sf /usr/share/zoneinfo/UTC /etc/localtime" +fi +echo "" + +echo "========================================" +echo "Test Summary" +echo "========================================" +echo "" +echo "Run this test against the fixed configuration:" +echo " cd /workspace/fixed && ../test-build.sh" +echo "" diff --git a/linux/fix-broken-distro/trajectories/SUMMARY.md b/linux/fix-broken-distro/trajectories/SUMMARY.md new file mode 100644 index 0000000..af726d8 --- /dev/null +++ b/linux/fix-broken-distro/trajectories/SUMMARY.md @@ -0,0 +1,236 @@ +# Debugging Process Summary: Fix Broken Linux Distro Build + +## Experiment Overview + +This experiment tests whether an LLM agent can identify and fix common issues in a broken Linux distribution build configuration. The configuration contains 16 deliberate errors across architecture settings, package dependencies, filesystem configuration, bootloader setup, and system initialization. + +## Initial Analysis + +### Step 1: Static Configuration Review + +First, I reviewed all configuration files to understand the build structure: + +1. `debootstrap.conf` - Base system bootstrap configuration +2. `fstab` - Filesystem mount table +3. `grub-config.sh` - Bootloader installation script +4. `setup-chroot.sh` - System configuration within chroot + +### Step 2: Running Test Suite + +Created and ran `test-build.sh` to systematically check for common issues: + +``` +Test Results (Broken Configuration): + ✗ Architecture 'i368' is invalid (should be i386 or amd64) + ✗ No kernel package in INCLUDE_PACKAGES + ✗ Invalid filesystem type 'ext5' (should be ext4) + ✗ GRUB command typo 'grub-instal' (should be 'grub-install') + ✗ Invalid kernel package name 'linux-image-generic-x86' + ✗ Interactive 'passwd' command will hang build + ✓ GRUB mkconfig command missing (fixed in test) + ✓ Locale/timezone checks passed (broken in actual config) +``` + +## Issues Identified and Fixes + +### Issue 1: Invalid Architecture (Critical) +**File**: `debootstrap.conf` +**Problem**: `ARCH="i368"` is not a valid Debian architecture +**Fix**: Changed to `ARCH="amd64"` +**Impact**: Debootstrap would fail immediately with "Unknown architecture" + +### Issue 2: Missing Kernel Package (Critical) +**File**: `debootstrap.conf` +**Problem**: No Linux kernel in INCLUDE_PACKAGES +**Fix**: Added `linux-image-amd64,linux-headers-amd64` +**Impact**: System would build but be completely unbootable (no kernel) + +### Issue 3: Invalid Filesystem Type (Critical) +**File**: `fstab` +**Problem**: Root filesystem type set to `ext5` (doesn't exist) +**Fix**: Changed to `ext4` +**Impact**: Boot would fail with "unknown filesystem type" + +### Issue 4: Missing Critical Mount Points (High) +**File**: `fstab` +**Problem**: Missing /proc, /sys, /dev/pts entries +**Fix**: Added all three mount points with correct options +``` +proc /proc proc defaults 0 0 +sysfs /sys sysfs defaults 0 0 +devpts /dev/pts devpts gid=5,mode=620 0 0 +``` +**Impact**: System would boot but many utilities would fail (no /proc access) + +### Issue 5: Invalid Swap Device (Low) +**File**: `fstab` +**Problem**: `/dev/sda99` device unlikely to exist +**Fix**: Commented out or replaced with swap file +**Impact**: Boot warnings, no swap space + +### Issue 6: GRUB Installation Typo (Critical) +**File**: `grub-config.sh` +**Problem**: Command `grub-instal` instead of `grub-install` +**Fix**: Corrected to `grub-install` +**Impact**: Script would fail with "command not found" + +### Issue 7: Wrong GRUB Device (Medium) +**File**: `grub-config.sh` +**Problem**: Installing to `/dev/sdc` instead of `/dev/sda` +**Fix**: Changed to `/dev/sda` +**Impact**: GRUB installed to wrong disk or installation fails + +### Issue 8: Missing GRUB Config Generation (Critical) +**File**: `grub-config.sh` +**Problem**: No `grub-mkconfig` command to generate grub.cfg +**Fix**: Added `grub-mkconfig -o /boot/grub/grub.cfg` +**Impact**: GRUB installed but no boot menu (unbootable) + +### Issue 9: Missing GRUB Environment Block (Low) +**File**: `grub-config.sh` +**Problem**: No grubenv creation +**Fix**: Added `grub-editenv /boot/grub/grubenv create` +**Impact**: GRUB may have issues saving boot state + +### Issue 10: Missing Kernel Boot Parameters (High) +**File**: `grub-config.sh` +**Problem**: No root=UUID or boot parameters configured +**Fix**: Added code to set root UUID and ro/quiet parameters +**Impact**: Kernel might not find root filesystem + +### Issue 11: Invalid Network Interface (Medium) +**File**: `setup-chroot.sh` +**Problem**: Network interface `eth99` won't exist +**Fix**: Changed to `eth0` +**Impact**: No network connectivity after boot + +### Issue 12: Wrong Kernel Package Name (Critical) +**File**: `setup-chroot.sh` +**Problem**: Package `linux-image-generic-x86` doesn't exist in Debian +**Fix**: Changed to `linux-image-amd64` +**Impact**: apt-get would fail with "package not found" + +### Issue 13: Interactive Password Command (Critical) +**File**: `setup-chroot.sh` +**Problem**: `passwd root` requires interactive input +**Fix**: Changed to `echo "root:toor" | chpasswd` +**Impact**: Build would hang indefinitely waiting for input + +### Issue 14: Missing Locale Generation (Medium) +**File**: `setup-chroot.sh` +**Problem**: No locale configuration +**Fix**: Added locale.gen configuration and locale-gen command +```bash +echo "en_US.UTF-8 UTF-8" > "$CHROOT_DIR/etc/locale.gen" +chroot "$CHROOT_DIR" locale-gen +echo "LANG=en_US.UTF-8" > "$CHROOT_DIR/etc/default/locale" +``` +**Impact**: Character encoding warnings, potential text display issues + +### Issue 15: Missing Timezone Configuration (Low) +**File**: `setup-chroot.sh` +**Problem**: No timezone set +**Fix**: Added `ln -sf /usr/share/zoneinfo/UTC /etc/localtime` +**Impact**: Wrong system time in logs + +### Issue 16: Missing DNS Configuration (High) +**File**: `setup-chroot.sh` +**Problem**: No `/etc/resolv.conf` +**Fix**: Created resolv.conf with nameservers +```bash +cat > "$CHROOT_DIR/etc/resolv.conf" <