Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
45 commits
Select commit Hold shift + click to select a range
d7c0cd8
Make yeast mito tRNA ref
jayhesselberth Feb 25, 2025
0b8e677
Remove linters for now
jayhesselberth Mar 8, 2025
7be3485
ruff format
jayhesselberth Mar 8, 2025
e553083
Download and install dorado and model (#56)
jayhesselberth Mar 13, 2025
dafa2c0
Call modified bases
jayhesselberth Mar 13, 2025
c62e4a9
Update model download strategy
jayhesselberth Mar 13, 2025
8020289
Install modkit (#59)
jayhesselberth Mar 13, 2025
642d452
Use cli for cargo install
jayhesselberth Mar 13, 2025
2bdfcc8
Install binary modkit on linux x86
jayhesselberth Mar 14, 2025
0a1b719
use bare version number
jayhesselberth Mar 15, 2025
d287cdf
Eliminate support of FAST5 files (#43)
jayhesselberth Mar 15, 2025
b8823ca
Remove optional bam inputs
jayhesselberth Mar 15, 2025
d45e40a
Increase threads for pod5 merge
jayhesselberth Mar 15, 2025
59c9af6
Add modkit rules
jayhesselberth Mar 16, 2025
e2f8399
Simplify modkit install
jayhesselberth Mar 16, 2025
33154b4
add bedtools
jayhesselberth Mar 16, 2025
4e848b6
update rule
jayhesselberth Mar 16, 2025
e4f006c
modkit testing
jayhesselberth Mar 16, 2025
9f0bfc9
Merge branch 'main' of ssh://github.com/rnabioco/aa-tRNA-seq-pipeline
jayhesselberth Mar 16, 2025
fb135b6
Reorganize output directory
jayhesselberth Mar 17, 2025
bc2a5d8
Rename charging tags during transfer
jayhesselberth Mar 17, 2025
2664321
Update tag for charging table
jayhesselberth Mar 17, 2025
24faab9
Update readme
jayhesselberth Mar 17, 2025
4dac8fc
Reduce dorado verbosity
jayhesselberth Mar 17, 2025
dacba62
Restore `-v` option in dorado
jayhesselberth Mar 17, 2025
1b57916
Add rule for modkit extract full
jayhesselberth Mar 17, 2025
c1efe37
Move comments out of shell call
jayhesselberth Mar 18, 2025
6159157
Increase thread number and remove arg
jayhesselberth Mar 18, 2025
38c948a
add modit full outputs
jayhesselberth Mar 18, 2025
59dbcb2
Update column names for tidy selection
jayhesselberth Mar 19, 2025
bc5441e
Increase memory for modkit rules
jayhesselberth Mar 20, 2025
4deec33
Merge branch 'main' of ssh://github.com/rnabioco/aa-tRNA-seq-pipeline
jayhesselberth Mar 20, 2025
9d77a3d
even more memory for modkit
jayhesselberth Mar 20, 2025
318fa67
Update workflow image
jayhesselberth Mar 22, 2025
5b730a7
Downgrade numpy to fix remora stats
jayhesselberth Apr 25, 2025
7d8eaf9
Merge branch 'main' of ssh://github.com/rnabioco/aa-tRNA-seq-pipeline
jayhesselberth Apr 25, 2025
514bdd7
update ignores
jayhesselberth Jun 21, 2025
914c39f
Reduce and make optional dorado verbosity
jayhesselberth Jun 21, 2025
57afd80
Use LSF-specific cluster config (#63)
jayhesselberth Jun 22, 2025
5006888
Rename to recommended test directory
jayhesselberth Jun 22, 2025
0850ae7
Reduce remora logging level
jayhesselberth Jun 22, 2025
ec81e1f
Prefer pandas as polars is flaky on some nodes
jayhesselberth Jun 23, 2025
9a01b4e
Initialize new project setup (#65)
jayhesselberth Nov 5, 2025
99405a5
Add comprehensive CI/CD build and test checks from claude code web (#68)
lkwhite Nov 6, 2025
677a72f
Add Claude Code session start hook for automated development setup (#69)
lkwhite Nov 8, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
39 changes: 39 additions & 0 deletions .claude/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
# Claude Code Configuration

This directory contains configuration and hooks for Claude Code development sessions.

## Session Start Hook

The `hooks/SessionStart` script runs automatically when you start a Claude Code session. It:

- Verifies conda/mamba is available
- Checks if the `aatrnaseqpipe` environment exists (creates it if not)
- Activates the environment
- Validates Snakemake installation
- Validates configuration files
- Sets up pre-commit hooks
- Runs a quick Snakemake syntax check

This ensures your development environment is properly configured before starting work.

## Manual Testing

You can run the session start hook manually:

```bash
bash .claude/hooks/SessionStart
```

## Development Workflow

1. **Before making changes**: Run `bash .tests/run_local_tests.sh` to verify everything works
2. **During development**: Use `snakemake -n --configfile=config/config-test.yml` for syntax checks
3. **Before committing**: Pre-commit hooks will automatically run linting and formatting
4. **After committing**: GitHub Actions will run full CI/CD checks

## Learn More

- Project documentation: `CLAUDE.md`
- Local test script: `.tests/run_local_tests.sh`
- CI/CD workflows: `.github/workflows/`
- Pre-commit config: `.pre-commit-config.yaml`
254 changes: 254 additions & 0 deletions .claude/SETUP.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,254 @@
# Complete Test and Build Check Setup

This document describes the comprehensive test and build infrastructure for the aa-tRNA-seq-pipeline.

## ✅ What's Installed

### 1. GitHub Actions CI/CD

**Location**: `.github/workflows/`

#### CI Pipeline (`ci.yml`)
Runs on: push to main/master/develop/claude/**, pull requests, manual trigger

- **Syntax Check Job**
- Sets up conda environment with Snakemake
- Validates Snakemake syntax with dry-run

- **Pipeline Integration Test Job**
- Downloads test data
- Sets up dorado and modkit tools
- Runs merge_pods rule (non-GPU test)
- Validates output directory creation

- **Configuration Validation Job**
- Validates YAML syntax in all config files
- Checks samples.tsv files exist and have content
- Verifies required directory structure

#### Lint Pipeline (`lint.yml`)
Runs on: push to main/master/develop/claude/**, pull requests, manual trigger

- **Snakemake Linting**: snakefmt format checking
- **Python Linting**: black and flake8 on workflow/scripts/
- **Markdown Linting**: markdownlint on all .md files
- **YAML Linting**: yamllint on config/ and .github/

### 2. Pre-commit Hooks

**Location**: `.pre-commit-config.yaml`

Automatically runs before each commit:
- Trailing whitespace removal
- End-of-file fixing
- YAML syntax validation
- Large file detection (>1MB warning)
- Merge conflict detection
- Line ending normalization
- Black formatting for Python scripts
- Flake8 linting for Python scripts
- Snakefmt formatting for Snakemake files

**Installation**:
```bash
pip install pre-commit
pre-commit install
```

### 3. Local Test Script

**Location**: `.tests/run_local_tests.sh`

Runs the same checks as CI locally:
1. Verifies Snakemake installation
2. Validates configuration files (YAML syntax)
3. Checks sample files exist
4. Runs Snakemake dry-run (syntax check)
5. Verifies required directory structure

**Usage**:
```bash
# Requires active conda environment
mamba activate aatrnaseqpipe
bash .tests/run_local_tests.sh
```

### 4. Claude Code Session Hooks

**Location**: `.claude/hooks/SessionStart`

Automatically runs when starting a Claude Code session:
- Verifies conda/mamba availability
- Creates aatrnaseqpipe environment if missing
- Activates the environment
- Validates Snakemake installation
- Checks config file syntax
- Installs pre-commit hooks if not present
- Runs quick Snakemake syntax check
- Displays helpful development commands

## 📋 Development Workflow

### Starting a New Session

When you start working with Claude Code, the SessionStart hook automatically:
1. Sets up your environment
2. Validates configurations
3. Shows available commands

### Before Making Changes

```bash
# Run full local test suite
bash .tests/run_local_tests.sh

# Or just syntax check
snakemake -n --configfile=config/config-test.yml
```

### During Development

```bash
# Test specific rule
snakemake <rule_name> -n --configfile=config/config-test.yml

# Run linting manually
pre-commit run --all-files

# Format Snakemake files
snakefmt workflow/

# Format Python scripts
black workflow/scripts/
```

### Before Committing

Pre-commit hooks run automatically. If they fail:
```bash
# Fix issues and re-stage
git add <fixed-files>

# Or skip hooks (not recommended)
git commit --no-verify
```

### After Pushing

GitHub Actions automatically runs:
- All syntax checks
- Integration tests
- All linters
- Configuration validation

Check status at: `https://github.com/<owner>/<repo>/actions`

## 🧪 Testing Levels

### Level 1: Quick Validation (< 30 seconds)
```bash
# Config and syntax only
python3 -c "import yaml; yaml.safe_load(open('config/config-test.yml'))"
snakemake -n --configfile=config/config-test.yml
```

### Level 2: Local Tests (~ 2 minutes)
```bash
# Full local test suite
bash .tests/run_local_tests.sh
```

### Level 3: Integration Test (~ 10-15 minutes)
```bash
# Download test data (first time only)
bash .tests/dl_test_data.sh

# Setup tools (first time only)
snakemake setup_dorado dorado_model setup_modkit --cores 1 --configfile=config/config-test.yml

# Run non-GPU pipeline rules
snakemake merge_pods --cores 2 --configfile=config/config-test.yml
```

### Level 4: Full Pipeline (requires GPU)
```bash
# Run complete pipeline with test data
snakemake --cores 12 --configfile=config/config-test.yml

# Or submit to LSF cluster
bsub < run-test.sh
```

## 🔍 Continuous Integration Details

### What Gets Tested on Every Push

1. **Snakemake syntax**: Dry-run validation
2. **Config files**: YAML syntax validation
3. **Sample files**: Existence and content checks
4. **Directory structure**: Required directories present
5. **Code formatting**: Black, flake8, snakefmt
6. **Documentation**: Markdown linting

### What Gets Tested on Pull Requests

All of the above, plus:
- Integration test with test data
- Tool setup (dorado, modkit)
- Pipeline execution (non-GPU rules)

### What's NOT Tested in CI

Due to GitHub Actions limitations:
- GPU-intensive rules (rebasecall, classify_charging)
- Full end-to-end pipeline
- LSF cluster execution

These should be tested locally or on your cluster before merging.

## 🛠️ Maintenance

### Updating Dependencies

```bash
# Update conda environment
mamba env update -n aatrnaseqpipe -f workflow/envs/aatrnaseqpipe-env.yml

# Update pre-commit hooks
pre-commit autoupdate
```

### Adding New Tests

1. **Local tests**: Edit `.tests/run_local_tests.sh`
2. **CI tests**: Edit `.github/workflows/ci.yml`
3. **Linting**: Edit `.pre-commit-config.yaml` and `.github/workflows/lint.yml`

### Troubleshooting

**Pre-commit hooks failing?**
```bash
# Run manually to see detailed errors
pre-commit run --all-files

# Update hooks to latest versions
pre-commit autoupdate
```

**CI failing but local tests pass?**
- Check GitHub Actions logs for specific errors
- Ensure all files are committed
- Verify config files are valid YAML

**SessionStart hook not running?**
- Check if Claude Code session hooks are enabled
- Run manually: `bash .claude/hooks/SessionStart`
- Ensure file is executable: `chmod +x .claude/hooks/SessionStart`

## 📚 Additional Resources

- **Project Overview**: `CLAUDE.md`
- **CI Workflows**: `.github/workflows/`
- **Pre-commit Config**: `.pre-commit-config.yaml`
- **Test Scripts**: `.tests/`
- **Snakemake Docs**: https://snakemake.readthedocs.io/
Loading
Loading