From fd0dcb56c4765763dbeca073d574fa1726ff65a4 Mon Sep 17 00:00:00 2001 From: Ben Emson Date: Wed, 23 Jul 2025 18:36:59 +0100 Subject: [PATCH 1/2] Organised the docs into directories --- docs/{ => features}/feature_dir_glob.md | 0 docs/{ => features}/feature_specs_tidy.md | 0 docs/{ => features}/feature_youtube_transcript_mcp.md | 0 docs/{ => notes}/notes_elf_sdk.md | 2 +- docs/{ => notes}/notes_fix_agent_principles.md | 0 docs/{ => notes}/notes_fix_agent_principles_cust_graph.md | 0 docs/{ => notes}/notes_fix_function_principles.md | 0 docs/{ => plans}/plan_dir_glob.md | 0 docs/{ => plans}/plan_opensource-min.md | 0 docs/{ => plans}/plan_opensource.md | 0 docs/{ => plans}/plan_specs_tidy.md | 0 docs/{ => plans}/plan_tui_refactor.md | 0 docs/{ => plans}/plan_youtube_transcript_mcp.md | 0 13 files changed, 1 insertion(+), 1 deletion(-) rename docs/{ => features}/feature_dir_glob.md (100%) rename docs/{ => features}/feature_specs_tidy.md (100%) rename docs/{ => features}/feature_youtube_transcript_mcp.md (100%) rename docs/{ => notes}/notes_elf_sdk.md (99%) rename docs/{ => notes}/notes_fix_agent_principles.md (100%) rename docs/{ => notes}/notes_fix_agent_principles_cust_graph.md (100%) rename docs/{ => notes}/notes_fix_function_principles.md (100%) rename docs/{ => plans}/plan_dir_glob.md (100%) rename docs/{ => plans}/plan_opensource-min.md (100%) rename docs/{ => plans}/plan_opensource.md (100%) rename docs/{ => plans}/plan_specs_tidy.md (100%) rename docs/{ => plans}/plan_tui_refactor.md (100%) rename docs/{ => plans}/plan_youtube_transcript_mcp.md (100%) diff --git a/docs/feature_dir_glob.md b/docs/features/feature_dir_glob.md similarity index 100% rename from docs/feature_dir_glob.md rename to docs/features/feature_dir_glob.md diff --git a/docs/feature_specs_tidy.md b/docs/features/feature_specs_tidy.md similarity index 100% rename from docs/feature_specs_tidy.md rename to docs/features/feature_specs_tidy.md diff --git a/docs/feature_youtube_transcript_mcp.md b/docs/features/feature_youtube_transcript_mcp.md similarity index 100% rename from docs/feature_youtube_transcript_mcp.md rename to docs/features/feature_youtube_transcript_mcp.md diff --git a/docs/notes_elf_sdk.md b/docs/notes/notes_elf_sdk.md similarity index 99% rename from docs/notes_elf_sdk.md rename to docs/notes/notes_elf_sdk.md index 824dc83..54d31b6 100644 --- a/docs/notes_elf_sdk.md +++ b/docs/notes/notes_elf_sdk.md @@ -373,7 +373,7 @@ advanced_result = advanced_analyzer.run("Perform deep analysis of market trends" - **Environment variables supported** - `.env` files and `export` commands ### Gradual Adoption Path -1. **Install**: Same package (`pip install elf0` or `uv pip install elf0`) +1. **Install**: Same package (development: `git clone` and `uv pip install -e .`) 2. **Use CLI**: Continue using CLI as before 3. **Import SDK**: Start with `from elf0 import ElfWorkflow` 4. **Migrate incrementally**: Replace CLI subprocess calls with SDK calls diff --git a/docs/notes_fix_agent_principles.md b/docs/notes/notes_fix_agent_principles.md similarity index 100% rename from docs/notes_fix_agent_principles.md rename to docs/notes/notes_fix_agent_principles.md diff --git a/docs/notes_fix_agent_principles_cust_graph.md b/docs/notes/notes_fix_agent_principles_cust_graph.md similarity index 100% rename from docs/notes_fix_agent_principles_cust_graph.md rename to docs/notes/notes_fix_agent_principles_cust_graph.md diff --git a/docs/notes_fix_function_principles.md b/docs/notes/notes_fix_function_principles.md similarity index 100% rename from docs/notes_fix_function_principles.md rename to docs/notes/notes_fix_function_principles.md diff --git a/docs/plan_dir_glob.md b/docs/plans/plan_dir_glob.md similarity index 100% rename from docs/plan_dir_glob.md rename to docs/plans/plan_dir_glob.md diff --git a/docs/plan_opensource-min.md b/docs/plans/plan_opensource-min.md similarity index 100% rename from docs/plan_opensource-min.md rename to docs/plans/plan_opensource-min.md diff --git a/docs/plan_opensource.md b/docs/plans/plan_opensource.md similarity index 100% rename from docs/plan_opensource.md rename to docs/plans/plan_opensource.md diff --git a/docs/plan_specs_tidy.md b/docs/plans/plan_specs_tidy.md similarity index 100% rename from docs/plan_specs_tidy.md rename to docs/plans/plan_specs_tidy.md diff --git a/docs/plan_tui_refactor.md b/docs/plans/plan_tui_refactor.md similarity index 100% rename from docs/plan_tui_refactor.md rename to docs/plans/plan_tui_refactor.md diff --git a/docs/plan_youtube_transcript_mcp.md b/docs/plans/plan_youtube_transcript_mcp.md similarity index 100% rename from docs/plan_youtube_transcript_mcp.md rename to docs/plans/plan_youtube_transcript_mcp.md From 142498dc9a2265610edd5d48f5fcbcbd14f81866 Mon Sep 17 00:00:00 2001 From: Ben Emson Date: Fri, 1 Aug 2025 20:35:57 +0100 Subject: [PATCH 2/2] Added new CONTRIBUTING.md doc. --- CONTRIBUTING.md | 301 ++++++++++++++++++++++++++++++++++ docs/notes/notes_auto_docs.md | 178 ++++++++++++++++++++ 2 files changed, 479 insertions(+) create mode 100644 CONTRIBUTING.md create mode 100644 docs/notes/notes_auto_docs.md diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..96c1082 --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,301 @@ +# ๐Ÿค Contributing to Elf0 + +Thank you for your interest in contributing to Elf0! We're excited to have you join our community of developers building the future of AI agent workflows. This project thrives because of contributors like you. + +## ๐ŸŒŸ Ways to Contribute + +We welcome all kinds of contributions, not just code: + +- **๐Ÿ› Bug reports** - Help us identify and fix issues +- **๐Ÿ’ก Feature requests** - Suggest new capabilities +- **๐Ÿ“ Documentation** - Improve guides, examples, and API docs +- **๐Ÿงช Testing** - Write tests, test new features, improve test coverage +- **๐Ÿ’ป Code** - Fix bugs, implement features, improve performance +- **๐ŸŽฏ Workflow examples** - Share useful YAML workflows in `specs/` +- **๐Ÿ› ๏ธ Tools** - Create MCP servers or Python functions +- **๐Ÿ“ข Community** - Help others in discussions, write blog posts + +## ๐Ÿš€ Quick Start for Contributors + +### 1. Set Up Your Development Environment + +```bash +# Fork the repository on GitHub, then clone your fork +git clone https://github.com/emson/elf0.git +cd elf0 + +# Set up the development environment +uv venv && source .venv/bin/activate # Windows: .venv\Scripts\activate +uv pip install -e . + +# Install development dependencies +uv pip install -e . --group dev + +# Verify installation +uv run elf0 --help +``` + +### 2. Get API Keys (for testing) + +You'll need at least one API key to test workflows: + +```bash +# Choose one or more: +export OPENAI_API_KEY="your-openai-key" +export ANTHROPIC_API_KEY="your-anthropic-key" +# Or install Ollama for local testing: https://ollama.ai/ +``` + +### 3. Run the Test Suite + +Before making changes, ensure everything works: + +```bash +# Run all tests +uv run pytest tests/ --timeout=300 + +# Run tests with coverage +uv run pytest tests/ --cov=src/elf0 --timeout=300 + +# Run only unit tests (faster) +uv run pytest tests/ -m "not requires_external" + +# Run integration tests (requires API keys) +ELF_RUN_INTEGRATION_TESTS=1 uv run pytest tests/ +``` + +### 4. Verify Code Quality + +```bash +# Lint code + uv run ruff check src/ tests/ + +# Type check +mypy src/ + +# Security scan +uv run bandit -r src/ +``` + +## ๐Ÿ“‹ Development Workflow + +### Making Changes + +1. **Create a branch** for your changes: + ```bash + git checkout -b feature/your-feature-name + # or: git checkout -b fix/issue-description + ``` + +2. **Make your changes** following our [coding guidelines](#-code-style-guidelines) + +3. **Test your changes**: + ```bash + # Run relevant tests + uv run pytest tests/ -k "test_your_feature" + + # Test with a real workflow + uv run elf0 agent specs/basic/chat_simple_v1.yaml --prompt "Test your changes" + ``` + +4. **Check code quality**: + ```bash + uv run ruff check src/ tests/ + mypy src/ + ``` + +5. **Commit your changes**: + ```bash + git add . + git commit -m "Add feature: brief description of what you added" + ``` + +### Submitting a Pull Request + +1. **Push your branch**: + ```bash + git push origin feature/your-feature-name + ``` + +2. **Create a Pull Request** on GitHub with: + - Clear title describing the change + - Description explaining what and why + - Link to any related issues + - Screenshots/examples if applicable + +3. **Address feedback** from maintainers + +4. **Celebrate** when your PR is merged! ๐ŸŽ‰ + +## ๐ŸŽฏ Code Style Guidelines + +We follow the guidelines established in `CLAUDE.md`. Key points: + +### Python Code Style +- **PEP 8** conventions with 4-space indentation +- **Type hints** required for all function parameters and return values +- **Pydantic** for data validation and model definitions +- **Early returns** when possible +- **Absolute imports** only (no relative imports) + +### Naming Conventions +- **Classes**: `PascalCase` +- **Functions/methods**: `snake_case` +- **Variables**: `snake_case` +- **Constants**: `UPPER_SNAKE_CASE` +- **Files**: `snake_case.py` + +### Documentation +- **Docstrings** for all classes and functions using Google style +- **File location comment** at the top: `# src/my_module/my_file.py` +- **Type aliases** for clarity: `StepID = str` + +### Git Conventions +- **Commit messages** in imperative mood: "Fix bug" not "Fixed bug" +- **Branch names**: `feature/add-logging`, `fix/memory-leak` +- **Check status** before committing: `git status && git diff` + +## ๐Ÿงช Testing Guidelines + +### Writing Tests +- Place tests in the `tests/` directory mirroring `src/` structure +- Use descriptive test names: `test_workflow_runs_successfully_with_valid_yaml` +- Test both happy paths and error conditions +- Mock external dependencies (API calls, file system) + +### Test Types +- **Unit tests** (`-m unit`): Fast, isolated, no external dependencies +- **Integration tests** (`-m integration`): Test component interactions +- **External tests** (`-m requires_external`): Require real APIs/services + +### Running Specific Tests +```bash +# Test a specific file +uv run pytest tests/core/test_compiler.py + +# Test a specific function +uv run pytest tests/core/test_compiler.py::test_compile_workflow + +# Test with markers +uv run pytest tests/ -m "unit and not slow" +``` + +## ๐Ÿ› Reporting Issues + +### Bug Reports +Use our [issue template](https://github.com/emson/elf0/issues/new) and include: + +- **Clear title** summarising the problem +- **Environment details**: OS, Python version, elf0 version +- **Steps to reproduce** the issue +- **Expected vs actual behaviour** +- **Error messages** (full stack traces help!) +- **YAML workflow** if relevant (anonymise sensitive data) + +### Feature Requests +- **Describe the problem** you're trying to solve +- **Propose a solution** if you have ideas +- **Consider backwards compatibility** +- **Think about edge cases** + +## ๐Ÿ“ Project Structure + +Understanding the codebase: + +``` +src/elf0/ +โ”œโ”€โ”€ cli.py # Command-line interface +โ”œโ”€โ”€ core/ # Core functionality +โ”‚ โ”œโ”€โ”€ compiler.py # YAML workflow compilation +โ”‚ โ”œโ”€โ”€ runner.py # Workflow execution +โ”‚ โ”œโ”€โ”€ llm_client.py # LLM integrations +โ”‚ โ””โ”€โ”€ nodes/ # Workflow node types +โ”œโ”€โ”€ functions/ # Python function library +โ””โ”€โ”€ utils/ # Utility modules + +specs/ # Example workflows +โ”œโ”€โ”€ basic/ # Simple examples +โ”œโ”€โ”€ content/ # Content generation +โ”œโ”€โ”€ examples/ # Feature demonstrations +โ””โ”€โ”€ utils/ # Workflow tools + +tests/ # Test suite +โ”œโ”€โ”€ core/ # Core functionality tests +โ”œโ”€โ”€ cli/ # CLI tests +โ””โ”€โ”€ integration/ # End-to-end tests +``` + +## ๐ŸŽ Types of Contributions We're Looking For + +### High Priority +- **Bug fixes** - Especially with test cases +- **Documentation improvements** - Make it clearer for new users +- **Workflow examples** - Useful real-world YAML specs +- **Test coverage** - Help us reach 90%+ coverage + +### Medium Priority +- **Performance improvements** - Faster workflow execution +- **New LLM integrations** - Support more models +- **MCP servers** - Useful tool integrations +- **CLI enhancements** - Better user experience + +### Ideas Welcome +- **New node types** - Extend workflow capabilities +- **Workflow debugger** - Help users troubleshoot +- **Web interface** - Visual workflow builder +- **Workflow sharing** - Community marketplace + +## ๐Ÿ’ฌ Getting Help + +- **GitHub Discussions**: Ask questions, share ideas +- **Issues**: Report bugs, request features +- **Discord** (coming soon): Real-time community chat + +## ๐Ÿ† Recognition + +Contributors are recognised in: +- **Release notes** for significant contributions +- **README acknowledgments** for major features +- **Git commit co-authorship** when pairing + +## ๐Ÿ“œ Code of Conduct + +We're committed to providing a welcoming and inclusive experience for everyone. We expect all community members to: + +### Our Standards +- **Be respectful** of differing viewpoints and experiences +- **Give and accept constructive feedback** gracefully +- **Focus on what's best** for the community and project +- **Show empathy** towards other community members + +### Unacceptable Behaviour +- Harassment, discrimination, or inappropriate conduct +- Personal attacks or inflammatory comments +- Publishing private information without permission +- Any conduct that could be considered inappropriate in a professional setting + +### Enforcement +Project maintainers will enforce these standards fairly and consistently. Reports can be made by contacting the project team at [ben@emson.dev](mailto:ben@emson.dev). + +## ๐Ÿ”’ Security + +If you find a security vulnerability, please email [ben@emson.dev](mailto:ben@emson.dev) instead of opening an issue. We'll work with you to resolve it promptly. + +## ๐Ÿ“„ License + +By contributing to Elf0, you agree that your contributions will be licensed under the Apache License 2.0, the same licence as the project. + +--- + +## ๐Ÿš€ Ready to Contribute? + +1. **Browse our [good first issues](https://github.com/emson/elf0/labels/good%20first%20issue)** +2. **Read the [project overview](docs/project_overview.md)** to understand our goals +3. **Join the conversation** in GitHub Discussions +4. **Say hello** - we're always happy to help new contributors get started! + +Thank you for helping make Elf0 better for everyone! ๐ŸŽ‰ + +--- + +*This contributing guide is inspired by open source best practices and tailored specifically for the Elf0 project. Have suggestions for improving it? Open an issue or PR!* diff --git a/docs/notes/notes_auto_docs.md b/docs/notes/notes_auto_docs.md new file mode 100644 index 0000000..5225d6f --- /dev/null +++ b/docs/notes/notes_auto_docs.md @@ -0,0 +1,178 @@ +# Professional Documentation Generation and Hosting Notes + +## Current State Analysis + +**Existing Setup:** +- โœ… MkDocs dependencies already configured in `pyproject.toml` (docs group) +- โœ… Comprehensive documentation structure in `/docs` directory +- โœ… Rich Markdown content across multiple categories +- โŒ No `mkdocs.yml` configuration file +- โŒ No automated CI/CD documentation deployment +- โŒ Documentation not currently hosted + +## Recommended Professional Documentation Strategy + +### Phase 1: MkDocs Material Setup +1. **Create `mkdocs.yml` configuration file** with: + - Material theme configuration + - Navigation structure based on existing docs + - Search and syntax highlighting + - Professional Material Design theme + +2. **Organize documentation structure**: + - User Guide (installation, quickstart, CLI reference) + - Developer Guide (architecture, contributing, testing) + - API Reference (auto-generated from docstrings using mkdocstrings) + - Examples and Tutorials + - Specifications and schemas + +3. **Add `__init__.py` version handling**: + - Create proper `__version__` variable for Hatch version management + - Enable automatic version detection in documentation + +### Phase 2: Auto-Generated API Documentation +1. **Configure mkdocstrings plugin** for automatic API documentation from docstrings +2. **Enhance existing docstrings** across core modules (`src/elf0/core/`, `src/elf0/cli.py`) +3. **Create API reference pages** for: + - Core workflow engine + - CLI commands + - Configuration options + - Function development + +### Phase 3: GitHub Pages Hosting with CI/CD +1. **Create GitHub Actions workflow** (`.github/workflows/docs.yml`): + - Automated builds on push to main branch + - Deploy to GitHub Pages using `mkdocs gh-deploy` + - Cache dependencies for faster builds + +2. **Configure GitHub repository settings**: + - Enable GitHub Pages + - Set gh-pages branch as source + - Update repository description and topics + +### Phase 4: Professional Documentation Content +1. **Restructure existing content**: + - Convert development notes to user-facing guides + - Create step-by-step tutorials + - Add code examples with syntax highlighting + +2. **Create missing documentation**: + - Installation guide (development and production) + - Workflow specification reference + - Troubleshooting guide + - Contributing guidelines + +## Hosting Options (Recommended: GitHub Pages) + +**Primary Recommendation: GitHub Pages** +- โœ… Free hosting for open source projects +- โœ… Automatic HTTPS and custom domain support +- โœ… Seamless integration with GitHub Actions +- โœ… Professional URL: `https://emson.github.io/elf0/` + +**Alternative Options:** +- Read the Docs (free, good for open source) +- Netlify (advanced features, generous free tier) +- Vercel (excellent performance, free tier) + +## Implementation Steps + +1. **Setup MkDocs configuration** and basic structure +2. **Migrate and organize existing documentation** +3. **Configure automated API documentation generation** +4. **Create GitHub Actions workflow for deployment** +5. **Test and refine documentation build process** +6. **Update repository README with documentation links** + +## Expected Outcomes + +- **Professional documentation site** with modern Material Design +- **Automated deployment** on every commit to main branch +- **Searchable documentation** with full-text search +- **Mobile-responsive design** for all devices +- **API documentation** automatically generated from code +- **Version-controlled documentation** alongside code changes + +This approach will transform Elf0's documentation from development notes into a professional, searchable, and automatically updated documentation site that serves both users and developers effectively. + +## Technical Implementation Details + +### MkDocs Configuration Structure +```yaml +site_name: Elf0 Documentation +site_url: https://emson.github.io/elf0/ +repo_url: https://github.com/emson/elf0 +theme: + name: material + features: + - navigation.tabs + - navigation.sections + - toc.integrate + - navigation.top + - search.suggest + - search.highlight + - content.tabs.link + - content.code.annotation + - content.code.copy +``` + +### GitHub Actions Workflow +```yaml +name: ci +on: + push: + branches: + - master + - main +permissions: + contents: write +jobs: + deploy: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-python@v4 + with: + python-version: 3.x + - run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV + - uses: actions/cache@v3 + with: + key: mkdocs-material-${{ env.cache_id }} + path: .cache + restore-keys: | + mkdocs-material- + - run: pip install mkdocs-material + - run: mkdocs gh-deploy --force +``` + +### Current Documentation Structure +``` +docs/ +โ”œโ”€โ”€ notes/ # Development notes (convert to guides) +โ”œโ”€โ”€ features/ # Feature documentation +โ”œโ”€โ”€ libs/ # Library guides +โ”œโ”€โ”€ docs_specs/ # Specifications +โ””โ”€โ”€ *.md # Various documentation files +``` + +### Proposed Professional Structure +``` +docs/ +โ”œโ”€โ”€ index.md # Home page +โ”œโ”€โ”€ getting-started/ # Installation, quickstart +โ”œโ”€โ”€ user-guide/ # CLI reference, workflows +โ”œโ”€โ”€ developer-guide/ # Architecture, contributing +โ”œโ”€โ”€ api-reference/ # Auto-generated API docs +โ”œโ”€โ”€ examples/ # Tutorials and examples +โ””โ”€โ”€ specifications/ # YAML schemas and specs +``` + +## Dependencies Analysis + +Current documentation dependencies in `pyproject.toml`: +- `mkdocs>=1.6.0` +- `mkdocs-material>=9.5.0` +- `mkdocstrings[python]>=0.26.0` +- `markdown>=3.7.0` + +All required dependencies are already configured, making implementation straightforward. \ No newline at end of file