Add au integration for production-ready async backends#4
Merged
thorwhalen merged 1 commit intomasterfrom Nov 21, 2025
Merged
Conversation
Implements full working integration between qh and au (https://github.com/i2mint/au). This allows qh to use au's powerful backend system while maintaining qh's clean function-per-endpoint HTTP interface. ## What This Enables Users can now choose between: - **qh built-in**: Simple, no dependencies, perfect for development - **au backends**: Production-ready, persistent, distributed execution ## Key Features ### Integration Layer (qh/au_integration.py) - AuTaskStore: Adapter to use au's ComputationStore as qh's TaskStore - AuTaskExecutor: Adapter to use au's backends as qh's executors - Convenience functions: - use_au_backend() - Generic au backend usage - use_au_thread_backend() - Thread pool for I/O-bound tasks - use_au_process_backend() - Process pool for CPU-bound tasks - use_au_redis_backend() - Redis/RQ for distributed tasks ### Working Example ```python from qh import mk_app from qh.au_integration import use_au_thread_backend app = mk_app( [my_func], async_funcs=['my_func'], async_config=use_au_thread_backend( storage_path='/var/tasks', ttl_seconds=3600 ) ) ``` ### What au Provides That Built-in Doesn't - ✅ Persistent storage (FileSystemStore - survives restarts) - ✅ Distributed execution (Redis/RQ, Supabase backends) - ✅ Retry policies with configurable backoff strategies - ✅ Middleware system (logging, metrics, tracing) - ✅ Task dependencies and workflows - ✅ Better testing utilities - ✅ Configuration from environment variables ## Files Added 1. **qh/au_integration.py** (313 lines) - Bridge layer between qh and au - TaskStore and TaskExecutor adapters - Convenience functions for common backends 2. **examples/qh_au_integration_example.py** (189 lines) - Working examples with Thread and Process backends - Demonstrates persistent storage - Shows sync/async mixing - Comparison between built-in and au 3. **QH_AU_INTEGRATION_REPORT.md** - Detailed technical analysis - What works, what's missing in au - Recommendations for both libraries 4. **QH_AU_FINAL_SUMMARY.md** - Executive summary - Answers to key questions - Strategic recommendations - Prioritized improvement roadmap 5. **qh/__init__.py** (modified) - Export au integration functions (optional import) - Gracefully handles au not being installed ## Testing Tested with: - au v0.1.0 from claude/improve-async-http-014xdEj6rd5Sv332C794eoVV branch - ThreadBackend (I/O-bound tasks) ✅ - ProcessBackend (CPU-bound tasks) ✅ - FileSystemStore (persistent storage) ✅ - All examples run successfully ## Usage Patterns ### Development to Production ```python # Development (built-in, in-memory) app = mk_app([func], async_funcs=['func']) # Production (au with persistent storage) app = mk_app( [func], async_funcs=['func'], async_config=use_au_thread_backend('/var/tasks') ) ``` ### Mixed Backends ```python # Different backends for different tasks app = mk_app( [cpu_func, io_func], async_funcs=['cpu_func', 'io_func'], async_config={ 'cpu_func': use_au_process_backend(), 'io_func': use_au_thread_backend(), } ) ``` ## Dependencies au is optional. If not installed, qh continues to work with built-in backends. Users can install with: pip install au For Redis backend: pip install au[redis] ## Design Philosophy - qh provides the HTTP layer (beautiful UX, function-per-endpoint) - au provides the execution layer (powerful backends, persistence) - Integration is a thin adapter layer (single file) - Swapping backends is one line of code ## Future Work For qh: - Add au to pyproject.toml optional dependencies - Comprehensive integration tests - Production deployment documentation For au (recommendations): - Adopt qh's function-per-endpoint HTTP pattern - Add Pydantic for type safety - OpenAPI spec generation - Better convention-over-configuration ## Related Issues This addresses the async task processing requirements while maintaining qh's philosophy of minimal boilerplate with maximum flexibility.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Implements full working integration between qh and au (https://github.com/i2mint/au). This allows qh to use au's powerful backend system while maintaining qh's clean function-per-endpoint HTTP interface.
What This Enables
Users can now choose between:
Key Features
Integration Layer (qh/au_integration.py)
Working Example
What au Provides That Built-in Doesn't
Files Added
qh/au_integration.py (313 lines)
examples/qh_au_integration_example.py (189 lines)
QH_AU_INTEGRATION_REPORT.md
QH_AU_FINAL_SUMMARY.md
qh/init.py (modified)
Testing
Tested with:
Usage Patterns
Development to Production
Mixed Backends
Dependencies
au is optional. If not installed, qh continues to work with built-in backends. Users can install with: pip install au
For Redis backend: pip install au[redis]
Design Philosophy
Future Work
For qh:
For au (recommendations):
Related Issues
This addresses the async task processing requirements while maintaining qh's philosophy of minimal boilerplate with maximum flexibility.