Support Async

[jeonghanjoo]

With claude code, our team can add async support to this lovely mongoengine.

Our team is testing in our application(fastAPI).

---

Comprehensive Async Support for MongoEngine

This PR adds full asynchronous support to MongoEngine, enabling high-performance async/await operations while maintaining 100% backward compatibility with existing synchronous code.

🎯 Overview

MongoEngine now supports comprehensive async operations using PyMongo's AsyncMongoClient. This implementation allows developers to build scalable applications with async/await syntax without requiring any changes to existing document models.

🚀 Key Features

Core Async Operations

• Document Operations: async_save(), async_delete(), async_reload(), async_ensure_indexes()
• QuerySet Operations: async_get(), async_first(), async_count(), async_create(), async_to_list()
• Bulk Operations: async_update(), async_update_one(), async_delete()
• Async Iteration: Full support for async for with QuerySets

Advanced Features

• Reference Fields: Async dereferencing with AsyncReferenceProxy and async_fetch()
• GridFS: Complete async file operations (async_put(), async_get(), async_read(), async_delete())
• Transactions: async_run_in_transaction() context manager
• Context Managers: async_switch_db(), async_switch_collection(), async_no_dereference()
• Aggregation: async_aggregate(), async_distinct()
• Cascade Operations: Full support for all deletion rules (CASCADE, NULLIFY, etc.)

🔧 Technical Implementation

Connection Management

• New connect_async() function with proper AsyncMongoClient handling
• Automatic connection type detection and enforcement
• Clear error messages for wrong connection usage
• PyMongo 4.13+ dependency management with graceful fallback

Backward Compatibility

• 100% Compatible: All existing sync code works unchanged
• No Model Changes: Document classes require no modifications
• Mixed Usage: Sync and async connections can coexist in the same application
• Clear Separation: Async methods use async_ prefix pattern

Error Handling

• Runtime errors prevent mixing sync/async methods with wrong connections
• Helpful error messages guide users to correct usage
• Proper ImportError handling for PyMongo version compatibility

📊 Test Coverage

• 79+ async-specific tests covering all features
• Complete test suite passes (1000+ existing tests remain functional)
• Integration tests for complex async workflows
• Error handling tests for connection type validation

📚 Documentation

Updated Documentation

• README.rst: Comprehensive async examples and installation guide
• docs/guide/async-support.rst: 500+ line detailed usage guide
• Migration examples: Step-by-step sync-to-async conversion
• Performance tips and best practices

Installation Support

# Install with async support
pip install mongoengine[async]

🔄 Intentionally Deferred Features

The following features were strategically deferred to maintain focus and ensure core stability:

• async_values(), async_values_list(): Low usage frequency, can use aggregation workaround
• async_explain(): Debugging feature, PyMongo direct access available
• Hybrid Signal System: Complex implementation, consider as separate enhancement
• ListField ReferenceField Auto-conversion: Requires deep structural changes

💻 Usage Examples

Basic Operations

import asyncio
from mongoengine import Document, StringField, connect_async

class User(Document):
    name = StringField(required=True)
    email = StringField(required=True)

async def main():
    # Connect asynchronously
    await connect_async('mydatabase')
    
    # Document operations
    user = User(name="John", email="john@example.com")
    await user.async_save()
    
    # QuerySet operations  
    users = await User.objects.filter(name="John").async_to_list()
    count = await User.objects.async_count()
    
    # Async iteration
    async for user in User.objects.filter(active=True):
        print(f"User: {user.name}")

asyncio.run(main())

Advanced Features

# Transactions
async with async_run_in_transaction():
    await user1.async_save()
    await user2.async_save()

# Context managers
async with async_switch_db(User, 'analytics_db'):
    await analytics_user.async_save()

# GridFS
await MyDoc.file.async_put(file_data, instance=doc)
content = await MyDoc.file.async_read(doc)

🏗️ Development Process

This implementation was developed through a systematic 4-phase approach:

1. Phase 1: Async connection foundation and basic document operations
2. Phase 2: QuerySet async methods and iteration support
3. Phase 3: Reference fields, GridFS, and complex field types
4. Phase 4: Advanced features (transactions, context managers, aggregation)

Each phase included comprehensive testing and documentation updates.

✅ Quality Assurance

• All pre-commit hooks pass: black, flake8, isort, pyupgrade
• CONTRIBUTING.rst compliance: Follows all project guidelines
• Dependency management: Proper PyMongo version handling
• Python 3.9-3.13 support: Aligned with upstream version policy

🔮 Future Compatibility

This implementation is designed for long-term stability:

• Follows PyMongo async patterns and best practices
• Maintains MongoEngine's existing architecture and conventions
• Extensible design allows for future enhancements
• Clean separation enables independent sync/async development

[jeonghanjoo]

I will leave our live test results later. We are also not sure if it is safe to use and if it is worth to use.

[jeonghanjoo]

We found some issues and are working on.

[jeonghanjoo]

We fully migrated and deployed our application project. There were some issues and we fixed.