Skip to content

Support Async #2891

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. Weโ€™ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 12 commits into
base: master
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
115 changes: 113 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,14 +51,22 @@ to both create the virtual environment and install the package. Otherwise, you c
download the source from `GitHub <https://github.com/MongoEngine/mongoengine>`_ and
run ``python setup.py install``.

For async support, you need PyMongo 4.13+ and can install it with:

.. code-block:: shell

# Install MongoEngine with async support
$ python -m pip install mongoengine[async]

The support for Python2 was dropped with MongoEngine 0.20.0

Dependencies
============
All of the dependencies can easily be installed via `python -m pip <https://pip.pypa.io/>`_.
At the very least, you'll need these two packages to use MongoEngine:
At the very least, you'll need these packages to use MongoEngine:

- pymongo>=3.12
- pymongo>=3.12 (for synchronous operations)
- pymongo>=4.13 (for asynchronous operations)

If you utilize a ``DateTimeField``, you might also use a more flexible date parser:

Expand Down Expand Up @@ -127,6 +135,109 @@ Some simple examples of what MongoEngine code looks like:
>>> BlogPost.objects(tags='mongodb').count()
1

Async Support
=============
MongoEngine provides comprehensive asynchronous support using PyMongo's AsyncMongoClient.
All major database operations are available with async/await syntax:

.. code :: python

import datetime
import asyncio
from mongoengine import *

async def main():
# Connect asynchronously
await connect_async('mydb')

# Document operations
post = TextPost(title='Async Post', content='Async content')
await post.async_save()
await post.async_reload()
await post.async_delete()

# QuerySet operations
post = await TextPost.objects.async_get(title='Async Post')
posts = await TextPost.objects.filter(tags='python').async_to_list()
count = await TextPost.objects.async_count()

# Async iteration
async for post in TextPost.objects.filter(published=True):
print(post.title)

# Bulk operations
await TextPost.objects.filter(draft=True).async_update(published=True)
await TextPost.objects.filter(old=True).async_delete()

# Reference field async fetching
# In async context, references return AsyncReferenceProxy
if hasattr(post, 'author'):
author = await post.author.async_fetch()

# Transactions
from mongoengine import async_run_in_transaction
async with async_run_in_transaction():
await post1.async_save()
await post2.async_save()

# GridFS async operations
from mongoengine import FileField
class MyDoc(Document):
file = FileField()

doc = MyDoc()
await MyDoc.file.async_put(file_data, instance=doc)

# Context managers
from mongoengine import async_switch_db
async with async_switch_db(MyDoc, 'other_db'):
await doc.async_save()

asyncio.run(main())

**Supported Async Features:**

- **Document Operations**: async_save(), async_delete(), async_reload()
- **QuerySet Operations**: async_get(), async_first(), async_count(), async_create()
- **Bulk Operations**: async_update(), async_delete(), async_update_one()
- **Async Iteration**: Support for ``async for`` with QuerySets
- **Reference Fields**: async_fetch() for explicit dereferencing
- **GridFS**: async_put(), async_get(), async_read(), async_delete()
- **Transactions**: async_run_in_transaction() context manager
- **Context Managers**: async_switch_db(), async_switch_collection()
- **Aggregation**: async_aggregate(), async_distinct()
- **Cascade Operations**: Full support for all delete rules (CASCADE, NULLIFY, etc.)

**Current Limitations:**

The following features are intentionally not implemented due to low priority or complexity:

- **async_values()**, **async_values_list()**: Field projection methods

*Reason*: Low usage frequency in typical applications. Can be implemented if needed.

- **async_explain()**: Query execution plan analysis

*Reason*: Debugging/optimization feature with limited general use.

- **Hybrid Signal System**: Automatic sync/async signal handling

*Reason*: High complexity due to backward compatibility requirements.
Consider as separate project if needed.

- **ListField with ReferenceField**: Automatic AsyncReferenceProxy conversion

*Reason*: Complex implementation requiring deep changes to ListField.
Manual async dereferencing is required for now.

**Migration Guide:**

- Use ``connect_async()`` instead of ``connect()``
- Add ``async_`` prefix to all database operations: ``save()`` โ†’ ``async_save()``
- Use ``async for`` for QuerySet iteration
- Explicitly fetch references with ``await ref.async_fetch()`` in async context
- Existing synchronous code remains 100% compatible when using ``connect()``

Tests
=====
To run the test suite, ensure you are running a local instance of MongoDB on
Expand Down
Loading
Loading