Skip to content

fix(vector_io): set WAL mode and busy_timeout on sqlite-vec connections#5428

Open
extrasmall0 wants to merge 2 commits intoogx-ai:mainfrom
extrasmall0:fix/sqlite-vec-wal-pragmas
Open

fix(vector_io): set WAL mode and busy_timeout on sqlite-vec connections#5428
extrasmall0 wants to merge 2 commits intoogx-ai:mainfrom
extrasmall0:fix/sqlite-vec-wal-pragmas

Conversation

@extrasmall0
Copy link
Copy Markdown

The inline sqlite-vec provider opens database connections without setting WAL mode or busy_timeout. When running with multiple uvicorn workers (server.workers > 1), concurrent writes fail immediately with sqlite3.OperationalError: database is locked instead of waiting and retrying.

The SQL store backend (sqlalchemy_sqlstore.py) already handles this correctly by setting journal_mode=WAL, busy_timeout=5000, and synchronous=NORMAL. This patch applies the same pragmas to the shared _create_sqlite_connection() helper used by the vector store.

Changes:

  • Set timeout=5.0 on sqlite3.connect() as a Python-level busy wait
  • Execute PRAGMA journal_mode=WAL for concurrent reader/writer support
  • Execute PRAGMA busy_timeout=5000 so SQLite retries instead of raising immediately
  • Execute PRAGMA synchronous=NORMAL for better write throughput (still safe with WAL)
  • Added a unit test verifying the pragmas are set on new connections

Fixes #5344

The inline::sqlite-vec provider opens database connections without
setting WAL mode or busy_timeout. With multiple uvicorn workers,
concurrent writes fail with 'database is locked' instead of retrying.

Apply the same pragmas already used by the SQL store backend:
- journal_mode=WAL for concurrent reader/writer support
- busy_timeout=5000 to retry instead of failing immediately
- synchronous=NORMAL for better performance (safe with WAL)
- timeout=5.0 on connect() as a Python-level fallback

Fixes ogx-ai#5344
@meta-cla
Copy link
Copy Markdown

meta-cla Bot commented Apr 2, 2026

Hi @extrasmall0!

Thank you for your pull request and welcome to our community.

Action Required

In order to merge any pull request (code, docs, etc.), we require contributors to sign our Contributor License Agreement, and we don't seem to have one on file for you.

Process

In order for us to review and merge your suggested changes, please sign at https://code.facebook.com/cla. If you are contributing on behalf of someone else (eg your employer), the individual CLA may not be sufficient and your employer may need to sign the corporate CLA.

Once the CLA is signed, our tooling will perform checks and validations. Afterwards, the pull request will be tagged with CLA signed. The tagging process may take up to 1 hour after signing. Please give it that time before contacting us about it.

If you have received this in error or have any questions, please contact us at cla@meta.com. Thanks!

@meta-cla meta-cla Bot added the CLA Signed This label is managed by the Meta Open Source bot. label Apr 16, 2026
@meta-cla
Copy link
Copy Markdown

meta-cla Bot commented Apr 16, 2026

Thank you for signing our Contributor License Agreement. We can now accept your code for this (and any) Meta Open Source project. Thanks!

@nidhishgajjar

This comment was marked as spam.

1 similar comment
@nidhishgajjar

This comment was marked as spam.

@extrasmall0
Copy link
Copy Markdown
Author

extrasmall0 commented Apr 21, 2026

Quick status refresh after another recheck: this PR is still approved, and the core sqlite-vec behavior exercised by the patch remains the same. The practical change is still just enabling WAL mode and a busy timeout on the sqlite-vec connection so concurrent access is less likely to fail with transient locking issues. The branch now needs a rebase because main moved forward, but there is no new design question on the fix itself. If the maintainers still want this path, I can rebase and refresh it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Meta Open Source bot.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

inline::sqlite-vec missing WAL mode and busy_timeout on vector store connections

3 participants