Skip to content

Storage format: JSON vs SQLite — input welcome #4

@Fredbcx

Description

@Fredbcx

Question

The local storage backend (see #1) needs a format decision before we lock in the API.

Option A — JSON

  • Simple, human-readable, easy to inspect and debug
  • Slower for large histories, no querying without loading everything into memory
  • Good for: small projects, early adopters, simple workflows

Option B — SQLite

  • Queryable without loading the full graph
  • Better for long-running models with hundreds of versions
  • Slightly more complex to implement and inspect manually

Input wanted

If you're the target user (running continual learning in production):

  • How many model versions do you typically accumulate over 6-12 months?
  • Do you need to query lineage data without loading a Python process?
  • Would you inspect the storage file directly, or always go through the API?

This will directly determine which we build first.

Metadata

Metadata

Assignees

No one assigned

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions