Skip to content

Implement local storage backend (JSON) #1

@Fredbcx

Description

@Fredbcx

Context

Currently Tracker holds all lineage data in memory and loses it when the process ends.
To be usable in real workflows, MLineage needs to persist the lineage graph to disk.

Task

Implement a JsonStorageBackend in mlineage/storage/ that:

  • serializes the LineageGraph to a .json file on tracker.save()
  • deserializes it on Tracker.load()
  • stores files under the path passed as storage= in the Tracker constructor

Interface to implement

# mlineage/storage/json_backend.py
class JsonStorageBackend:
    def save(self, graph: LineageGraph, path: str) -> None: ...
    def load(self, path: str) -> LineageGraph: ...

Notes

  • All core models already have @dataclass — use dataclasses.asdict() as a starting point for serialization
  • Datetime fields need special handling (ISO format strings)
  • Tests go in tests/unit/test_storage.py

This is a self-contained task with a clear interface — good entry point into the codebase.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions