Skip to content

[Cleanup] Remove deprecated code after distributed architecture migration #55

@zhexuany

Description

@zhexuany

Summary

Track code cleanup tasks after the distributed architecture is fully implemented.

Parent Epic

When to Execute

After all distributed features are complete and tested (Phase 10+)

Code to Remove

1. TiKV Backup Directory

  • Delete crates/roboflow-distributed/src/tikv_backup/ (duplicate of tikv/)

Code to Refactor

2. OssStorage Full-Object Buffering

3. Sequential Multipart Uploads

4. Sequential Batch Processing

5. Sequential Reader Usage

  • crates/roboflow-pipeline/src/orchestrator/orchestrator.rs lines 398-403
  • Enable parallel readers

Code to Deprecate

6. Python Examples

  • Add deprecation warning to examples/python/roboflow_utils.py::BatchProcessor
  • Add deprecation notice to examples/python/batch_conversion.py
  • Document migration path to distributed CLI

TODOs to Address

  • crates/roboflow-pipeline/src/fluent/builder.rs line 397 - Standard pipeline
  • crates/roboflow-pipeline/src/fluent/builder.rs lines 543, 596, 609, 657, 672, 690
  • examples/rust/lerobot_convert.rs lines 51, 73, 92
  • tests/kps_integration_tests.rs line 18

Tests to Update

  • tests/sequential_parallel_comparison_tests.rs - Add distributed tests
  • tests/pipeline_round_trip_tests.rs - Add job processing tests
  • tests/streaming_converter_tests.rs - Add distributed streaming tests

Acceptance Criteria

  • No duplicate code remains
  • Full-object buffering replaced with streaming
  • Sequential uploads replaced with parallel
  • Batch processing uses TiKV queue
  • Deprecated code has warnings
  • All TODOs addressed or tracked
  • Tests updated for distributed architecture

Metadata

Metadata

Assignees

No one assigned

    Labels

    priority/lowLow priority, nice to havesize/MMedium: 3-5 daystype/refactorCode refactoring without behavior change

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions