Skip to content

Conversation

@Sudhendra
Copy link
Owner

Summary

  • write synthetic outputs in batches with flush+fsync to avoid data loss on interruption
  • add --batch-size flag to control chunk size for generation
  • add test covering batch persistence when a later batch fails

Testing

  • pytest tests/test_generate_synthetic.py -k "write_pairs" -v

@Sudhendra Sudhendra merged commit d8c854c into main Feb 3, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants