Skip to content

Implement streaming decode support in live ONNX/Vitis inference #40

@Shreyas582

Description

@Shreyas582

Scope

Add token streaming decode output path for live ONNX/Vitis inference so operators can observe progress before full completion.

Acceptance Criteria

  • Live runtime supports incremental token decode/emit mode.
  • JSON output remains contract-compatible when streaming is enabled/disabled.
  • Fallback behavior remains deterministic with --live-fallback-policy dry-run-on-error.
  • Integration tests cover at least one successful streaming live path.
  • Docs updated (README + docs/live-mode-operations.md + docs/upgrades.md + CHANGELOG.md).

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureNew user-facing capability or enhancementmilestone:v1.0.0Tracking label for v1.0.0 roadmap workpriority:p1High-priority issue for current milestone

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions