evict largest sequence on KV exhaustion and retry with deferred-commi…

[mcharytoniuk]

…t scheduler loop

[Copilot]

Pull request overview

Updates the continuous batch scheduler to handle KV cache exhaustion by evicting the largest sequence and retrying decode without unbounded recursion, and adjusts/streamlines related model tests and build targets.

Changes:

• Reworked ContinuousBatchScheduler::execute_one_iteration to use a retry loop with “deferred commit” of per-request state until decode succeeds.
• Added a KV-pressure eviction regression test and reformatted several existing continuous-batch tests.
• Tweaked build ergonomics in the Makefile (new build / build.cuda targets) and simplified KV-cache-dtype tests.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 3 comments.

Show a summary per file

File	Description
paddler_model_tests/tests/continuous_batch_shutdown_during_generation.rs	Removed a shutdown-during-generation test.
paddler_model_tests/tests/continuous_batch_clean_shutdown.rs	Removed a clean-shutdown-after-inference test.
paddler_model_tests/tests/continuous_batch_partial_offload.rs	Formatting-only changes to the partial-offload test.
paddler_model_tests/tests/continuous_batch_mixed_multimodal_plain.rs	Formatting-only changes to mixed plain/multimodal test.
paddler_model_tests/tests/continuous_batch_concurrent_multimodal.rs	Formatting-only changes to concurrent multimodal test.
paddler_model_tests/tests/continuous_batch_kv_pressure_eviction.rs	Added regression test targeting KV pressure eviction behavior.
paddler_model_tests/tests/continuous_batch_kv_cache_dtypes.rs	Reduced KV-cache-dtype coverage to a single test case.
paddler_integration_tests/tests/agent_cuda_clean_shutdown.rs	Formatting-only assertion change.
paddler/src/agent/continuous_batch_scheduler.rs	Core scheduler logic change: retry loop + deferred state commit after successful decode; eviction on NoKvCacheSlot.
paddler/src/agent/continuous_batch_embedding_processor.rs	Removed a redundant KV cache clear (still cleared per embedding-batch decode).
Makefile	Added build/build.cuda targets; adjusted target placement.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

[Copilot]

Pull request overview

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.