Optimize carousel video preview switching: prime/pause/center-mode pipeline management

[Copilot]

Fullscreen crossfade carousels (center playing, left/right primed-paused) suffered from slow first-frame latency and unnecessary audio decode on non-center slots. Pipeline teardown on every navigation prevented instant resume.

GStreamerVideo — new carousel APIs

• prime(file, enableAudio=false) — prerolls a slot to GST_STATE_PAUSED with audio decode conditionally excluded. Sets GST_PLAY_FLAG_AUDIO on the playbin before the URI is applied so the decode chain is never built for side slots. Same session-ID / lifetime gating as play().
• setCenterMode(bool) — manages center↔side audio transitions without pipeline teardown:
  • true (side→center): enqueues flags update to VIDEO|AUDIO + brief READY→PAUSED cycle to activate the audio chain. Handles the resume()-before-task race by checking targetState_ == Playing and continuing to GST_STATE_PLAYING if needed.
  • false (center→side): synchronously zeros AudioBus gain — no pipeline disruption.
• hibernate() — GST_STATE_READY + full appsink drain + bus flush + audio mute, for slots no longer adjacent.

GStreamerVideo — pipeline fixes

• element-setup connected unconditionally (was SW-decode path only). multiqueue use-interleave=FALSE and queue caps now apply in HW accel mode too. Decoder-specific tweaks inside the callback remain conditional on !HardwareVideoAccel.
• Flags: GST_PLAY_FLAG_BUFFERING and GST_PLAY_FLAG_TEXT now explicitly cleared in all flag-setting paths.
• unload() appsink drain: both sinks drained after GST_STATE_READY to prevent stale frames flashing on the next play().

ScrollingList / VideoComponent wiring

notifyVideoCenterMode() iterates all slots and calls VideoComponent::setCenterMode(i == selectedOffsetIndex_) after both initial allocateSpritePoints() and every components_.rotate() in scroll(). VideoComponent passes the call through to the underlying GStreamerVideo via dynamic_cast.

// Side slot: audio decode skipped, prerolls fast
video->prime("/path/to/game.mp4", /*enableAudio=*/false);

// When it scrolls to center:
videoComponent->setCenterMode(true);   // re-enables audio pipeline
// When it leaves center:
videoComponent->setCenterMode(false);  // gain → 0, pipeline untouched
// When scrolled far away:
video->hibernate();                    // READY + drain

Original prompt

Repository: CoinOPS-Official/RetroFE (branch: current)

Goal
You are optimizing video preview switching behavior for fullscreen crossfading previews using multiple simultaneous GStreamerVideo instances (left/center/right). The UI shows a center fullscreen video playing; left and right videos are paused offscreen and should resume instantly when they become center. Crossfade is done visually (SDL textures) and audio should fade between videos.

Task
Investigate the existing implementation of GStreamerVideo (GStreamerVideo.cpp/.h) and the higher-level code that orchestrates preview video instances (likely a menu/view/carousel component). Then implement improvements for fastest first-frame and instant resume for neighbor videos while maintaining correctness and avoiding memory leaks.

Requirements

1. Identify where the application switches videos and calls GStreamerVideo::play/unload/pause/resume/stop across the multiple instances.
2. Modify behavior so that left/right neighbor videos remain in GST_STATE_PAUSED (primed with preroll/sample staged) rather than being moved to GST_STATE_READY via unload() during normal navigation.
3. Introduce or refactor API(s) in GStreamerVideo to support "prime paused" behavior and a "hibernate" (READY) behavior for far-away videos.
   • Example: add methods like prime(const std::string& file, bool enableAudio), setCenterMode(bool), or similar. Exact naming up to you.
   • Must preserve thread-safety/lifetime gating (session IDs, AsyncState alive flag, CallbackCtx self atomic).
4. Ensure multiqueue tuning (element-setup callback) applies in both HW accel and SW paths:
   • Currently element-setup is only connected when !Configuration::HardwareVideoAccel; change to connect unconditionally (once per pipeline), while keeping decoder-specific property tweaks conditional.
   • Keep multiqueue property use-interleave set to FALSE when available.
5. Implement per-instance mode for audio:
   • Center instance: audio enabled and fades in.
   • Non-center instances: audio disabled (or muted) to reduce preroll latency and CPU.
   • Use playbin3 flags to disable audio where appropriate (GST_PLAY_FLAG_AUDIO off).
6. Avoid unnecessary pipeline rebuilds for navigation; keep pipeline per instance and switch URIs only when assigning a new file to a slot.
7. Ensure no regressions in cleanup:
   • stop() remains full teardown.
   • unload()/hibernate should still prevent memory growth (READY or equivalent) when instance is not needed.
   • When switching URIs, ensure appsinks are drained and stagedSample_ cleared so stale frames don’t flash.
8. Add logging to make it easy to understand transitions (center->side, side->center, prime start/complete, etc.).

Deliverables

• A pull request against branch current implementing the changes.
• Update any relevant documentation/comments if needed.

Notes

• The user wants the changes for fastest first frame and instant resume in a carousel-like UI.
• Prefer SDL crossfade using existing textures; do not try to synchronize pipeline clocks.

The following is the prior conversation context from the user's chat exploration (may be truncated):

User: anything i should be doing here to facilitate faster play/unload/play cycles when switching videos?

Assistant: Yes—your current design is already pointed in the right direction (keep a single playbin3 alive and “soft reset” between files), but there are a few concrete things you can do to make play → unload → play cycles faster and more deterministic.

1) Prefer a single “reset to READY + flush + drain” path, not multiple teardown styles

Right now you have two different cleanup modes:

• unload() keeps the pipeline and moves it to GST_STATE_READY (good for fast switching)
• stop() goes full teardown (GST_STATE_NULL, destroy watch source, unref pipeline, destroy textures, etc.) (slow)

If “switch videos” is the hot path, make sure the UI uses unload() (or a new resetForNextMedia()), and reserve stop() only for final shutdown.

Key detail: after GST_STATE_READY, also drain both appsinks and clear your probe-side caches so you don’t carry stale samples into the next play.

• You already drain in stop() via detachAndDrainSink()
• You do not drain in unload()

Recommendation: reuse detachAndDrainSink() in unload() (without removing probes if you want), and always clear stagedSample_ (you already do) plus drain appsink queues to avoid leftover preroll/sample being delivered immediately on next file.

2) When switching files, set pipeline to READY before setting a new URI

For fastest and cleanest transitions, the typical sequence is:

1. gst_element_set_state(pipeline, GST_STATE_READY)
2. flush bus (optional)
3. set new "uri"
4. gst_element_set_state(pipeline, GST_STATE_PAUSED) (for preroll)
5. later PLAYING when needed

You mostly do this across unload() and play(), but they’re separated and asynchronous, so you can end up with:

• a play() arrivin...

This pull request was created from Copilot chat.