Describe the bug
PlotBrain.plot_timesteps(..., show_stimuli=True) crashes when the input
video clip has no speech or no audio track, because
BasePlotBrain.plot_stimuli assumes both (a) get_audio(...) returns a
non-None object and (b) every segment.events dataframe contains a
type column (i.e. at least one Word event).
Both assumptions break in realistic workflows — e.g. screen recordings,
silent clips, or any video where WhisperX produces zero words. The model
itself handles the missing-modality case correctly (main.py auto-drops
the text extractor when no Word events exist), so this is strictly
a plotting-side bug.
Reproduction
import pandas as pd
from pathlib import Path
from tribev2.demo_utils import TribeModel, get_audio_and_text_events
model = TribeModel.from_pretrained("facebook/tribev2", cache_folder="./cache")
df = get_audio_and_text_events(
pd.DataFrame([{
"type": "Video", "filepath": "silent_clip.mp4",
"start": 0, "timeline": "default", "subject": "default",
}]),
audio_only=True,
)
preds, segments = model.predict(events=df)
model._plotter.plot_timesteps(preds[:5], segments=segments[:5], show_stimuli=True)
# AttributeError: 'NoneType' object has no attribute 'to_soundarray'
# OR KeyError: 'type' when events is None / missing the column
Expected behaviour
plot_stimuli should gracefully skip the audio waveform / word-overlay
subplots when those modalities are not present in the segments, mirroring
the model's existing "auto-drop missing extractor" behaviour.
Environment
OS: Ubuntu 24.04 aarch64 (NVIDIA DGX Spark, GB10)
Python 3.12.3
torch 2.11.0+cu130 (Blackwell sm_121)
Proposed fix
Two defensive guards in: tribev2/plotting/base.py::BasePlotBrain.plot_stimuli
Only plot the audio waveform when get_audio(...) returns non-None.
Only iterate words when segment.events is not None and the dataframe has a type column.
No change in behaviour when audio / words ARE present.
Describe the bug
PlotBrain.plot_timesteps(..., show_stimuli=True)crashes when the inputvideo clip has no speech or no audio track, because
BasePlotBrain.plot_stimuliassumes both (a)get_audio(...)returns anon-None object and (b) every
segment.eventsdataframe contains atypecolumn (i.e. at least oneWordevent).Both assumptions break in realistic workflows — e.g. screen recordings,
silent clips, or any video where WhisperX produces zero words. The model
itself handles the missing-modality case correctly (
main.pyauto-dropsthe
textextractor when noWordevents exist), so this is strictlya plotting-side bug.
Reproduction
Expected behaviour
plot_stimuli should gracefully skip the audio waveform / word-overlay
subplots when those modalities are not present in the segments, mirroring
the model's existing "auto-drop missing extractor" behaviour.
Environment
OS: Ubuntu 24.04 aarch64 (NVIDIA DGX Spark, GB10)
Python 3.12.3
torch 2.11.0+cu130 (Blackwell sm_121)
Proposed fix
Two defensive guards in:
tribev2/plotting/base.py::BasePlotBrain.plot_stimuliOnly plot the audio waveform when get_audio(...) returns non-None.
Only iterate words when segment.events is not None and the dataframe has a type column.
No change in behaviour when audio / words ARE present.