Skip to content

How to use? I just get errors #1

@Disonantemus

Description

@Disonantemus

I was able to compile, but ...

Models downloaded and a wav test file:

$ du -ah
2.2G	./mistralai_Voxtral-Mini-3B-2507-IQ4_XS.gguf
683M	./mmproj-Voxtral-Mini-3B-2507-Q8_0.gguf
2.4G	./mistralai_Voxtral-Mini-3B-2507-Q4_K_M.gguf
5.1M	./1.wav
5.1G	.

Using only test file: doesn't find previously downloaded models:

$ voxtral-cli 1.wav
Select model (↑/+ Enter):A2TB/Voxtral/models:
  ▶ q4 - Mini Q4 (2.3GB) - Fast, good quality
    q8 - Mini Q8 (4.0GB) - Better quality
    bf16 - Mini BF16 (7.5GB) - Full precision
    small-q4 - Small Q4 (14GB) - Best quality, slower
    small-q8 - Small Q8 (25GB) - Maximum quality
Selected: q4
Error: Model not found: models/Voxtral-Mini-3B-2507-Q4_K_M.gguf
Download models from HuggingFace:
  https://huggingface.co/bartowski/mistralai_Voxtral-Mini-3B-2507-GGUF

Adding model flag: ask for projector

$ voxtral-cli -m mistralai_Voxtral-Mini-3B-2507-Q4_K_M.gguf 1.wav
Error: Multimodal projector not found: models/mmproj-Voxtral-Mini-3B-2507-Q8_0.gguf

Adding projector file (flag not --help or README.md): still don't find it

$ voxtral-cli -m mistralai_Voxtral-Mini-3B-2507-Q4_K_M.gguf --mmproj mmproj-Voxtral-Mini-3B-2507-Q8_0.gguf 1.wav
Error: Multimodal projector not found: models/mmproj-Voxtral-Mini-3B-2507-Q8_0.gguf

Going one level up in folder tree: surprisingly doesn't show "projector not found", but some strange function error:

$ cd ..
$ voxtral-cli -m models/mistralai_Voxtral-Mini-3B-2507-Q4_K_M.gguf --mmproj models/mmproj-Voxtral-Mini-3B-2507-Q8_0.gguf models/1.wav
Loading model...
llama_model_load_from_file_impl: no backends are loaded. hint: use ggml_backend_load() or ggml_backend_load_all() to load a backend before calling this function
Error: failed to load model
  • How it is used this tool?

    I can do transcription with llama-mtmd-cli, but not this, what I'm doing wrong?

System

distro: Arch Linux x86_64
kernel: 6.12.33-1-lts
shell: bash 5.3.9
term: foot
cpu: Intel i7-4790 (8) @ 3.600GHz
gpu: NVIDIA GeForce GTX 1660 SUPER

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions