slow visualization llama cpp

[iChristGit]

latest v0.9.1 version of open-webui:

[iChristGit]

1.Used Qwen3.5-27B, Qwen3.6-Moe, Gemma4-31B.
Sometimes no visualization, sometimes its very very slow, always its outputs all closing text before even starting the visualization.
Not sure what can cause this, but seems kinda buggy

[Classic298]

pipe? cannot reproduce. The model IS ALLOWED to output the "closing" text before outputting the visualization, in fact, is prompted to do so - not a bug

any logs on frontend in browser console? can you share the chat? did the AI finish? is the HTML perhaps broken?

[iChristGit]

It might be my machine freezing , does it send multiple requests to the LLM? il try to debug this more to understand why its not fluid.
I have 3090Ti+64GB ram, il try limiting to 16k tokens and avoiding being close to vram exhaust.

[iChristGit]

Seems that the core issue is normal speeds for the text generation (100tk/s) and very slow speeds for the visualization for some reason (1-2tk/s)

I use llama cpp as backend

[Classic298]

llama cpp? Gemini on llama?

[Classic298]

and no its one continuous request

[ccdv-ai]

Have the same problem with qwen3.5 using vLLM (tool + skill enabled)
OWUI 0.9.1

[iChristGit]

llama cpp? Gemini on llama?

Its llama.cpp
Just called the Gemma4-31B model "Gemini" in the UI.

[iChristGit]

Il need to test of this is an issue with ollama as well.

Have you tested it on 0.9.1? And with non api models? (Not openai, google etc)

[Classic298]

so for me i tested with litellm, openrouter, openai and anthropic connections and models today and it always works fine. Without logs or reproduction steps it will be difficult for me to debug and fix this.

[iChristGit]

I updated llama cpp, disabled thinking and disabled all the title, tags, follow up features in openwebui and its now very fast.

Might be a model that goes into a loop once thinking enabled, because with 100tk/s thinking should not slow it down so much.

Will upload logs once I fully understand whats the issue.

[iChristGit]

Seems to be a thinking loop if I disable thinking visualization starts right away, with thinking its dependent on how long the model thinks.

Not an issue with the tool
Closing the issue