Hands On Waidrin

[accemlcc]

Hey Pew, the AI coder and I thought we'd pimp Waidrin a little:

• Main feature is dynamic image generation with Flux 2 Klein via ComfyUI
• Every NPC gets an avatar
• Avatars are embedded in scenes via multi-reference Kontext conditioning
• Each dialogue turn generates a scene illustration
• Genre selection (Fantasy / Sci-Fi) – influences prompts, races, image style
• D&D-lite RPG stats (STR/DEX/CON/INT/WIS/CHA + HP) with genre-specific enforcement
• Language selection at the beginning → prompts are automatically translated
• Random character/world generation with re-roll dice button
• Optional custom appearance text field for the protagonist
• State JSON and the rest remain largely original (minor schema addition for image references)

I think that's pretty cool for immersion. The images are still a bit cringe, but that should be manageable.

Disadvantages:

Currently only runs with multi-GPU (1× llama.cpp / 2× ComfyUI), though single-GPU would be theoretically possible with sequential loading
Image generation lags slightly behind the text
The question is: are you interested in a PR? The changes to Waidrin are not insignificant, and until it's cleanly integrated, it could be a minor undertaking ;-)

The multi-GPU setup in particular is likely to be a deal breaker for most people. So the question of effort/return arises – ultimately, I don't care, I can just make a fork at some point.

[accemlcc]

I've debugged all of this a little further. In my opinion, there is one fundamental problem: corporate speech/corporate image generation.

1. Speech/text generation: even with my completely liberated Mistral Small, people are all nice, smiling, their eyes are shining, the scenes are harmless, etc. -> I think this could almost certainly be solved with prompting.

2. Image generation with Flux 2: It's an absolutely fantastic model, the embedding capabilities are unique. BUT: as soon as you activate digital art/painting, it immediately falls back into its “safe-mode comic princess look.” All prompting doesn't help there. The only thing that would help would be a LORA that basically sets the “dark fantasy sci-fi look” as the default.

There are more problems in the current state, but these are the two main ones at the moment. Let's see, maybe I'll make a LORA. At the moment, it's more of a proof of concept; it would work if. From that perspective, IMO, it's not worth the PR.