https://dnhkng.github.io/posts/rys/
https://dnhkng.github.io/posts/rys-ii/
Hello!
Please tell me, are you familiar with these works?
Perhaps you could team up with the author and try converting the Gemma 4 series models to the OpenMythos architecture?
I really miss a powerful yet compact model. Gemma 4 E4B is great for conversation, but it's incapable of writing proofs in Lean 4 even for the simplest algorithm...
Have you found a way to train this architecture with repeated layers?
https://dnhkng.github.io/posts/rys/
https://dnhkng.github.io/posts/rys-ii/
Hello!
Please tell me, are you familiar with these works?
Perhaps you could team up with the author and try converting the Gemma 4 series models to the OpenMythos architecture?
I really miss a powerful yet compact model. Gemma 4 E4B is great for conversation, but it's incapable of writing proofs in Lean 4 even for the simplest algorithm...
Have you found a way to train this architecture with repeated layers?