Skip to content

PAI review: install fixes, numerical correctness, MLA cache, generate hardening (+ roadmap) #45

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
baseflux wants to merge 10 commits into
base: main
Choose a base branch
from
Open

PAI review: install fixes, numerical correctness, MLA cache, generate hardening (+ roadmap) #45

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
10 commits
Select commit Hold shift + click to select a range
327a73b
fix(install): unpin torch, add loguru, trim public API, fix README va…
Apr 22, 2026
9e62d6c
feat(config): add MythosConfig.__post_init__ validation
Apr 22, 2026
9b00ea2
fix(numerics): LTI fp32, loop-index fp32, ACT remainder, tie-after-init
Apr 22, 2026
fae15a0
feat(generate): bounds check, eval mode, EOS stopping
Apr 22, 2026
116acba
fix(mla): cache shared k_rope once, broadcast per-head at compute time
Apr 22, 2026
69e2ed7
test: add PAI-review regression tests + fix Cyrillic test class name
Apr 22, 2026
4465f78
fix(tests): slice full freqs_cis to T in unit tests
Apr 22, 2026
79c6ec3
chore: gitignore uv.lock (tool artifact, not part of project)
Apr 22, 2026
0d04570
docs: add PAI review roadmap (landed + deferred tiers)
Apr 22, 2026
f0e05b2
perf: F.scaled_dot_product_attention + gradient checkpointing
Apr 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -428,3 +428,4 @@ flycheck_*.el
# network security
/network-security.data

uv.lock
502 changes: 502 additions & 0 deletions PAI_REVIEW_2026-04.md
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -114,7 +114,7 @@ from open_mythos import (
OpenMythos,
)

cfg = mythos_7b() # returns a MythosConfig
cfg = mythos_3b() # returns a MythosConfig
model = OpenMythos(cfg)

total = sum(p.numel() for p in model.parameters())
Expand Down
35 changes: 2 additions & 33 deletions open_mythos/__init__.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,20 +1,4 @@
from open_mythos.main import (
ACTHalting,
Expert,
GQAttention,
LoRAAdapter,
LTIInjection,
MLAttention,
MoEFFN,
MythosConfig,
OpenMythos,
RecurrentBlock,
RMSNorm,
TransformerBlock,
apply_rope,
loop_index_embedding,
precompute_rope_freqs,
)
from open_mythos.main import MythosConfig, OpenMythos
from open_mythos.tokenizer import MythosTokenizer
from open_mythos.variants import (
mythos_1b,
Expand All @@ -28,28 +12,13 @@

__all__ = [
"MythosConfig",
"RMSNorm",
"GQAttention",
"MLAttention",
"Expert",
"MoEFFN",
"LoRAAdapter",
"TransformerBlock",
"LTIInjection",
"ACTHalting",
"RecurrentBlock",
"OpenMythos",
"precompute_rope_freqs",
"apply_rope",
"loop_index_embedding",
"MythosTokenizer",
"mythos_1b",
"mythos_3b",
"mythos_10b",
"mythos_50b",
"mythos_100b",
"mythos_500b",
"mythos_1t",
"load_tokenizer",
"get_vocab_size",
"MythosTokenizer",
]
Loading