Add Kimi-K2-Instruct-0905 contrib model (1T MoE on trn2.48xlarge)#131

Open

jimburtoft wants to merge 10 commits intoaws-neuron:mainfrom

jimburtoft:contrib/kimi-k2-instruct-0905

Commits on Apr 17, 2026

Add Kimi-K2-Instruct-0905 contrib model
jimburtoft
committed
Switch to LNC=2/TP=32/EP=2 config (53% faster: 5.2 tok/s vs 3.4 tok/s)
jimburtoft
committed
Update benchmark results: 165.5ms TPOT, 6.0 tok/s at LNC=2 (76% faster than LNC=1)
jimburtoft
committed

Commits on Apr 19, 2026

Add SDK 2.29 compatibility notes and workaround for blockwise MoE CTE issue
jimburtoft
committed

Commits on Apr 20, 2026

Document batch size finding: NxDI processes sequences sequentially, BS>1 provides no throughput benefit
jimburtoft
committed

Commits on Apr 23, 2026

Update SDK 2.29 notes: confirm blockwise kernel padding produces depressed logits, EP workaround verified correct
jimburtoft
committed

Commits on Apr 25, 2026

Identify root cause: SDK 2.29 removed shard_hidden kernel, replacement CTE kernels produce wrong output with EP=2
jimburtoft
committed

Commits on Apr 26, 2026

Document shard_hidden kernel porting attempt: NKI 0.3.0 fixes and compiler BIR verifier bug on SDK 2.29
jimburtoft
committed

Commits on Apr 30, 2026

Switch to TP=64 EP=1 LNC=2 with selective loading: 2x throughput improvement
jimburtoft
committed

Commits on May 1, 2026

Switch to per-channel FP8: 3.5x throughput improvement (41.1ms TPOT, 24.3 tok/s at seq_len=1024)
jimburtoft
committed