🏛️ AlignmentBank

AlignmentBank is an ongoing repository of paper notes and selective code reproductions, focused on foundational LLM techniques, reasoning research, and, above all, alignment. It is a “bank” which aims to facilitate alignment-related knowledge flow with zero interest rate 😎.

The mission is to foster a continuously evolving, high-quality archive of both theoretical and empirical developments in LLM alignment. The broader vision is to scale this initiative into a community-oriented resource in the future.

The repository is actively maintained with weekly updates.

📂 Areas Covered (To be expanded)

Theme	Subtopics
Attention Mechanisms	MHA, MQA, SWA, GQA
LLM Architectures	Fundamentals, Implementations
LLM Family	GPT, LLaMA, Qwen, DeepSeek, etc.
LLM Alignment	Algorithms, Frameworkds, LLM Reasoning etc.
xxxxx	xxxxx

📚 Table of Contents

• 🧠 Attention Mechanism

• 🏗️ LLM Architectures

  • Fundamentals
  • Implementations

• 🧬 LLM Family

  • GPTs
  • Llamas
  • Qwens
  • DeepSeek
  • Mistral
  • AI2
  • Google

• 🎯 LLM Alignment

  • Algorithms

    • Supervised Fine-Tuning
    • Reinforcement Learning

  • Frameworks

  • LLM Reasoning

    • Verifiers
    • Proposer
    • Planning

🧠 Attention Mechanism

• Scaled dot-product self-attention / Multi-Head Attention (MHA): Attnetion is all you need

• Multi-Query Attention (MQA): Fast Transformer Decoding: One Write-Head is All You Need

• Sliding Window Attention (SWA): Longformer: The Long-Document Transformer

• Grouped-Query Attention: Training Generalized Multi-Query Transformer Models from Multi-Head Checkpoints

🏗️ LLM Architectures

Fundamentals

• RoFormer: Enhanced Transformer with Rotary Position Embedding (Su et al., 2021)

• GLU Variants Improve Transformer (Noam Shazeer, 2020)

Implementations

• Reproducing dense models with GPT-2 as an example (largely inspired by Andrej Karpathy's video)

• Reproducing MoE models

🧬 LLM Family

GPTs

• GPT: Improving Language Understanding by Generative Pre-Training (Radford et al., 2018)

• GPT-2: Language Models are Unsupervised Multitask Learners (Radford et al., 2019)

• GPT-3: Language Models are Few-Shot Learners (Brown et al., 2020)

• InstructGPT: Training language models to follow instructions with human feedback (Ouyang et al., 2022)

Llamas

• LLaMA: Open and Efficient Foundation Language Models (Touvron et al., 2023)

• Llama 2: Open Foundation and Fine-Tuned Chat Models (GenAI, Meta, 2023.07)

• Llama 3

• Llama 4

Qwens

• Qwen Technical Report

• Qwen2 Technical Report

• Qwen2.5 Technical Report

• Qwen3

DeepSeek

• DeepSeek LLM Scaling Open-Source Language Models with Longtermism

• DeepSeekMoE: Towards Ultimate Expert Specialization in Mixture-of-Experts Language Models

• DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

• DeepSeek-V3

• DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning

Mistral

• Mistral 7B

• Mixtral of Experts

AI2

• OLMo

• 2OLMo 2 Furious

Google

• Gemma

• Gemma 2

🎯 LLM Alignment

Algorithms

Supervised Fine-Tuning

• Injecting New Knowledge into Large Language Models via Supervised Fine-Tuning (Mecklenburg et al., 2024)

Reinforcement Learning

• PPO: Proximal Policy Optimization Algorithms (Schulman, et al., 2017)

• DPO: Direct Preference Optimization: Your Language Model is Secretly a Reward Model (Rafailov et al., 2023)

• GRPO:

• DAPO:

• VAPO:

Frameworks

• Megatron

  • Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism (Shoeybi et al., 2020))

LLM Reasoning

Verifiers

• Training Verifiers to Solve Math Word Problems (Cobbe et al., 2021)

• Solving math word problems with process and outcome-based feedback (Ursato et al., 2022)

• Let’s Verify Step by Step (Lightman et al., 2023)

• Improve Mathematical Reasoning in Language Models by Automated Process Supervision (Luo et al., 2024)

• Generative Verifiers: Reward Modeling as Next-Token Prediction

Proposer

• REFT: Reasoning with REinforced Fine-Tuning (Luong el al., 2024)

• Logic-RL: Unleashing LLM Reasoning with Rule-Based Reinforcement Learning (Xie et al.)

Planning

• Reasoning with Language Model is Planning with World Model (Hao et al., 2023)

• Tree of Thoughts: Deliberate Problem Solving with Large Language Models