MiniGPT

A minimal and modular implementation of a GPT-style Transformer for text generation, built with PyTorch.

Features

• GPT-style architecture implemented from scratch
• Dataset loading via Hugging Face datasets
• Modular code structure for easy customization
• Integrated training and text generation workflows

Dataset

This project uses the WikiText-2 dataset by default, loaded with:

from datasets import load_dataset
dataset = load_dataset("wikitext", "wikitext-2-raw-v1")

The dataset is automatically cached by the Hugging Face datasets library.

Installation

Install dependencies using Conda:

conda env create -f environment.yaml
conda activate minigpt

Usage

You can run training and generation directly from the command line:

python MiniGPT/main.py     # Train the model
python MiniGPT/generation.py  # Generate text from trained model

For a step-by-step walkthrough in notebook form, see:

MiniGPT_Example.ipynb

It shows how to load configs, prepare data, train the model, and generate text interactively.

License

MIT