I told Cursor to play Tetris by watching a TV and physically manipulating an NES controller...

---

TetrAIs

A self-contained rig that plays NES Tetris on real hardware. A Raspberry Pi watches the TV through a camera, runs a heuristic AI to decide piece placement, and physically presses the NES controller buttons with solenoids.

  Camera ──► Pi (CV + AI) ──► Solenoids ──► NES Controller
     ▲                                          │
     │            NES Console ◄─────────────────┘
     └──────────── TV Screen ◄───── video out

How It Works

1. Capture — A Pi Camera Module 3 Wide captures the TV at ~30 fps.
2. Vision — The frame is perspective-corrected to NES resolution (256×240), then parsed into a 20×10 board grid, current piece, and next piece using HSV colour thresholds.
3. Solver — A heuristic engine enumerates all valid placements, scores them (aggregate height, line clears, holes, bumpiness), and picks the best one.
4. Controller — GPIO pins drive solenoids through a ULN2803A Darlington array, physically pressing the NES controller's D-pad, A, B, and Start buttons.

Hardware

See hardware/bom.md for the full parts list (~$125 total).

The frame is 3D-printed from five OpenSCAD models in hardware/models/:

Part	Description
base_plate.scad	Controller cradle with standoff and bracket mounts
top_plate.scad	Solenoid mount plate positioned above each button
camera_arm.scad	250 mm vertical post with tilting camera platform
finger_cap.scad	Plunger extensions (straight and angled D-pad variants)
pi_mount.scad	L-bracket for Raspberry Pi 4

Build instructions: hardware/assembly.md Wiring diagram: hardware/wiring.md

Software Setup

On the Raspberry Pi

sudo apt update
sudo apt install -y python3-picamera2 python3-gpiozero python3-opencv python3-numpy

git clone <repo-url> ~/tetrais
cd ~/tetrais

Calibration

Position the rig in front of the TV with NES Tetris on screen, then:

python3 calibrate.py

Click the four screen corners (TL → TR → BR → BL). The homography matrix is saved to calibration.json.

Running

Dry run (no GPIO, prints moves to console):

python3 -m src.main --dry-run

Live:

python3 -m src.main

Testing

Solver and vision tests run on any machine (no Pi required):

pip install numpy opencv-python-headless pytest
python -m pytest tests/ -v

Project Structure

tetrais/
  calibrate.py              One-time screen calibration tool
  calibration.json          Generated homography (not committed)
  requirements.txt          Python dependencies
  src/
    config.py               GPIO pins, screen geometry, piece data, weights
    capture.py              Camera init + perspective warp
    vision.py               Board/piece parsing from warped frames
    solver.py               Heuristic AI + move generation
    controller.py           Solenoid GPIO driver
    main.py                 Main game loop
  tests/
    test_solver.py           Solver unit tests (34 cases)
    test_vision.py           Vision parser tests
  hardware/
    bom.md                  Bill of materials
    assembly.md             Step-by-step build guide
    wiring.md               GPIO ↔ ULN2803A ↔ solenoid pin map
    models/                 OpenSCAD 3D-printable parts