[Enhancement] Add Snake Environment - Classic Single-Agent RL Game

[AlirezaShamsoshoara]

Add Snake Environment - Classic Single-Agent RL Game

Summary

This PR adds a new Snake Environment to OpenEnv, providing a classic snake game implementation based on marlenv's Snake-v1. The environment offers a clean, OpenEnv-compatible interface for reinforcement learning research and experimentation with a well-known game mechanic.

What's New

Core Environment (src/envs/snake_env/)

• Client API (client.py): HTTP client for connecting to snake environment servers
• Data Models (models.py): Type-safe action and observation models
  • SnakeAction: Discrete actions (turn left/right or directional movement)
  • SnakeObservation: Rich observations including grid state, score, and metadata
• Server Implementation (server/):
  • FastAPI-based HTTP server
  • Wraps marlenv Snake-v1 with OpenEnv interface
  • Configurable grid size, vision range, and reward functions

Example Code

• examples/snake_simple.py: Comprehensive example demonstrating:
  • Docker and local server modes
  • Automated gameplay with random/simple policies
  • Multi-episode training loops with statistics
  • Real-time visualization with matplotlib
  • Performance tracking and analysis

Infrastructure

• Docker Support: Full containerization with automated builds
  • Dockerfile for snake environment
  • Updated CI/CD workflow for automated image builds
• Documentation: Extensive README with:
  • Quick start guide
  • API reference
  • Configuration options
  • Troubleshooting tips

Key Features

Gameplay

• Grid-based environment: Configurable grid size (default: 20×20)
• Classic mechanics: Navigate, eat fruits, grow, avoid walls and self-collision
• Two control modes:
  • Snake mode: Relative actions (turn left/right)
  • Human mode: Global directions (up/down/left/right)

Configurability

• Adjustable grid dimensions
• Optional partial observability (vision range)
• Customizable reward function for different learning objectives
• Configurable episode length

Observations

Each step provides:

• Full grid state (2D array)
• Encoded observation (supports vision range)
• Episode statistics (score, steps, fruits collected)
• Alive status

Deployment Options

• Docker: One-command deployment with SnakeEnv.from_docker_image()
• Local server: Direct server connection for development
• Web interface: Optional browser-based interaction

Example Usage

from envs.snake_env import SnakeAction, SnakeEnv

# Start from Docker
client = SnakeEnv.from_docker_image("snake-env:latest")

# Reset and play
result = client.reset()
while not result.done:
    action = SnakeAction(action=0)  # Go straight
    result = client.step(action)
    print(f"Score: {result.observation.episode_score}")

client.close()

Files Changed

New Files

• src/envs/snake_env/ - Complete environment implementation
  • __init__.py, client.py, models.py
  • server/app.py, server/snake_environment.py, server/Dockerfile
  • README.md, pyproject.toml, openenv.yaml
• examples/snake_simple.py - Interactive example with visualization (378 lines)

Modified Files

• .github/workflows/docker-build.yml - Added snake-env to build matrix
• .gitignore - Updated to exclude uv.lock files more broadly

Technical Details

Dependencies

• marlenv>=1.0.0 - Core snake game implementation
• gym==0.24.1 - Required by marlenv
• numpy>=1.24.0 - Grid and array operations
• Standard OpenEnv dependencies (FastAPI, Pydantic, Uvicorn)

Architecture

The implementation follows OpenEnv's standard pattern:

1. Environment server wraps marlenv Snake-v1
2. HTTP API exposes reset/step/state endpoints
3. Type-safe client provides Pythonic interface
4. Docker containerization for easy deployment

Reward Customization

Supports flexible reward shaping:

reward_dict = {
    'fruit': 1.0,      # Eating fruit
    'lose': -1.0,      # Death penalty
    'time': 0.0,       # Per-step reward/penalty
    'kill': 0.0,       # Multi-agent (unused in single-agent)
    'win': 0.0,        # Multi-agent (unused in single-agent)
}

Testing

The example script demonstrates:

• Docker container startup and connection
• Local server connection
• Reset and step operations
• State tracking across episodes
• Reward collection and scoring
• Multi-episode statistics gathering
• Real-time visualization

To test:

# Docker mode
python examples/snake_simple.py --mode docker --play-mode auto

# Local server mode (terminal 1)
cd src/envs/snake_env && uv run --project . server

# Local server mode (terminal 2)
python examples/snake_simple.py --mode local --play-mode multi --episodes 10

Future Enhancements

Potential additions for future PRs:

• Pre-trained baseline agents
• Additional reward shaping examples
• Multi-agent support (multiple snakes)
• Advanced visualization options
• Performance benchmarks

Credits

Based on marlenv by ML2 (KC-ML2)

---

Lines of Code: ~1,246 additions across 14 files
Documentation: Comprehensive README with examples and troubleshooting
CI/CD: Automated Docker builds integrated

[github-actions]

✅ Validation succeeded for snake_env

Your env passes the vibe check. However, most environments should go straight to the hub, they will automatically be added to the official Env Hub collection on a nightly basis. Environments in the official specification repo are only meant to demonstrate usage of a specific spec feature for educational purposes. Re-run locally with:

• openenv validate --verbose src/envs/snake_env

[OK] snake: Ready for multi-mode deployment

Supported deployment modes:
  [YES] docker
  [YES] openenv_serve
  [YES] uv_run
  [YES] python_module

Usage examples:
  cd snake_env && uv run server
  cd snake_env && openenv build
  cd snake_env && openenv push

You can deploy the environment to Hugging Face Spaces by running openenv push.

• Success run: https://github.com/meta-pytorch/OpenEnv/actions/runs/19521160596

[AlirezaShamsoshoara]

cc @init27 @HamidShojanazeri @Darktex (Since I cannot add a reviewer directly)

[burtenshaw]

Thanks for the great contribution @AlirezaShamsoshoara ! Can you deploy it to the hf hub with openenv push and drop a link on this PR.

[AlirezaShamsoshoara]

Thank you @burtenshaw for taking a look at this! Just ran the openenv push from the snake_env directory and changed a little bit of files to make it compatible with the deployment and the web interface!
The environment is available here and seems running fine:

https://huggingface.co/spaces/Crashbandicoote2/snake_env

[burtenshaw]

@AlirezaShamsoshoara great. Could you update docs/environments.md and then I think we can merge it.

[AlirezaShamsoshoara]

@burtenshaw Thanks for the comment! Just rebased it and added the snake env to docs/environments.md file!
Please let me know if anything else left on this.