AI Mario makes little progress in level.

[Dabnsky]

Task: Improve Path Finding and Progress in Mario RL Agent

Current Problem

The agent does not make good progress through the level. It tends to wander, hop in place, or get stuck, rather than following the intended path or making meaningful forward progress. The current reward system only weakly incentivizes exploration and does not provide a clear signal for path following or directional movement.

Changes Needed

1. Increase Frame Resolution:
   • Use a higher observation resolution (e.g., 128x128) to better capture level details and visible paths.
2. Finer Grid for Exploration:
   • Reduce the grid cell size (e.g., from 10x10 to 5x5 pixels) to allow more granular tracking of Mario's position and progress.
3. Path Detection:
   • Implement edge or line detection (e.g., Hough transform) to identify prominent paths or platforms in the level.
4. Directional Progress Reward:
   • Add a reward for net movement in the intended direction (e.g., rightward for most Mario levels).
5. Configurable Parameters:
   • Make resolution and grid size easily adjustable for experimentation.

Relevant Code Locations

• env/mario_env.py
  • capture_frame(): Handles frame capture and resizing. Update to use higher resolution.
  • step() and calculate_reward(): Main logic for agent actions and reward calculation. Update to use finer grid and add directional/path rewards.
  • detect_walkable_area(), is_position_walkable(): Used for walkable area detection and exploration tracking. May need to integrate path detection logic.
• vision/interpret_frame.py
  • Add or update functions for edge/line detection to support path identification.
• rewards/rewards.py
  • If reward logic is further modularized, add new reward functions for path following and directional progress.

High-Level Overview

• The environment captures higher-resolution frames and processes them with improved vision utilities to detect paths.
• The agent's position is tracked on a finer grid, and rewards are given for moving along detected paths and making forward progress.
• All changes are modular, allowing for easy adjustment and experimentation with different path-finding and reward strategies.
• The overall goal is to provide the agent with a clearer signal for following the intended level path, resulting in more purposeful exploration and progress.