ESP32-S3 AVI Video Player for Waveshare 4.3" Touch LCD

This project demonstrates how to play MJPEG-encoded AVI video files from an SD card on the Waveshare ESP32-S3 4.3" Touch LCD (B variant) using the ESP-IDF framework. The player is optimized to resolve common issues like screen tearing, color distortion, and slow playback, providing a stable foundation for multimedia applications on this hardware.

The player includes a benchmarking tool to measure real-time decoding performance (Frames Per Second).

Key Features

• Video Playback: Plays 800x480 AVI files with MJPEG video streams.
• Target Hardware: Specifically configured for the Waveshare ESP32-S3 4.3" Touch LCD (B).
• Optimized Performance: All code and configuration settings are tuned to maximize playback speed and stability.
• Built-in Benchmarking: Includes an FPS counter to measure the board's decoding performance.
• Robust Display Driver: Uses the ESP-IDF esp_lcd component with a stable RGB timing configuration.

Hardware Required

• Waveshare ESP32-S3 4.3" Touch LCD (B)
• MicroSD Card: A reasonably fast card (Class 10 or better is recommended).

Software Prerequisites

• ESP-IDF v5.5.1: This project is built and tested against this specific version of the Espressif IoT Development Framework.

Getting Started

1. Clone the Repository

git clone https://github.com/KiranPranay/waveshare-esp32s3-4.3b-avi-player.git
cd waveshare-esp32s3-4.3b-avi-player

2. Prepare the SD Card

• Format the MicroSD card to FAT32.
• Create a directory named videos in the root of the SD card.
• Place your converted AVI files inside the videos directory.

3. Convert Your Video File

To ensure smooth playback, you must convert your video files (e.g., .mp4, .mov, etc.) into the specific AVI/MJPEG format. Use the following ffmpeg command:

ffmpeg -i "your_input_video.mp4" -c:v mjpeg -q:v 2 -vf "fps=15,scale=800:480" -an sample.avi

Explanation of the parameters:

• -c:v mjpeg: Sets the video codec to Motion JPEG.
• -q:v 2: Uses very low compression for faster decoding (larger files, but faster).
• fps=15: Sets the framerate to 15 FPS, which is a realistic target for this hardware at 800x480 resolution.
• scale=800:480: Resizes the video to match the display.
• -an: Removes the audio track.

4. Build and Flash the Project

idf.py build
idf.py flash monitor

Performance & Limitations

While the ESP32-S3 is a powerful microcontroller, it lacks a dedicated hardware video decoder. The MJPEG video decoding is done in software using the CPU cores.

• CPU-Bound: The primary bottleneck is the CPU's ability to decode 800x480 JPEG frames in real-time.
• Expected Performance: With optimizations in place, you can expect a playback speed of around 12-15 FPS, depending on the video settings.
• Why not 30 FPS?: The board will decode at a maximum speed of approximately 15 FPS. Attempting to play a video with a higher framerate (e.g., 30 FPS) will result in slow-motion playback.

Troubleshooting Common Issues

1. Video is Flickering, Tearing, or Rolling (Horizontal/Vertical)

This is usually due to display timing issues.

• Solution 1: Ensure the timing values in main/panel_rgb.h are correct. The stable values for this board are:

  #define HSYNC_PW       48
  #define HSYNC_BP       46
  #define HSYNC_FP       40
  #define VSYNC_PW       3
  #define VSYNC_BP       23
  #define VSYNC_FP       22

• Solution 2: Enable VSYNC Restart. This software fix addresses timing drift. In idf.py menuconfig, go to Component config -> ESP LCD -> RGB LCD and enable the option Restart RGB controller in VSYNC interrupt.

2. Video Colors are Distorted or Incorrect

This is typically a color byte ordering issue.

• Solution: In main/main.c, inside the on_video_frame function, ensure the color swap flag is set to false:

  jpeg_cfg.flags.swap_color_bytes = false;

3. Video Plays in Slow Motion

This is a classic CPU performance bottleneck.

• Solution 1: Optimize the Video File: Use the ffmpeg command provided in the "Getting Started" section with -q:v 2 or -q:v 3 for faster decoding.

• Solution 2: Maximize CPU Performance: In idf.py menuconfig, ensure the following settings:

  • CPU Frequency: Set to 240 MHz.
  • Compiler Optimization Level: Set to -O2 (Optimize for performance).

• Solution 3: Increase I/O Speeds (if applicable).

Conclusion

This project demonstrates the capabilities of the ESP32-S3 when optimized for video playback. With the right configuration and optimizations, the system can handle MJPEG video streams efficiently, providing a reliable solution for multimedia applications.

For further improvements or contributions, feel free to open issues or pull requests in the repository.

---

Note: Ensure all videos are converted to the specified MJPEG format for optimal performance.