PSRAM DMA Mode does not work with JPEGs

[RubenKelevra]

Have you recently tested this mode?

success depends on different things. It does not work well for JPEG, but works fairly well for other modes. That is if XCLK is 16MHz... other sample clocks do not work well. You can see issues if PSRAM is otherwise heavily used too. Overall experimental feature

Yeah I think I found the issue why it's not working for JPEGs:

The test for the start of the byte marker for JPEGs does not work as far as I can tell, because the length field is not updated. So previous to #760 it would just try to read random bytes from the memory.

Now it's just returning because there's nothing to read, because the length is zero.

What needs to be done is, that if we get the first EOF for a completed DMA copy that we copy the first section from the PSRAM back into a temporary buffer in the SRAM, and give this temporary buffer to the SOI checker, so it can verify that there's a valid JPEG start.

I'll provide an PR to do this as soon as I get time for that.

Originally posted by @RubenKelevra in PR #774 (comment)

[jameszah]

Is this the problem that presents as follows (100 lines maybe), and then starts working normally ... but comes and goes ???

21:14:24.293 -> E (4204) cam_hal: DMA overflow
21:14:24.387 -> E (4292) cam_hal: DMA overflow
21:14:24.387 -> E (4332) cam_hal: DMA overflow
...
21:14:28.749 -> Camera Capture Failed

This is using old code with VGA frames at 25 fps, after 3.2.0 -> 3.3.0 upgrade, on esp32-cam ai-thinker.

[RubenKelevra]

@jameszah not sure what this version number is referring to, as this library is currently at version 2.1.0.

If you use 2.1.0, PSRAM DMA is only active if you select 16 MHz as XCLK on an ESP32S2 or ESP32S3.

If you use the master of this library, it's only active if you set CONFIG_CAMERA_PSRAM_DMA to 1 on an ESP32S2 or ESP32S3.

The module you're describing sounds like an ESP32S, which does not support psram_mode.

[RubenKelevra]

What you're describing sounds like you're not using psram_mode, but the master of this library and you're hitting this new check I've added:

#772

This check makes sure, that if the image the camera is providing is larger than our expectations how large an image should be, memory behind it isn't overwritten in a buffer overflow.

You can manually adjust the expected image size:

Set CONFIG_CAMERA_JPEG_MODE_FRAME_SIZE_AUTO to false and set CONFIG_CAMERA_JPEG_MODE_FRAME_SIZE to a size which fits the images of your camera plus a couple percent.

This way the buffer slots are larger than the images and the check isn't triggered.

[RubenKelevra]

@me-no-dev what do you think about improving the cam_obj->recv_size = cam_obj->width * cam_obj->height / 5; heuristic (line in code) by fetching an image from the camera instead, counting how large it really is, to set this size?

[jameszah]

Hi, I'm referring to esp32-arduino 3.2.0 -> 3.3.0, with the original esp32, with psram, but with the arduino code, so fewer compile options. I'm setting config.xclk_freq_hz = 20000000;

I thought the discussion here #774 (comment) might account for the issue I was having. It might be more an esp32-arduino change than esp32-camera issue.

[RubenKelevra]

@jameszah there might be some confusion what psram_mode is: It's not about if there's PSRAM or not or if it's used or not: psram_mode is what the ESP32-camera library calls a direct DMA from the camera module to the PSRAM. Meaning the camera module does not copy it's data temporarily in the regular RAM and the CPU then copies the data into PSRAM, instead the camera directly writes into the PSRAM, without the CPU being active. This feature is only available on ESP32S2 and ESP32 S3.

So your ESP32 doesn't have the hardware capability of using psram_mode, so it's not the issue I've described above.

Instead the "DMA overflow" is the message I've introduced in #772 which hints to the fact that the image supplied by the camera is larger than the space reserved in the PSRAM for it.

[RubenKelevra]

@jameszah the current heuristic would make a memory slot for your images, 61440 bytes (if you're using VGA). Can you please try to set a higher number, like make it 256 kByte or something like this and report back if the issue goes away?

[jameszah]

I don't think I allocate those buffers. My images, after the series of error messages, are about 15100 byes. When I call the esp_camera_fb_get, I just receive the pointer to the memory in psram that the driver provides. I'm setting the jpeg quality to 8, and it is set as VGA in the initialization and not changed afterwards, and I think it defaults to store in psram. When I copy the jpeg into my own code, I'm getting accurate sizes and good jpegs of 15kb or so. The errors come immediately after the camera_init, and then

  sensor_t * ss = esp_camera_sensor_get();

  if (ss->id.PID == OV5640_PID ) {
    //Serial.println("56 - going mirror");
    ss->set_hmirror(ss, 1);        // 0 = disable , 1 = enable
  } else {
    ss->set_hmirror(ss, 0);        // 0 = disable , 1 = enable
  }

  ss->set_brightness(ss, 1);  //up the blightness just a bit
  ss->set_saturation(ss, -2); //lower the saturation

... and then I just fb_get() and fb_return 30 images to warm up the camera. Once the first image returns, the next 30 arrive fine, and then about another 400 arrive before I get another DMA overflow message,

I switched back to 3.2.0, and my 30 warmup images started at 27494 bytes and dropped to 14222. but when I ran it with 3.3.0, I got the errors, and then the first successful warm-up image was about 14000. Maybe the camera is sending some scrambled data that is too long before it gets warmed up. Maybe leave the camera to warmup before the series of get and return. I'll try that tomorrow.

[me-no-dev]

3.3.0 has the latest esp32-camera master, so @RubenKelevra it includes everything from you. 3.2.0 is earlier version.

As for getting an image to gauge the needed buffer size, this is also not a great idea, because the image size changes with light, position and many other things. Formula was to guess about what the maximum size could be and use that for buffer. It's in no way perfect, but there really is no formula to use... JPEG size theoretically can even go larger than it's RGB counterpart

[RubenKelevra]

3.3.0 has the latest esp32-camera master, so @RubenKelevra it includes everything from you. 3.2.0 is earlier version.

Ah thanks!

As for getting an image to gauge the needed buffer size, this is also not a great idea, because the image size changes with light, position and many other things. Formula was to guess about what the maximum size could be and use that for buffer. It's in no way perfect, but there really is no formula to use... JPEG size theoretically can even go larger than it's RGB counterpart

True, JPEG sizes do vary, based on the image content. But from my experience it's not by a lot, as the noise of the camera modules keep the JPEG size up, even when it's dark.

We could also do a fallback, that if we grab an image and it's larger, that we rerun this evaluation and re-init the buffers with the larger size. This way we're not blocking the image output continously, but just stutter for 2-3 frames and then continue.