Using rolling shutter in phase based video processing

[alienhd]

Hi itberrios,
I heard that for some applications like motion amplification or passive sound recovery using vibrations from sound waves the user has to have a high speed camera in order to be able to process the data. I also read about using the rolling shutter of CMOS cameras to leverage the effective sampling rate for such use cases. Could you help me to understand how to use or implement this effect?

For reference I found this article from the university of Ottawa talking about this topic: https://ruor.uottawa.ca/items/0d6100d4-fec2-4539-b1b6-fbf24a28c62c

[itberrios]

Hello, this sounds interesting. I might be able to help, do you have any specific questions? Or issues that I could look into?

[alienhd]

Well I just don't get the implementation to work, I have tried a lot of different github repos and even tried coding it by myself.

The project itself for the whole visual microphone project requires upwards of 2000 fps of footage because it translates to a resulting 2000 Hz of sound frequenzy. The CMOS sensor should be able to make normal footage usable because of the line delay.

[itberrios]

ok, I see. Can you send over any of the links to the repos? I can't seem to find anything useful on this.

[alienhd]

This one for example: https://github.com/jia-kai/hearv/tree/master