Virtual room simulation

[sakshamsingh1]

Hi,

Thanks for the wonderful work.

I have a question regarding generating virtual rooms with customizable microphone and speaker placements, similar to Fig. 2 in the paper.
Is this feature already implemented or planned for a future release?

Thanks,
Saksham

[iranroman]

Thank you Saksham.

We are working on an improved version of the room engine, using the realistic rlr-audio-propagation. At the moment it's in development (and it's fully functional), but we could give you access to it, assuming you will help us along the way with development.

[sakshamsingh1]

Thanks, Iran.
It would be really helpful to have access and will be eager to help in development.

[iranroman]

Hello all,

I would like to revive the discussion of this issue in light of the work that @karinnacvijanovich has been carrying out.

In a nutshell, she has demonstrated that it is possible to use this engine (called rlr-audio-propagation) to generate room impulse responses in virtual environments, and obtain the same performance on SELDnet when using simulated data in these virtual rooms (compared to the original spatial mixtures provided for its training released to train SELDnet).

Therefore, I believe we are now ready to start work that implements this virtual room generation into SpatialScaper! At this point I would like to invite thoughts from @ChrisIck @adrianSRoman @beasteers @sivannavis @wassim843 @chuyangchencd

Here's a rough pipeline for soundscape generation using this new method (for static sound sources to keep things simple):

• Define a room using a dedicated class. Could be shoebox or a mesh such as the ones in GibsonEnv. As part of the room definition, the location of the microphone(s) need to be defined too.
• Next, define the soundscape, in a fashion similar to this, passing the room to ss.Scaper
• When adding events, the RIR for the location of each sound source is rendered using rlr-audio-propagation instead of loading from the SOFA file (as it is done currently). Repeat for as many events as wanted in the soundscape.

The rest of the process stays more or less the same.

For moving sound sources, it would be a similar procedure, but the IR generation would have to be repeated for each of the N locations along the trajectory that will interpolate IRs to render the sound source moving using the current method in main.

I'll start creating a draft of this implementation and share with you. I'll start with FOA format to keep things simpler. This initial sketch will be to identify the main components that we need to flesh out and test, and we will implement each independently to keep our work modular and concise. Feedback welcome.

[iranroman]

@karinnacvijanovich and I have created a "quick and dirty" implementation that integrates the audio propagation engine used by SoundSpaces and the current SpatialScaper API. You can check it out here.

@ChrisIck
@beasteers
@adrianSRoman
@sivannavis
@sakshamsingh1
@chuyangchencd

I would love to get your thoughts, as the next steps will be to implement these changes in a branch to merge with main.