Step on multiple environment in parallel

[jwyang]

Hi, thanks for open sourcing this code.

I am wondering whether we can load multiple environments and then step on them in parallel? It will be extremely useful to accelerate the data loading process.

thanks!

[ppwwyyxx]

https://github.com/facebookresearch/House3D/blob/master/INSTRUCTION.md#concurrency-solutions

[jwyang]

thanks! I noticed this before. But it might not what I want. basically, let's say, we have already created multiple environments:

env_list = [Environment(...) for i in range(nun_envs) ]

then, I want to use a list of new cam poses as input and get the rendering results for the new viewpoints in parallel, something like the parallelization in openAI gym:

envs = SubprocVecEnvs(env_list)
obs = envs.step(actions)

what do you think?

[ppwwyyxx]

I think you can easily write a wrapper to do that.

[jwyang]

thanks for your reply.

I tried multiple wrappers to do that. When using multiprocessing, it always says that the objrender cannot be pickled, which is hard to solve. Then, I tried to used SubprocVecEnvs to wrapper, it then get stuck, while DummyVecEnvs works fine. I also tried to use Pytorch data loader to wrapper, but still get stuck when running to api.render(). Now I can hardly find a solution for this.

[ppwwyyxx]

it always says that the objrender cannot be pickled

That means you did not follow bullet point 3 in https://github.com/facebookresearch/House3D/blob/master/INSTRUCTION.md#concurrency-solutions.

[jwyang]

Yes, I referred to this. The code works fine here. But when I used a list of envs as the inputs along with the camera poses to the multi-processing, and I encountered the "cannot pickle problem". Will try more on this.

[ppwwyyxx]

Could you post the entire logs and errors instead of describing them? This may help better diagnose the issue.

[jwyang]

Yes, will post here tonight

…

On Fri, Oct 26, 2018 at 20:38 Yuxin Wu ***@***.***> wrote: Could you post the entire logs and errors instead of describing them? This may help better diagnose the issue. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub <#46 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADtr58AZgH2yfKy2TRbgxXygff4geSezks5uo6sEgaJpZM4X7atW> .