memory usage growth after a while

[babak-badnava]

I have tested the code on GTX 1080 with 32g Ram but when I run the code memory usage increases over time and after about 30 hours it will take all 32g of ram and make system to dies

[ifrosio]

Hi, can you provide some more details? Are you using the basic version of the algorithm with Pong? Automatic scheduling enabled? How many agents / trainers / predictors? Is the number of agents / training / predictor growing during training or does it stabilize? Thanks

[babak-badnava]

Hi, yes I do use the basic version with Breakout game and Dynamic setting is off and 32 agents running with 2 trainer threads and 2 predictor threads. all the settings are fixed during training but memory usage will growth. I think this may be because of tensorflow memory leak.

[ifrosio]

Tensor flow version?

[babak-badnava]

tensorflow version 1.2.1
installing from binary
cuda version 7.5