question for the gradient and pertrain

[hhhmoan]

at first,in function calc_reward, when you calc the J,you use p_loc made by mean_locs and sample_locs, but both the mean_locs and the sample_locs are stop_gradients. so I think tf.log(p_loc + SMALL_NUM) * (R - no_grad_b) is no use when calc the gradients.
and why this need to use pretrain.but in paper,i never found this method.

thanks for you release your code,can you solve my doubts, and have you finish this experiment in translate clutter mnist data 100 * 100. if you have,please @me. thanks.

[qihongl]

@hhhmoan Hi! Thank you very much for spending time reading the code and pointing out mysterious aspects!

The code actually works WITHOUT the pretraining. I was just a little bit curious to see if pretraining can improve anything. By the way, I just realized that the implementation for the pretraining part incomplete (it is not training the location network)...

On the other hand, I totally understand your suggestion about the stop_gradient for the location output. I will do some testing about that! We are still actively working on debugging. I will report our attempts to replicate the original results by Mnih et al. ASAP!

Thank you very much for your suggestion again!

[hhhmoan]

^_^. To say the truth, i failed to implement this paper in translate clutter mnist data in tensorflow.if you success and release your result,it can really help me.

my best wish to you work

[qihongl]

I'm glad to hear that! We are working on it!

[Lzc6996]

there are some ops in RAM is not differentiable, and in the paper, use the reward to replace the gradient so that we can do the BP.But in the code, your reward is in the loss, so the reward just like a normal loss and tf will compute the gradient include the ops which is not differentiable. i see that some tf.stop_gradient you use. is that mean you just stop the gradient of these ops? if it is true ,i dont know why the model work,because the params in the LocNet is frozen.

[qihongl]

@Lzc6996 You are right! Our StopGrad operation is incorrect and we still haven't figure out how to resolve this issue.

We failed to realize that because the network still "works". It might be the case that other layers can adapt (by tuning their parameters) the terrible parameters in the LocNet.

[JasonZhao001]

Qihong, have you work out how to update the parameters in LocNet yet?

[qihongl]

Sorry, not yet. Our code is incorrect for some more fundamental reasons. I am still not exactly sure how to fix it. You can take a look at this repo: https://github.com/zhongwen/RAM

This RAM implementation beats ours on the 28x28 standard MNIST.

[JasonZhao001]

Thanks for your favorable and prompt reply. And do you think the implementation that you mentioned above realize the parameters updates as Mnih's paper describes?

[qihongl]

@JasonZhao001 I am not sure. I plan to replicate Minh's results with that implementation. I would be more certain if the replication is successful.

[qihongl]

@JasonZhao001 And I also plan to visualize that implementation with tensor board.

[JasonZhao001]

Yeah, I found that the work done by Zhongwen drops check point and summary(tensorboard), its will be helpful if they are added.
And there are a question that still confuses me when I try to work on your codes :
The "DRAW WINDOW" functions don't work on my machine, even if I have set the control parameters to True. I have put a "print" in the animate block and it prints as I set when training but still no windows shows. I wonder its the problem only in my case, so can you tell me if it works in your platform now?

[qihongl]

@JasonZhao001 That's strange. "draw" should work when you set draw to 1. Can you send me the error message? Thanks!

[JasonZhao001]

There is no error message but just don't show the window as it should be. So I doubt it may because of my platform's problem.

[qihongl]

@JasonZhao001 I see. Let me know if you get more clues about what is going on. I am more than happy to help!

[JasonZhao001]

@qihongl found the reason. It's because that the matplotlib has something wrong in my platform, when I $sudo pip uninstall matplotlib, it worked!
I may be because I had install two versions of matplotlib and when I installed the second one, I set to ignore the excited one.
Thanks a lot!