initial value used in rmsprop

[yorkerlin]

@bayerj
The 152-th line of rmsprop.py

        self.moving_mean_squared = 1

I think the initial value should be 0 instead of 1.
Any reason why 1 is better than 0

[yorkerlin]

I look at several implementations of rmsprop

https://github.com/Lasagne/Lasagne/blob/master/lasagne/updates.py
https://github.com/torch/optim/blob/master/rmsprop.lua

They use 0 instead of 1.

[bayerj]

The reason is that it starts out as gradient descent then and transitions to rmsprop then. I have no idea if this change has a huge impact. If someone shows that it works better, I am happy with changing it.