experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Correct the formulation in the description of the centered rmsprop

Browse Source 
PiperOrigin-RevId: 299879992
Change-Id: I8e0ec675cc933a11ff8ac8a7e29ca0f4de030d36
This commit is contained in:
A. Unique TensorFlower 2020-03-09 10:50:11 -07:00 committed by TensorFlower Gardener
parent 105c91f2c4
commit b5085d2014
1 changed files with 1 additions and 1 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tensorflow/python/training/rmsprop.py
View File

@ -30,7 +30,7 @@ This implementation of RMSProp uses plain momentum, not Nesterov momentum.
The centered version additionally maintains a moving (discounted) average of the
gradients, and uses that average to estimate the variance:
mean_grad = decay * mean_square{t-1} + (1-decay) * gradient
mean_grad = decay * mean_grad{t-1} + (1-decay) * gradient
mean_square = decay * mean_square{t-1} + (1-decay) * gradient ** 2
mom = momentum * mom{t-1} + learning_rate * g_t /
sqrt(mean_square - mean_grad**2 + epsilon)