experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Polish the tf.optimizers.Adagrad class documentation a bit.

Browse Source 
Before:
https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/optimizers/Adagrad
PiperOrigin-RevId: 236410591
This commit is contained in:
A. Unique TensorFlower 2019-03-01 17:57:19 -08:00 committed by TensorFlower Gardener
parent 0c702c4f95
commit 4006467c5f
1 changed files with 9 additions and 9 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

18
tensorflow/python/keras/optimizer_v2/adagrad.py
View File

@ -41,18 +41,18 @@ class Adagrad(optimizer_v2.OptimizerV2):
the smaller the updates. the smaller the updates.
Initialization: Initialization:
$$accum_{g_0} := \text{initial_accumulator_value}$$
$$accum_g_0 := initial_accumulator_value$$ Update step:
$$t := t + 1$$ $$t := t + 1$$
$$accum_g_t := accum_g_{t-1} + g * g$$ $$accum_{g_t} := accum_{g_{t-1}} + g^2$$
$$theta_t := theta_{t-1} - lr * g / (\sqrt{accum_g_t} + \epsilon)$$ $$\theta_t := \theta_{t-1} - lr * g / (\sqrt{accum_{g_t}} + \epsilon)$$
References References:
See [paper]
(http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf) * [Paper](http://www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf).
or this * [Introduction]
[intro](https://ppasupat.github.io/a9online/uploads/proximal_notes.pdf). (https://ppasupat.github.io/a9online/uploads/proximal_notes.pdf).
""" """
def __init__(self, def __init__(self,