experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Merge pull request #27738 from mosesmarin:patch-5

Browse Source 
PiperOrigin-RevId: 245491629
This commit is contained in:
TensorFlower Gardener 2019-04-26 16:03:08 -07:00
commit 38310b582d
1 changed files with 8 additions and 8 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

16
tensorflow/python/keras/optimizer_v2/adam.py
View File

@ -32,16 +32,16 @@ class Adam(optimizer_v2.OptimizerV2):
"""Optimizer that implements the Adam algorithm. """Optimizer that implements the Adam algorithm.
Adam optimization is a stochastic gradient descent method that is based on Adam optimization is a stochastic gradient descent method that is based on
adaptive estimation of first-order and second-order moments. According to the adaptive estimation of first-order and second-order moments.
reference, the method is 'computationally efficient, has little memory According to the paper
[Adam: A Method for Stochastic Optimization. Kingma et al.,
2014](http://arxiv.org/abs/1412.6980),
the method is "*computationally efficient, has little memory
requirement, invariant to diagonal rescaling of gradients, and is well suited requirement, invariant to diagonal rescaling of gradients, and is well suited
for problems that are large in terms of data/parameters'. for problems that are large in terms of data/parameters*".
# References For AMSGrad see [On The Convergence Of Adam And Beyond.
See [Kingma et al., 2014](http://arxiv.org/abs/1412.6980) Reddi et al., 5-8](https://openreview.net/pdf?id=ryQu7f-RZ).
([pdf](http://arxiv.org/pdf/1412.6980.pdf)).
For AMSGrad see [Reddi et al., 2-18]
(https://openreview.net/pdf?id=ryQu7f-RZ)
""" """
def __init__(self, def __init__(self,