experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Update weight_decay_optimizers.py

Browse Source
This commit is contained in:
Ouwen Huang 2018-11-15 14:28:48 -05:00 committed by GitHub
parent 59617ccaca
commit 71ba0ec86e
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 4 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

8
tensorflow/contrib/opt/python/training/weight_decay_optimizers.py
View File

@ -64,10 +64,10 @@ class DecoupledWeightDecayExtension(object):
the decay to the `weight_decay` as well. For example:
```python
decay = tf.train.piecewise_constant(tf.train.get_global_step(),
[10000, 15000], [1e-1, 1e-2, 1e-3])
lr = 1*decay
wd = 1e-4*decay
schedule = tf.train.piecewise_constant(tf.train.get_global_step(),
[10000, 15000], [1e-0, 1e-1, 1e-2])
lr = 1e-1 * schedule()
wd = lambda: 1e-4 * schedule()
# ...