experiments/STT
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

adding amp doc

Browse Source
This commit is contained in:
Vinh Nguyen 2019-10-27 23:29:05 +00:00
parent f3694efbca
commit b105640d28
1 changed files with 13 additions and 0 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

13
TRAINING.rst
View File

@ -144,6 +144,19 @@ Each dataset has a corresponding importer script in ``bin/`` that can be used to
If you've run the old importers (in ``util/importers/``\ ), they could have removed source files that are needed for the new importers to run. In that case, simply remove the extracted folders and let the importer extract and process the dataset from scratch, and things should work. If you've run the old importers (in ``util/importers/``\ ), they could have removed source files that are needed for the new importers to run. In that case, simply remove the extracted folders and let the importer extract and process the dataset from scratch, and things should work.
Training with automatic mixed precision
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
Automatic Mixed Precision (AMP) training on GPU for TensorFlow has been recently [introduced](https://medium.com/tensorflow/automatic-mixed-precision-in-tensorflow-for-faster-ai-training-on-nvidia-gpus-6033234b2540).
Mixed precision training makes use of both FP32 and FP16 precisions where appropriate. FP16 operations can leverage the Tensor cores on NVIDIA GPUs (Volta, Turing or newer architectures) for improved throughput. Mixed precision training also often allows larger batch sizes. DeepSpeech GPU automatic mixed precision training can be enabled via the flag value `--auto_mixed_precision=True`.
```
DeepSpeech.py --train_files ./train.csv --dev_files ./dev.csv --test_files ./test.csv --automatic_mixed_precision=True
```
On a Volta generation V100 GPU, automatic mixed precision speeds up DeepSpeech training and evaluation by ~30%-40%.
Checkpointing Checkpointing
^^^^^^^^^^^^^ ^^^^^^^^^^^^^