Docs changes for 1.0.0
This commit is contained in:
Reuben Morais
parent
0b36745338
commit
29e980473f
@ -1 +1,95 @@
|
||||
Test automatic release notes.
|
||||
# General
|
||||
|
||||
This is the 1.0.0 release for Coqui STT, the deep learning toolkit for speech-to-text. In accordance with [semantic versioning](https://semver.org/), this version is not completely backwards compatible with previous versions. The compatibility guarantees of our semantic versioning cover the inference APIs: the C API and all the official language bindings: Python, Node.JS/ElectronJS and Android. You can get started today with Coqui STT 1.0.0 by following the steps in our [documentation](https://stt.readthedocs.io/).
|
||||
|
||||
This release includes pre-trained English models, available in the Coqui Model Zoo:
|
||||
|
||||
- [Coqui English STT v1.0.0-huge-vocab](https://coqui.ai/english/coqui/v1.0.0-huge-vocab)
|
||||
- [Coqui English STT v1.0.0-yesno](https://coqui.ai/english/coqui/v1.0.0-yesno)
|
||||
- [Coqui English STT v1.0.0-large-vocab](https://coqui.ai/english/coqui/v1.0.0-large-vocab)
|
||||
- [Coqui English STT v1.0.0-digits](https://coqui.ai/english/coqui/v1.0.0-digits)
|
||||
|
||||
all under the Apache 2.0 license.
|
||||
|
||||
The acoustic models were trained on American English data with synthetic noise augmentation. The model achieves a 4.5% word error rate on the [LibriSpeech clean test corpus](http://www.openslr.org/12) and 13.6% word error rate on the [LibriSpeech other test corpus](http://www.openslr.org/12) with the largest release language model.
|
||||
|
||||
Note that the model currently performs best in low-noise environments with clear recordings. This does not mean the model cannot be used outside of these conditions, but that accuracy may be lower. Some users may need to further fine tune the model to meet their intended use-case.
|
||||
|
||||
We also include example audio files:
|
||||
|
||||
[audio-1.0.0.tar.gz](https://github.com/coqui-ai/STT/releases/download/v1.0.0/audio-1.0.0.tar.gz)
|
||||
|
||||
which can be used to test the engine, and checkpoint files for the English model:
|
||||
|
||||
[coqui-stt-1.0.0-checkpoint.tar.gz](https://github.com/coqui-ai/STT/releases/download/v1.0.0/coqui-stt-1.0.0-checkpoint.tar.gz)
|
||||
|
||||
which are under the Apache 2.0 license and can be used as the basis for further fine-tuning. Finally this release also includes a source code tarball:
|
||||
|
||||
[v1.0.0.tar.gz](https://github.com/coqui-ai/STT/archive/v1.0.0.tar.gz)
|
||||
|
||||
Under the [MPL-2.0 license](https://www.mozilla.org/en-US/MPL/2.0/). Note that this tarball is for archival purposes only since GitHub does not include submodules in the automatic tarballs. For usage and development with the source code, clone the repository using Git, following our [documentation](https://stt.readthedocs.io/).
|
||||
|
||||
|
||||
# Notable changes
|
||||
|
||||
- Removed support for protocol buffer input in native client and consolidated all packages under a single "STT" name accepting TFLite inputs
|
||||
- Added programmatic interface to training code and example Jupyter Notebooks, including how to train with Common Voice data
|
||||
- Added transparent handling of mixed sample rates and stereo audio in training inputs
|
||||
- Moved CI setup to GitHub Actions, making code contributions easier to test
|
||||
- Added configuration management via Coqpit, providing a more flexible config interface that's compatible with Coqui TTS
|
||||
- Handle Opus audio files transparently in training inputs
|
||||
- Added support for automatic dataset subset splitting
|
||||
- Added support for automatic alphabet generation and loading
|
||||
- Started publishing the training code CI for a faster notebook setup
|
||||
- Refactor training code into self-contained modules and deprecate train.py as universal entry point for training
|
||||
|
||||
# Training Regimen + Hyperparameters for fine-tuning
|
||||
|
||||
The hyperparameters used to train the model are useful for fine tuning. Thus, we document them here along with the training regimen, hardware used (a server with 8 NVIDIA A100 GPUs each with 40GB of VRAM), along with the full training hyperparameters. The full training configuration in JSON format is available [here](https://gist.github.com/reuben/6ced6a8b41e3d0849dafb7cae301e905).
|
||||
|
||||
The datasets used were:
|
||||
- Common Voice 7.0 (with custom train/dev/test splits)
|
||||
- Multilingual LibriSpeech (English, Opus)
|
||||
- LibriSpeech
|
||||
|
||||
The optimal `lm_alpha` and `lm_beta` values with respect to the Common Voice 7.0 (custom Coqui splits) and a large vocabulary language model:
|
||||
|
||||
- lm_alpha: 0.5891777425167632
|
||||
- lm_beta: 0.6619145283338659
|
||||
|
||||
# Documentation
|
||||
|
||||
Documentation is available on [stt.readthedocs.io](https://stt.readthedocs.io/).
|
||||
|
||||
# Contact/Getting Help
|
||||
|
||||
1. [GitHub Discussions](https://github.com/coqui-ai/STT/discussions/) - best place to ask questions, get support, and discuss anything related to 🐸STT with other users.
|
||||
3. [Gitter](https://gitter.im/coqui-ai/) - You can also join our Gitter chat.
|
||||
4. [Issues](https://github.com/coqui-ai/STT/issues) - If you have discussed a problem and identified a bug in 🐸STT, or if you have a feature request, please open an issue in our repo. Please make sure you search for an already existing issue beforehand!
|
||||
|
||||
# Contributors to 1.0.0 release
|
||||
|
||||
- Alexandre Lissy
|
||||
- Anon-Artist
|
||||
- Anton Yaroshenko
|
||||
- Catalin Voss
|
||||
- CatalinVoss
|
||||
- dag7dev
|
||||
- Dustin Zubke
|
||||
- Eren Gölge
|
||||
- Erik Ziegler
|
||||
- Francis Tyers
|
||||
- Ideefixze
|
||||
- Ilnar Salimzianov
|
||||
- imrahul3610
|
||||
- Jeremiah Rose
|
||||
- Josh Meyer
|
||||
- Kathy Reid
|
||||
- Kelly Davis
|
||||
- Kenneth Heafield
|
||||
- NanoNabla
|
||||
- Neil Stoker
|
||||
- Reuben Morais
|
||||
- zaptrem
|
||||
|
||||
We’d also like to thank all the members of our [Gitter chat room](https://gitter.im/coqui-ai/STT) who have been helping to shape this release!
|
||||
|
||||
@ -23,12 +23,20 @@ You can deploy 🐸STT models either via a command-line client or a language bin
|
||||
Download trained Coqui STT models
|
||||
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
||||
|
||||
You can find pre-trained models ready for deployment on the 🐸STT `releases page <https://github.com/coqui-ai/STT/releases>`_. You can also download the latest acoustic model (``.tflite``) and language model (``.scorer``) from the command line as such:
|
||||
You can find pre-trained models ready for deployment on the `Coqui Model Zoo <https://coqui.ai/models>`_. You can also use the 🐸STT Model Manager to download and try out the latest models:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
wget https://github.com/coqui-ai/STT/releases/download/v0.9.3/coqui-stt-0.9.3-models.tflite
|
||||
wget https://github.com/coqui-ai/STT/releases/download/v0.9.3/coqui-stt-0.9.3-models.scorer
|
||||
# Create a virtual environment
|
||||
$ python3 -m venv venv-stt
|
||||
$ source venv-stt/bin/activate
|
||||
|
||||
# Install 🐸STT model manager
|
||||
$ python -m pip install -U pip
|
||||
$ python -m pip install coqui-stt-model-manager
|
||||
|
||||
# Run the model manager. A browser tab will open and you can then download and test models from the Model Zoo.
|
||||
$ stt-model-manager
|
||||
|
||||
In every 🐸STT official release, there are different model files provided. The acoustic model uses the ``.tflite`` extension. Language models use the extension ``.scorer``. You can read more about language models with regard to :ref:`the decoding process <decoder-docs>` and :ref:`how scorers are generated <language-model>`.
|
||||
|
||||
@ -47,7 +55,7 @@ How well a 🐸STT model transcribes your audio will depend on a lot of things.
|
||||
|
||||
If you take a 🐸STT model trained on English, and pass Spanish into it, you should expect the model to perform horribly. Imagine you have a friend who only speaks English, and you ask her to make Spanish subtitles for a Spanish film, you wouldn't expect to get good subtitles. This is an extreme example, but it helps to form an intuition for what to expect from 🐸STT models. Imagine that the 🐸STT models are like people who speak a certain language with a certain accent, and then think about what would happen if you asked that person to transcribe your audio.
|
||||
|
||||
An acoustic model (i.e. ``.tflite`` file) has "learned" how to transcribe a certain language, and the model probably understands some accents better than others. In addition to languages and accents, acoustic models are sensitive to the style of speech, the topic of speech, and the demographics of the person speaking. The language model (``.scorer``) has been trained on text alone. As such, the language model is sensitive to how well the topic and style of speech matches that of the text used in training. The 🐸STT `release notes <https://github.com/coqui-ai/STT/releases/tag/v0.9.3>`_ include detailed information on the data used to train the models. If the data used for training the off-the-shelf models does not align with your intended use case, it may be necessary to adapt or train new models in order to improve transcription on your data.
|
||||
An acoustic model (i.e. ``.tflite`` file) has "learned" how to transcribe a certain language, and the model probably understands some accents better than others. In addition to languages and accents, acoustic models are sensitive to the style of speech, the topic of speech, and the demographics of the person speaking. The language model (``.scorer``) has been trained on text alone. As such, the language model is sensitive to how well the topic and style of speech matches that of the text used in training. The 🐸STT `release notes <https://github.com/coqui-ai/STT/releases/latest>`_ include detailed information on the data used to train the models. If the data used for training the off-the-shelf models does not align with your intended use case, it may be necessary to adapt or train new models in order to improve transcription on your data.
|
||||
|
||||
Training your own language model is often a good way to improve transcription on your audio. The process and tools used to generate a language model are described in :ref:`language-model` and general information can be found in :ref:`decoder-docs`. Generating a scorer from a constrained topic dataset is a quick process and can bring significant accuracy improvements if your audio is from a specific topic.
|
||||
|
||||
@ -91,7 +99,7 @@ The following command assumes you :ref:`downloaded the pre-trained models <downl
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
(coqui-stt-venv)$ stt --model stt-0.9.3-models.tflite --scorer stt-0.9.3-models.scorer --audio my_audio_file.wav
|
||||
(coqui-stt-venv)$ stt --model model.tflite --scorer huge-vocabulary.scorer --audio my_audio_file.wav
|
||||
|
||||
See :ref:`the Python client <py-api-example>` for an example of how to use the package programatically.
|
||||
|
||||
@ -170,7 +178,7 @@ Assuming you have :ref:`downloaded the pre-trained models <download-models>`, yo
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
./stt --model coqui-stt-0.9.3-models.tflite --scorer coqui-stt-0.9.3-models.scorer --audio audio_input.wav
|
||||
./stt --model model.tflite --scorer huge-vocabulary.scorer --audio audio_input.wav
|
||||
|
||||
See the help output with ``./stt -h`` for more details.
|
||||
|
||||
|
||||
@ -23,10 +23,10 @@
|
||||
|
||||
BUILDING
|
||||
|
||||
Quickstart: Deployment
|
||||
^^^^^^^^^^^^^^^^^^^^^^
|
||||
Quickstart
|
||||
^^^^^^^^^^
|
||||
|
||||
The fastest way to deploy a pre-trained 🐸STT model is with `pip` with Python 3.6, 3.7, 3.8 or 3.9:
|
||||
The fastest way to use a pre-trained 🐸STT model is with the 🐸STT model manager, a tool that lets you quickly test and demo models locally. You'll need Python 3.6, 3.7, 3.8 or 3.9:
|
||||
|
||||
.. code-block:: bash
|
||||
|
||||
@ -34,20 +34,12 @@ The fastest way to deploy a pre-trained 🐸STT model is with `pip` with Python
|
||||
$ python3 -m venv venv-stt
|
||||
$ source venv-stt/bin/activate
|
||||
|
||||
# Install 🐸STT
|
||||
# Install 🐸STT model manager
|
||||
$ python -m pip install -U pip
|
||||
$ python -m pip install stt
|
||||
$ python -m pip install coqui-stt-model-manager
|
||||
|
||||
# Download 🐸's pre-trained English models
|
||||
$ curl -LO https://github.com/coqui-ai/STT/releases/download/v0.9.3/coqui-stt-0.9.3-models.tflite
|
||||
$ curl -LO https://github.com/coqui-ai/STT/releases/download/v0.9.3/coqui-stt-0.9.3-models.scorer
|
||||
|
||||
# Download some example audio files
|
||||
$ curl -LO https://github.com/coqui-ai/STT/releases/download/v0.9.3/audio-0.9.3.tar.gz
|
||||
$ tar -xvf audio-0.9.3.tar.gz
|
||||
|
||||
# Transcribe an audio file
|
||||
$ stt --model coqui-stt-0.9.3-models.tflite --scorer coqui-stt-0.9.3-models.scorer --audio audio/2830-3980-0043.wav
|
||||
# Run the model manager. A browser tab will open and you can then download and test models from the Model Zoo.
|
||||
$ stt-model-manager
|
||||
|
||||
.. toctree::
|
||||
:maxdepth: 1
|
||||
|
||||
@ -219,12 +219,12 @@ Next, we need to install the `native_client` package, which contains the `genera
|
||||
|
||||
The `generate_scorer_package`, once installed via the `native client` package, is usable on _all platforms_ supported by 🐸STT. This is so that developers can generate scorers _on-device_, such as on an Android device, or Raspberry Pi 3.
|
||||
|
||||
To install `generate_scorer_package`, first download the relevant `native client` package from the [🐸STT GitHub releases page](https://github.com/coqui-ai/STT/releases/tag/v0.9.3) into the `data/lm` directory. The Docker image uses Ubuntu Linux, so you should use either the `native_client.amd64.cuda.linux.tar.xz` package if you are using `cuda` or the `native_client.amd64.cpu.linux.tar.xz` package if not.
|
||||
To install `generate_scorer_package`, first download the relevant `native client` package from the [🐸STT GitHub releases page](https://github.com/coqui-ai/STT/releases/latest) into the `data/lm` directory. The Docker image uses Ubuntu Linux, so you should use either the `native_client.amd64.cuda.linux.tar.xz` package if you are using `cuda` or the `native_client.amd64.cpu.linux.tar.xz` package if not.
|
||||
|
||||
The easiest way to download the package and extract it is using `curl [URL] | tar -Jxvf [FILENAME]`:
|
||||
The easiest way to download the package and extract it is using `curl -L [URL] | tar -Jxvf [FILENAME]`:
|
||||
|
||||
```
|
||||
root@dcb62aada58b:/STT/data/lm# curl https://github.com/coqui-ai/STT/releases/download/v0.9.3/native_client.amd64.cuda.linux.tar.xz | tar -Jxvf native_client.amd64.cuda.linux.tar.xz
|
||||
root@dcb62aada58b:/STT/data/lm# curl -L https://github.com/coqui-ai/STT/releases/download/v1.0.0/native_client.tflite.Linux.tar.xz | tar -Jxvf -
|
||||
libstt.so
|
||||
generate_scorer_package
|
||||
LICENSE
|
||||
@ -233,7 +233,7 @@ coqui-stt.h
|
||||
README.coqui
|
||||
```
|
||||
|
||||
You can now generate a `ken.lm` scorer file.
|
||||
You can now generate a KenLM scorer file.
|
||||
|
||||
```
|
||||
root@dcb62aada58b:/STT/data/lm# ./generate_scorer_package \
|
||||
|
||||
@ -26,8 +26,8 @@ class SpeechRecognitionImpl : NSObject, AVCaptureAudioDataOutputSampleBufferDele
|
||||
private var audioData = Data()
|
||||
|
||||
override init() {
|
||||
let modelPath = Bundle.main.path(forResource: "coqui-stt-0.9.3-models", ofType: "tflite")!
|
||||
let scorerPath = Bundle.main.path(forResource: "coqui-stt-0.9.3-models", ofType: "scorer")!
|
||||
let modelPath = Bundle.main.path(forResource: "model", ofType: "tflite")!
|
||||
let scorerPath = Bundle.main.path(forResource: "huge-vocab", ofType: "scorer")!
|
||||
|
||||
model = try! STTModel(modelPath: modelPath)
|
||||
try! model.enableExternalScorer(scorerPath: scorerPath)
|
||||
|
||||
Loading…
Reference in New Issue
Block a user