experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Move TFLite converter 2.0 docs to default.

Browse Source 
PiperOrigin-RevId: 270805335
This commit is contained in:
Nupur Garg 2019-09-23 18:16:43 -07:00 committed by TensorFlower Gardener
parent 882b91d247
commit 1ddcb42e1d
14 changed files with 498 additions and 506 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

14
tensorflow/lite/g3doc/_book.yaml
View File

@ -36,22 +36,14 @@ upper_tabs:
path: /lite/convert/ path: /lite/convert/
- title: "Python API" - title: "Python API"
path: /lite/convert/python_api path: /lite/convert/python_api
- title: "Command line examples" - title: "Command line"
path: /lite/convert/cmdline_examples path: /lite/convert/cmdline
- title: "Command line reference"
path: /lite/convert/cmdline_reference
- title: "Convert quantized models" - title: "Convert quantized models"
path: /lite/convert/quantization path: /lite/convert/quantization
- title: "Convert RNN models" - title: "Convert RNN models"
path: /lite/convert/rnn path: /lite/convert/rnn
- heading: "Convert a TensorFlow 2.0 model"
- title: "TensorFlow Lite converter and 2.0"
path: /lite/r2/convert/
- title: "Generate a concrete function" - title: "Generate a concrete function"
path: /lite/r2/convert/concrete_function path: /lite/convert/concrete_function
- title: "Python API"
path: /lite/r2/convert/python_api
- heading: "Inference" - heading: "Inference"
- title: "Overview" - title: "Overview"

2
tensorflow/lite/g3doc/r2/convert/cmdline.md → tensorflow/lite/g3doc/convert/cmdline.md
View File

@ -41,7 +41,7 @@ install the nightly build using [pip](https://www.tensorflow.org/install/pip) or
use `bazel`. An example can be seen below. use `bazel`. An example can be seen below.
``` ```
bazel run //third_party/tensorflow/lite/python:tflite_convert -- \ bazel run //tensorflow/lite/python:tflite_convert -- \
--saved_model_dir=/tmp/mobilenet_saved_model \ --saved_model_dir=/tmp/mobilenet_saved_model \
--output_file=/tmp/mobilenet.tflite --output_file=/tmp/mobilenet.tflite
``` ```

0
tensorflow/lite/g3doc/r2/convert/concrete_function.md → tensorflow/lite/g3doc/convert/concrete_function.md
View File

58
tensorflow/lite/g3doc/convert/index.md
View File

@ -1,48 +1,28 @@
# TensorFlow Lite converter # TensorFlow Lite converter
The TensorFlow Lite converter is used to convert TensorFlow models into an The TensorFlow Lite converter takes a TensorFlow model and generates a
optimized [FlatBuffer](https://google.github.io/flatbuffers/) format, so that TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/) file
they can be used by the TensorFlow Lite interpreter. (`.tflite`). The converter supports
[SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model),
[`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview), and
[concrete functions](concrete_function.md).
Note: This page contains documentation on the converter API for TensorFlow 1.x. Note: This page contains documentation on the converter API for TensorFlow 2.0.
The API for TensorFlow 2.0 is available The API for TensorFlow 1.X is available
[here](https://www.tensorflow.org/lite/r2/convert/). [here](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/r1/convert/index.md).
## FlatBuffers ## Device deployment
FlatBuffer is an efficient open-source cross-platform serialization library. It The TensorFlow Lite `FlatBuffer` file is then deployed to a client device (e.g.
is similar to mobile, embedded) and run locally using the TensorFlow Lite interpreter. This
[protocol buffers](https://developers.google.com/protocol-buffers), with the conversion process is shown in the diagram below:
distinction that FlatBuffers do not need a parsing/unpacking step to a secondary
representation before data can be accessed, avoiding per-object memory
allocation. The code footprint of FlatBuffers is an order of magnitude smaller
than protocol buffers.
## From model training to device deployment
The TensorFlow Lite converter generates a TensorFlow Lite
[FlatBuffer](https://google.github.io/flatbuffers/) file (`.tflite`) from a
TensorFlow model.
The converter supports the following input formats:
* [SavedModels](https://www.tensorflow.org/guide/saved_model#using_savedmodel_with_estimators)
* Frozen `GraphDef`: Models generated by
[freeze_graph.py](https://www.tensorflow.org/code/tensorflow/python/tools/freeze_graph.py).
* `tf.keras` HDF5 models.
* Any model taken from a `tf.Session` (Python API only).
The TensorFlow Lite `FlatBuffer` file is then deployed to a client device, and
the TensorFlow Lite interpreter uses the compressed model for on-device
inference. This conversion process is shown in the diagram below:
![TFLite converter workflow](../images/convert/workflow.svg) ![TFLite converter workflow](../images/convert/workflow.svg)
## Options ## Converting models
The TensorFlow Lite Converter can be used from either of these two options: The TensorFlow Lite converter should be used from the
[Python API](python_api.md). Using the Python API makes it easier to convert
* [Python](python_api.md) (**Preferred**): Using the Python API makes it models as part of a model development pipeline and helps mitigate
easier to convert models as part of a model development pipeline, and helps [compatibility](../guide/ops_compatibility.md) issues early on. Alternatively,
mitigate [compatibility](../tf_ops_compatibility.md) issues early on. the [command line tool](cmdline.md) supports basic models.
* [Command line](cmdline_examples.md)

358
tensorflow/lite/g3doc/convert/python_api.md
View File

@ -1,187 +1,280 @@
# Converter Python API guide # Converter Python API guide
This page describes how to convert TensorFlow models into the TensorFlow Lite This page provides examples on how to use the
format using the TensorFlow Lite Converter Python API. [TensorFlow Lite converter](index.md) using the Python API in TensorFlow 2.0.
If you're looking for information about how to run a TensorFlow Lite model,
see [TensorFlow Lite inference](../guide/inference.md).
Note: This page describes the converter in the TensorFlow nightly release,
installed using `pip install tf-nightly`. For docs describing older versions
reference ["Converting models from TensorFlow 1.12"](#pre_tensorflow_1.12).
[TOC] [TOC]
## Python API
## High-level overview The Python API for converting TensorFlow models to TensorFlow Lite in TensorFlow
2.0 is `tf.lite.TFLiteConverter`. `TFLiteConverter` provides the following
classmethods to convert a model based on the original model format:
While the TensorFlow Lite Converter can be used from the command line, it is * `TFLiteConverter.from_saved_model()`: Converts
often convenient to use in a Python script as part of the model development [SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model).
pipeline. This allows you to know early that you are designing a model that can * `TFLiteConverter.from_keras_model()`: Converts
be targeted to devices with mobile. [`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview).
* `TFLiteConverter.from_concrete_functions()`: Converts
[concrete functions](concrete_function.md).
## API Note: The TensorFlow Lite 2.0 alpha had a different version of the
`TFLiteConverter` API which only contained the classmethod
[`from_concrete_function`](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/lite/TFLiteConverter#from_concrete_function).
The API detailed in this document can be installed using the
[`tf-nightly-2.0-preview`](#installing_the_tensorflow_20_nightly_) pip install.
The API for converting TensorFlow models to TensorFlow Lite is This document contains [example usages](#examples) of the API, a detailed list
`tf.lite.TFLiteConverter`, which provides class methods based on the original of [changes in the API between 1.X and 2.0](#differences), and
format of the model. For example, `TFLiteConverter.from_session()` is available [instructions](#versioning) on running the different versions of TensorFlow.
for GraphDefs, `TFLiteConverter.from_saved_model()` is available for
SavedModels, and `TFLiteConverter.from_keras_model_file()` is available for
`tf.Keras` files.
Example usages for simple float-point models are shown in ## Examples <a name="examples"></a>
[Basic Examples](#basic). Examples usages for more complex models is shown in
[Complex Examples](#complex).
## Basic examples <a name="basic"></a> ### Converting a SavedModel <a name="saved_model"></a>
The following section shows examples of how to convert a basic float-point model The following example shows how to convert a
from each of the supported data formats into a TensorFlow Lite FlatBuffers. [SavedModel](https://www.tensorflow.org/alpha/guide/saved_model) into a
TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/).
### Exporting a GraphDef from tf.Session <a name="basic_graphdef_sess"></a>
The following example shows how to convert a TensorFlow GraphDef into a
TensorFlow Lite FlatBuffer from a `tf.Session` object.
```python ```python
import tensorflow as tf import tensorflow as tf
img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3)) # Construct a basic model.
var = tf.get_variable("weights", dtype=tf.float32, shape=(1, 64, 64, 3)) root = tf.train.Checkpoint()
val = img + var root.v1 = tf.Variable(3.)
out = tf.identity(val, name="out") root.v2 = tf.Variable(2.)
root.f = tf.function(lambda x: root.v1 * root.v2 * x)
with tf.Session() as sess: # Save the model.
sess.run(tf.global_variables_initializer()) export_dir = "/tmp/test_saved_model"
converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out]) input_data = tf.constant(1., shape=[1, 1])
to_save = root.f.get_concrete_function(input_data)
tf.saved_model.save(root, export_dir, to_save)
# Convert the model.
converter = tf.lite.TFLiteConverter.from_saved_model(export_dir)
tflite_model = converter.convert() tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
``` ```
### Exporting a GraphDef from file <a name="basic_graphdef_file"></a> This API does not have the option of specifying the input shape of any input
arrays. If your model requires specifying the input shape, use the
[`from_concrete_functions`](#concrete_function) classmethod instead. The code
looks similar to the following:
The following example shows how to convert a TensorFlow GraphDef into a ```python
TensorFlow Lite FlatBuffer when the GraphDef is stored in a file. Both `.pb` and model = tf.saved_model.load(export_dir)
`.pbtxt` files are accepted. concrete_func = model.signatures[
tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([1, 256, 256, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])
```
The example uses ### Converting a Keras model <a name="keras"></a>
[Mobilenet_1.0_224](https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz).
The function only supports GraphDefs frozen using The following example shows how to convert a
[freeze_graph.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py). [`tf.keras` model](https://www.tensorflow.org/alpha/guide/keras/overview) into a
TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/).
```python ```python
import tensorflow as tf import tensorflow as tf
graph_def_file = "/path/to/Downloads/mobilenet_v1_1.0_224/frozen_graph.pb" # Create a simple Keras model.
input_arrays = ["input"] x = [-1, 0, 1, 2, 3, 4]
output_arrays = ["MobilenetV1/Predictions/Softmax"] y = [-3, -1, 1, 3, 5, 7]
converter = tf.lite.TFLiteConverter.from_frozen_graph( model = tf.keras.models.Sequential(
graph_def_file, input_arrays, output_arrays) [tf.keras.layers.Dense(units=1, input_shape=[1])])
model.compile(optimizer='sgd', loss='mean_squared_error')
model.fit(x, y, epochs=50)
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert() tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
``` ```
### Exporting a SavedModel <a name="basic_savedmodel"></a> ### Converting a concrete function <a name="concrete_function"></a>
The following example shows how to convert a SavedModel into a TensorFlow Lite The following example shows how to convert a TensorFlow
FlatBuffer. [concrete function](concrete_function.md) into a TensorFlow Lite
[`FlatBuffer`](https://google.github.io/flatbuffers/).
```python ```python
import tensorflow as tf import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) # Construct a basic model.
root = tf.train.Checkpoint()
root.v1 = tf.Variable(3.)
root.v2 = tf.Variable(2.)
root.f = tf.function(lambda x: root.v1 * root.v2 * x)
# Create the concrete function.
input_data = tf.constant(1., shape=[1, 1])
concrete_func = root.f.get_concrete_function(input_data)
# Convert the model.
#
# `from_concrete_function` takes in a list of concrete functions, however,
# currently only supports converting one function at a time. Converting multiple
# functions is under development.
converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
tflite_model = converter.convert() tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
``` ```
For more complex SavedModels, the optional parameters that can be passed into ### End-to-end MobileNet conversion <a name="mobilenet"></a>
`TFLiteConverter.from_saved_model()` are `input_arrays`, `input_shapes`,
`output_arrays`, `tag_set` and `signature_key`. Details of each parameter are
available by running `help(tf.lite.TFLiteConverter)`.
### Exporting a tf.keras File <a name="basic_keras_file"></a> The following example shows how to convert and run inference on a pre-trained
`tf.keras` MobileNet model to TensorFlow Lite. It compares the results of the
The following example shows how to convert a `tf.keras` model into a TensorFlow TensorFlow and TensorFlow Lite model on random data. In order to load the model
Lite FlatBuffer. This example requires from file, use `model_path` instead of `model_content`.
[`h5py`](http://docs.h5py.org/en/latest/build.html) to be installed.
```python
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_keras_model_file("keras_model.h5")
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
```
The `tf.keras` file must contain both the model and the weights. A comprehensive
example including model construction can be seen below.
```python ```python
import numpy as np import numpy as np
import tensorflow as tf import tensorflow as tf
# Generate tf.keras model. # Load the MobileNet tf.keras model.
model = tf.keras.models.Sequential() model = tf.keras.applications.MobileNetV2(
model.add(tf.keras.layers.Dense(2, input_shape=(3,))) weights="imagenet", input_shape=(224, 224, 3))
model.add(tf.keras.layers.RepeatVector(3))
model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(3)))
model.compile(loss=tf.keras.losses.MSE,
optimizer=tf.keras.optimizers.RMSprop(lr=0.0001),
metrics=[tf.keras.metrics.categorical_accuracy],
sample_weight_mode='temporal')
x = np.random.random((1, 3)) # Convert the model.
y = np.random.random((1, 3, 3)) converter = tf.lite.TFLiteConverter.from_keras_model(model)
model.train_on_batch(x, y)
model.predict(x)
# Save tf.keras model in HDF5 format.
keras_file = "keras_model.h5"
tf.keras.models.save_model(model, keras_file)
# Convert to TensorFlow Lite model.
converter = tf.lite.TFLiteConverter.from_keras_model_file(keras_file)
tflite_model = converter.convert() tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Test the TensorFlow Lite model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
# The function `get_tensor()` returns a copy of the tensor data.
# Use `tensor()` in order to get a pointer to the tensor.
tflite_results = interpreter.get_tensor(output_details[0]['index'])
# Test the TensorFlow model on random input data.
tf_results = model(tf.constant(input_data))
# Compare the result.
for tf_result, tflite_result in zip(tf_results, tflite_results):
np.testing.assert_almost_equal(tf_result, tflite_result, decimal=5)
``` ```
## Complex examples <a name="complex"></a> ## Summary of changes in Python API between 1.X and 2.0 <a name="differences"></a>
For models where the default value of the attributes is not sufficient, the The following section summarizes the changes in the Python API from 1.X to 2.0.
attribute's values should be set before calling `convert()`. In order to call If any of the changes raise concerns, please file a
any constants use `tf.lite.constants.<CONSTANT_NAME>` as seen below with [GitHub issue](https://github.com/tensorflow/tensorflow/issues).
`QUANTIZED_UINT8`. Run `help(tf.lite.TFLiteConverter)` in the Python
terminal for detailed documentation on the attributes.
Although the examples are demonstrated on GraphDefs containing only constants. ### Formats supported by `TFLiteConverter`
The same logic can be applied irrespective of the input data format.
### Exporting a quantized GraphDef <a name="complex_quant"></a> `TFLiteConverter` in 2.0 supports SavedModels and Keras model files generated in
both 1.X and 2.0. However, the conversion process no longer supports frozen
`GraphDefs` generated in 1.X. Users who want to convert frozen `GraphDefs` to
TensorFlow Lite should use `tf.compat.v1.TFLiteConverter`.
The following example shows how to convert a quantized model into a TensorFlow ### Quantization-aware training
Lite FlatBuffer.
The following attributes and methods associated with
[quantization-aware training](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize)
have been removed from `TFLiteConverter` in TensorFlow 2.0:
* `inference_type`
* `inference_input_type`
* `quantized_input_stats`
* `default_ranges_stats`
* `reorder_across_fake_quant`
* `change_concat_input_ranges`
* `post_training_quantize` - Deprecated in the 1.X API
* `get_input_arrays()`
The rewriter function that supports quantization-aware training does not support
models generated by TensorFlow 2.0. Additionally, TensorFlow Lite’s quantization
API is being reworked and streamlined in a direction that supports
quantization-aware training through the Keras API. These attributes will be
removed in the 2.0 API until the new quantization API is launched. Users who
want to convert models generated by the rewriter function can use
`tf.compat.v1.TFLiteConverter`.
### Changes to `TFLiteConverter` attributes
The `target_ops` attribute has become an attribute of `TargetSpec` and renamed
to `supported_ops` in line with future additions to the optimization framework.
Additionally, the following attributes have been removed:
* `drop_control_dependency` (default: `True`) - Control flow is currently not
supported by TFLite so it is always `True`.
* _Graph visualization_ - The recommended approach for visualizing a
TensorFlow Lite graph in TensorFlow 2.0 will be to use
[visualize.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/tools/visualize.py).
Unlike GraphViz, it enables users to visualize the graph after post training
quantization has occurred. The following attributes related to graph
visualization will be removed:
* `output_format`
* `dump_graphviz_dir`
* `dump_graphviz_video`
### General API changes
#### Conversion methods
The following methods that were previously deprecated in 1.X will no longer be
exported in 2.0:
* `lite.toco_convert`
* `lite.TocoConverter`
#### `lite.constants`
The `lite.constants` API was removed in 2.0 in order to decrease duplication
between TensorFlow and TensorFlow Lite. The following list maps the
`lite.constant` type to the TensorFlow type:
* `lite.constants.FLOAT`: `tf.float32`
* `lite.constants.INT8`: `tf.int8`
* `lite.constants.INT32`: `tf.int32`
* `lite.constants.INT64`: `tf.int64`
* `lite.constants.STRING`: `tf.string`
* `lite.constants.QUANTIZED_UINT8`: `tf.uint8`
Additionally, `lite.constants.TFLITE` and `lite.constants.GRAPHVIZ_DOT` were
removed due to the deprecation of the `output_format` flag in `TFLiteConverter`.
#### `lite.OpHint`
The `OpHint` API is currently not available in 2.0 due to an incompatibility
with the 2.0 APIs. This API enables conversion of LSTM based models. Support for
LSTMs in 2.0 is being investigated. All related `lite.experimental` APIs have
been removed due to this issue.
## Installing TensorFlow <a name="versioning"></a>
### Installing the TensorFlow 2.0 nightly <a name="2.0-nightly"></a>
The TensorFlow 2.0 nightly can be installed using the following command:
```
pip install tf-nightly-2.0-preview
```
### Using TensorFlow 2.0 from a 1.X installation <a name="use-2.0-from-1.X"></a>
TensorFlow 2.0 can be enabled from recent 1.X installations using the following
code snippet.
```python ```python
import tensorflow as tf import tensorflow.compat.v2 as tf
img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3)) tf.enable_v2_behavior()
const = tf.constant([1., 2., 3.]) + tf.constant([1., 4., 4.])
val = img + const
out = tf.fake_quant_with_min_max_args(val, min=0., max=1., name="output")
with tf.Session() as sess:
converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out])
converter.inference_type = tf.lite.constants.QUANTIZED_UINT8
input_arrays = converter.get_input_arrays()
converter.quantized_input_stats = {input_arrays[0] : (0., 1.)} # mean, std_dev
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
``` ```
## Additional instructions
### Build from source code <a name="latest_package"></a> ### Build from source code <a name="latest_package"></a>
In order to run the latest version of the TensorFlow Lite Converter Python API, In order to run the latest version of the TensorFlow Lite Converter Python API,
@ -189,14 +282,3 @@ either install the nightly build with
[pip](https://www.tensorflow.org/install/pip) (recommended) or [pip](https://www.tensorflow.org/install/pip) (recommended) or
[Docker](https://www.tensorflow.org/install/docker), or [Docker](https://www.tensorflow.org/install/docker), or
[build the pip package from source](https://www.tensorflow.org/install/source). [build the pip package from source](https://www.tensorflow.org/install/source).
### Converting models from TensorFlow 1.12 <a name="pre_tensorflow_1.12"></a>
Reference the following table to convert TensorFlow models to TensorFlow Lite in
and before TensorFlow 1.12. Run `help()` to get details of each API.
TensorFlow Version | Python API
------------------ | ---------------------------------
1.12 | `tf.contrib.lite.TFLiteConverter`
1.9-1.11 | `tf.contrib.lite.TocoConverter`
1.7-1.8 | `tf.contrib.lite.toco_convert`

2
tensorflow/lite/g3doc/images/convert/workflow.svg
View File

File diff suppressed because one or more lines are too long
Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 109 KiB

After

Width:  |  Height:  |  Size: 98 KiB

2
tensorflow/lite/g3doc/convert/cmdline_examples.md → tensorflow/lite/g3doc/r1/convert/cmdline_examples.md
View File

@ -18,7 +18,7 @@ There are two approaches to running the converter in the command line.
[clone the TensorFlow repository](https://www.tensorflow.org/install/source) [clone the TensorFlow repository](https://www.tensorflow.org/install/source)
and use `bazel`. and use `bazel`.
* Example: `bazel run * Example: `bazel run
//tensorflow/lite/python:tflite_convert -- //third_party/tensorflow/lite/python:tflite_convert --
--output_file=...` --output_file=...`
### Converting models prior to TensorFlow 1.9 <a name="pre_tensorflow_1.9"></a> ### Converting models prior to TensorFlow 1.9 <a name="pre_tensorflow_1.9"></a>

0
tensorflow/lite/g3doc/convert/cmdline_reference.md → tensorflow/lite/g3doc/r1/convert/cmdline_reference.md
View File

48
tensorflow/lite/g3doc/r1/convert/index.md Normal file
View File

@ -0,0 +1,48 @@
# TensorFlow Lite converter
The TensorFlow Lite converter is used to convert TensorFlow models into an
optimized [FlatBuffer](https://google.github.io/flatbuffers/) format, so that
they can be used by the TensorFlow Lite interpreter.
Note: This page contains documentation on the converter API for TensorFlow 1.x.
The API for TensorFlow 2.0 is available
[here](https://www.tensorflow.org/lite/r2/convert/).
## FlatBuffers
FlatBuffer is an efficient open-source cross-platform serialization library. It
is similar to
[protocol buffers](https://developers.google.com/protocol-buffers), with the
distinction that FlatBuffers do not need a parsing/unpacking step to a secondary
representation before data can be accessed, avoiding per-object memory
allocation. The code footprint of FlatBuffers is an order of magnitude smaller
than protocol buffers.
## From model training to device deployment
The TensorFlow Lite converter generates a TensorFlow Lite
[FlatBuffer](https://google.github.io/flatbuffers/) file (`.tflite`) from a
TensorFlow model.
The converter supports the following input formats:
* [SavedModels](https://www.tensorflow.org/guide/saved_model#using_savedmodel_with_estimators)
* Frozen `GraphDef`: Models generated by
[freeze_graph.py](https://www.tensorflow.org/code/tensorflow/python/tools/freeze_graph.py).
* `tf.keras` HDF5 models.
* Any model taken from a `tf.Session` (Python API only).
The TensorFlow Lite `FlatBuffer` file is then deployed to a client device, and
the TensorFlow Lite interpreter uses the compressed model for on-device
inference. This conversion process is shown in the diagram below:
![TFLite converter workflow](../images/convert/workflow.svg)
## Options
The TensorFlow Lite Converter can be used from either of these two options:
* [Python](python_api.md) (**Preferred**): Using the Python API makes it
easier to convert models as part of a model development pipeline, and helps
mitigate [compatibility](../tf_ops_compatibility.md) issues early on.
* [Command line](cmdline_examples.md)

202
tensorflow/lite/g3doc/r1/convert/python_api.md Normal file
View File

@ -0,0 +1,202 @@
# Converter Python API guide
This page describes how to convert TensorFlow models into the TensorFlow Lite
format using the TensorFlow Lite Converter Python API.
If you're looking for information about how to run a TensorFlow Lite model,
see [TensorFlow Lite inference](../guide/inference.md).
Note: This page describes the converter in the TensorFlow nightly release,
installed using `pip install tf-nightly`. For docs describing older versions
reference ["Converting models from TensorFlow 1.12"](#pre_tensorflow_1.12).
[TOC]
## High-level overview
While the TensorFlow Lite Converter can be used from the command line, it is
often convenient to use in a Python script as part of the model development
pipeline. This allows you to know early that you are designing a model that can
be targeted to devices with mobile.
## API
The API for converting TensorFlow models to TensorFlow Lite is
`tf.lite.TFLiteConverter`, which provides class methods based on the original
format of the model. For example, `TFLiteConverter.from_session()` is available
for GraphDefs, `TFLiteConverter.from_saved_model()` is available for
SavedModels, and `TFLiteConverter.from_keras_model_file()` is available for
`tf.Keras` files.
Example usages for simple float-point models are shown in
[Basic Examples](#basic). Examples usages for more complex models is shown in
[Complex Examples](#complex).
## Basic examples <a name="basic"></a>
The following section shows examples of how to convert a basic float-point model
from each of the supported data formats into a TensorFlow Lite FlatBuffers.
### Exporting a GraphDef from tf.Session <a name="basic_graphdef_sess"></a>
The following example shows how to convert a TensorFlow GraphDef into a
TensorFlow Lite FlatBuffer from a `tf.Session` object.
```python
import tensorflow as tf
img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3))
var = tf.get_variable("weights", dtype=tf.float32, shape=(1, 64, 64, 3))
val = img + var
out = tf.identity(val, name="out")
with tf.Session() as sess:
sess.run(tf.global_variables_initializer())
converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out])
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
```
### Exporting a GraphDef from file <a name="basic_graphdef_file"></a>
The following example shows how to convert a TensorFlow GraphDef into a
TensorFlow Lite FlatBuffer when the GraphDef is stored in a file. Both `.pb` and
`.pbtxt` files are accepted.
The example uses
[Mobilenet_1.0_224](https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz).
The function only supports GraphDefs frozen using
[freeze_graph.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py).
```python
import tensorflow as tf
graph_def_file = "/path/to/Downloads/mobilenet_v1_1.0_224/frozen_graph.pb"
input_arrays = ["input"]
output_arrays = ["MobilenetV1/Predictions/Softmax"]
converter = tf.lite.TFLiteConverter.from_frozen_graph(
graph_def_file, input_arrays, output_arrays)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
```
### Exporting a SavedModel <a name="basic_savedmodel"></a>
The following example shows how to convert a SavedModel into a TensorFlow Lite
FlatBuffer.
```python
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
```
For more complex SavedModels, the optional parameters that can be passed into
`TFLiteConverter.from_saved_model()` are `input_arrays`, `input_shapes`,
`output_arrays`, `tag_set` and `signature_key`. Details of each parameter are
available by running `help(tf.lite.TFLiteConverter)`.
### Exporting a tf.keras File <a name="basic_keras_file"></a>
The following example shows how to convert a `tf.keras` model into a TensorFlow
Lite FlatBuffer. This example requires
[`h5py`](http://docs.h5py.org/en/latest/build.html) to be installed.
```python
import tensorflow as tf
converter = tf.lite.TFLiteConverter.from_keras_model_file("keras_model.h5")
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
```
The `tf.keras` file must contain both the model and the weights. A comprehensive
example including model construction can be seen below.
```python
import numpy as np
import tensorflow as tf
# Generate tf.keras model.
model = tf.keras.models.Sequential()
model.add(tf.keras.layers.Dense(2, input_shape=(3,)))
model.add(tf.keras.layers.RepeatVector(3))
model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(3)))
model.compile(loss=tf.keras.losses.MSE,
optimizer=tf.keras.optimizers.RMSprop(lr=0.0001),
metrics=[tf.keras.metrics.categorical_accuracy],
sample_weight_mode='temporal')
x = np.random.random((1, 3))
y = np.random.random((1, 3, 3))
model.train_on_batch(x, y)
model.predict(x)
# Save tf.keras model in HDF5 format.
keras_file = "keras_model.h5"
tf.keras.models.save_model(model, keras_file)
# Convert to TensorFlow Lite model.
converter = tf.lite.TFLiteConverter.from_keras_model_file(keras_file)
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
```
## Complex examples <a name="complex"></a>
For models where the default value of the attributes is not sufficient, the
attribute's values should be set before calling `convert()`. In order to call
any constants use `tf.lite.constants.<CONSTANT_NAME>` as seen below with
`QUANTIZED_UINT8`. Run `help(tf.lite.TFLiteConverter)` in the Python
terminal for detailed documentation on the attributes.
Although the examples are demonstrated on GraphDefs containing only constants.
The same logic can be applied irrespective of the input data format.
### Exporting a quantized GraphDef <a name="complex_quant"></a>
The following example shows how to convert a quantized model into a TensorFlow
Lite FlatBuffer.
```python
import tensorflow as tf
img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3))
const = tf.constant([1., 2., 3.]) + tf.constant([1., 4., 4.])
val = img + const
out = tf.fake_quant_with_min_max_args(val, min=0., max=1., name="output")
with tf.Session() as sess:
converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out])
converter.inference_type = tf.lite.constants.QUANTIZED_UINT8
input_arrays = converter.get_input_arrays()
converter.quantized_input_stats = {input_arrays[0] : (0., 1.)} # mean, std_dev
tflite_model = converter.convert()
open("converted_model.tflite", "wb").write(tflite_model)
```
## Additional instructions
### Build from source code <a name="latest_package"></a>
In order to run the latest version of the TensorFlow Lite Converter Python API,
either install the nightly build with
[pip](https://www.tensorflow.org/install/pip) (recommended) or
[Docker](https://www.tensorflow.org/install/docker), or
[build the pip package from source](https://www.tensorflow.org/install/source).
### Converting models from TensorFlow 1.12 <a name="pre_tensorflow_1.12"></a>
Reference the following table to convert TensorFlow models to TensorFlow Lite in
and before TensorFlow 1.12. Run `help()` to get details of each API.
TensorFlow Version | Python API
------------------ | ---------------------------------
1.12 | `tf.contrib.lite.TFLiteConverter`
1.9-1.11 | `tf.contrib.lite.TocoConverter`
1.7-1.8 | `tf.contrib.lite.toco_convert`

1
tensorflow/lite/g3doc/r1/images/convert/workflow.svg Normal file
View File

File diff suppressed because one or more lines are too long
Side by Side

After

Width:  |  Height:  |  Size: 109 KiB

28
tensorflow/lite/g3doc/r2/convert/index.md
View File

@ -1,28 +0,0 @@
# TensorFlow Lite converter
The TensorFlow Lite converter takes a TensorFlow model and generates a
TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/) file
(`.tflite`). The converter supports
[SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model),
[`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview), and
[concrete functions](concrete_function.md).
Note: This page contains documentation on the converter API for TensorFlow 2.0.
The API for TensorFlow 1.X is available
[here](https://www.tensorflow.org/lite/convert/).
## Device deployment
The TensorFlow Lite `FlatBuffer` file is then deployed to a client device (e.g.
mobile, embedded) and run locally using the TensorFlow Lite interpreter. This
conversion process is shown in the diagram below:
![TFLite converter workflow](../images/convert/workflow.svg)
## Converting models
The TensorFlow Lite converter should be used from the
[Python API](python_api.md). Using the Python API makes it easier to convert
models as part of a model development pipeline and helps mitigate
[compatibility](../../guide/ops_compatibility.md) issues early on.
Alternatively, the [command line tool](cmdline.md) supports basic models.

284
tensorflow/lite/g3doc/r2/convert/python_api.md
View File

@ -1,284 +0,0 @@
# Converter Python API guide
This page provides examples on how to use the
[TensorFlow Lite converter](index.md) using the Python API in TensorFlow 2.0.
[TOC]
## Python API
The Python API for converting TensorFlow models to TensorFlow Lite in TensorFlow
2.0 is `tf.lite.TFLiteConverter`. `TFLiteConverter` provides the following
classmethods to convert a model based on the original model format:
* `TFLiteConverter.from_saved_model()`: Converts
[SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model).
* `TFLiteConverter.from_keras_model()`: Converts
[`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview).
* `TFLiteConverter.from_concrete_functions()`: Converts
[concrete functions](concrete_function.md).
Note: The TensorFlow Lite 2.0 alpha had a different version of the
`TFLiteConverter` API which only contained the classmethod
[`from_concrete_function`](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/lite/TFLiteConverter#from_concrete_function).
The API detailed in this document can be installed using the
[`tf-nightly-2.0-preview`](#installing_the_tensorflow_20_nightly_) pip install.
This document contains [example usages](#examples) of the API, a detailed list
of [changes in the API between 1.X and 2.0](#differences), and
[instructions](#versioning) on running the different versions of TensorFlow.
## Examples <a name="examples"></a>
### Converting a SavedModel <a name="saved_model"></a>
The following example shows how to convert a
[SavedModel](https://www.tensorflow.org/alpha/guide/saved_model) into a
TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/).
```python
import tensorflow as tf
# Construct a basic model.
root = tf.train.Checkpoint()
root.v1 = tf.Variable(3.)
root.v2 = tf.Variable(2.)
root.f = tf.function(lambda x: root.v1 * root.v2 * x)
# Save the model.
export_dir = "/tmp/test_saved_model"
input_data = tf.constant(1., shape=[1, 1])
to_save = root.f.get_concrete_function(input_data)
tf.saved_model.save(root, export_dir, to_save)
# Convert the model.
converter = tf.lite.TFLiteConverter.from_saved_model(export_dir)
tflite_model = converter.convert()
```
This API does not have the option of specifying the input shape of any input
arrays. If your model requires specifying the input shape, use the
[`from_concrete_functions`](#concrete_function) classmethod instead. The code
looks similar to the following:
```python
model = tf.saved_model.load(export_dir)
concrete_func = model.signatures[
tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY]
concrete_func.inputs[0].set_shape([1, 256, 256, 3])
converter = TFLiteConverter.from_concrete_functions([concrete_func])
```
### Converting a Keras model <a name="keras"></a>
The following example shows how to convert a
[`tf.keras` model](https://www.tensorflow.org/alpha/guide/keras/overview) into a
TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/).
```python
import tensorflow as tf
# Create a simple Keras model.
x = [-1, 0, 1, 2, 3, 4]
y = [-3, -1, 1, 3, 5, 7]
model = tf.keras.models.Sequential(
[tf.keras.layers.Dense(units=1, input_shape=[1])])
model.compile(optimizer='sgd', loss='mean_squared_error')
model.fit(x, y, epochs=50)
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
```
### Converting a concrete function <a name="concrete_function"></a>
The following example shows how to convert a TensorFlow
[concrete function](concrete_function.md) into a TensorFlow Lite
[`FlatBuffer`](https://google.github.io/flatbuffers/).
```python
import tensorflow as tf
# Construct a basic model.
root = tf.train.Checkpoint()
root.v1 = tf.Variable(3.)
root.v2 = tf.Variable(2.)
root.f = tf.function(lambda x: root.v1 * root.v2 * x)
# Create the concrete function.
input_data = tf.constant(1., shape=[1, 1])
concrete_func = root.f.get_concrete_function(input_data)
# Convert the model.
#
# `from_concrete_function` takes in a list of concrete functions, however,
# currently only supports converting one function at a time. Converting multiple
# functions is under development.
converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func])
tflite_model = converter.convert()
```
### End-to-end MobileNet conversion <a name="mobilenet"></a>
The following example shows how to convert and run inference on a pre-trained
`tf.keras` MobileNet model to TensorFlow Lite. It compares the results of the
TensorFlow and TensorFlow Lite model on random data. In order to load the model
from file, use `model_path` instead of `model_content`.
```python
import numpy as np
import tensorflow as tf
# Load the MobileNet tf.keras model.
model = tf.keras.applications.MobileNetV2(
weights="imagenet", input_shape=(224, 224, 3))
# Convert the model.
converter = tf.lite.TFLiteConverter.from_keras_model(model)
tflite_model = converter.convert()
# Load TFLite model and allocate tensors.
interpreter = tf.lite.Interpreter(model_content=tflite_model)
interpreter.allocate_tensors()
# Get input and output tensors.
input_details = interpreter.get_input_details()
output_details = interpreter.get_output_details()
# Test the TensorFlow Lite model on random input data.
input_shape = input_details[0]['shape']
input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32)
interpreter.set_tensor(input_details[0]['index'], input_data)
interpreter.invoke()
# The function `get_tensor()` returns a copy of the tensor data.
# Use `tensor()` in order to get a pointer to the tensor.
tflite_results = interpreter.get_tensor(output_details[0]['index'])
# Test the TensorFlow model on random input data.
tf_results = model(tf.constant(input_data))
# Compare the result.
for tf_result, tflite_result in zip(tf_results, tflite_results):
np.testing.assert_almost_equal(tf_result, tflite_result, decimal=5)
```
## Summary of changes in Python API between 1.X and 2.0 <a name="differences"></a>
The following section summarizes the changes in the Python API from 1.X to 2.0.
If any of the changes raise concerns, please file a
[GitHub issue](https://github.com/tensorflow/tensorflow/issues).
### Formats supported by `TFLiteConverter`
`TFLiteConverter` in 2.0 supports SavedModels and Keras model files generated in
both 1.X and 2.0. However, the conversion process no longer supports frozen
`GraphDefs` generated in 1.X. Users who want to convert frozen `GraphDefs` to
TensorFlow Lite should use `tf.compat.v1.TFLiteConverter`.
### Quantization-aware training
The following attributes and methods associated with
[quantization-aware training](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize)
have been removed from `TFLiteConverter` in TensorFlow 2.0:
* `inference_type`
* `inference_input_type`
* `quantized_input_stats`
* `default_ranges_stats`
* `reorder_across_fake_quant`
* `change_concat_input_ranges`
* `post_training_quantize` - Deprecated in the 1.X API
* `get_input_arrays()`
The rewriter function that supports quantization-aware training does not support
models generated by TensorFlow 2.0. Additionally, TensorFlow Lite’s quantization
API is being reworked and streamlined in a direction that supports
quantization-aware training through the Keras API. These attributes will be
removed in the 2.0 API until the new quantization API is launched. Users who
want to convert models generated by the rewriter function can use
`tf.compat.v1.TFLiteConverter`.
### Changes to `TFLiteConverter` attributes
The `target_ops` attribute has become an attribute of `TargetSpec` and renamed
to `supported_ops` in line with future additions to the optimization framework.
Additionally, the following attributes have been removed:
* `drop_control_dependency` (default: `True`) - Control flow is currently not
supported by TFLite so it is always `True`.
* _Graph visualization_ - The recommended approach for visualizing a
TensorFlow Lite graph in TensorFlow 2.0 will be to use
[visualize.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/tools/visualize.py).
Unlike GraphViz, it enables users to visualize the graph after post training
quantization has occurred. The following attributes related to graph
visualization will be removed:
* `output_format`
* `dump_graphviz_dir`
* `dump_graphviz_video`
### General API changes
#### Conversion methods
The following methods that were previously deprecated in 1.X will no longer be
exported in 2.0:
* `lite.toco_convert`
* `lite.TocoConverter`
#### `lite.constants`
The `lite.constants` API was removed in 2.0 in order to decrease duplication
between TensorFlow and TensorFlow Lite. The following list maps the
`lite.constant` type to the TensorFlow type:
* `lite.constants.FLOAT`: `tf.float32`
* `lite.constants.INT8`: `tf.int8`
* `lite.constants.INT32`: `tf.int32`
* `lite.constants.INT64`: `tf.int64`
* `lite.constants.STRING`: `tf.string`
* `lite.constants.QUANTIZED_UINT8`: `tf.uint8`
Additionally, `lite.constants.TFLITE` and `lite.constants.GRAPHVIZ_DOT` were
removed due to the deprecation of the `output_format` flag in `TFLiteConverter`.
#### `lite.OpHint`
The `OpHint` API is currently not available in 2.0 due to an incompatibility
with the 2.0 APIs. This API enables conversion of LSTM based models. Support for
LSTMs in 2.0 is being investigated. All related `lite.experimental` APIs have
been removed due to this issue.
## Installing TensorFlow <a name="versioning"></a>
### Installing the TensorFlow 2.0 nightly <a name="2.0-nightly"></a>
The TensorFlow 2.0 nightly can be installed using the following command:
```
pip install tf-nightly-2.0-preview
```
### Using TensorFlow 2.0 from a 1.X installation <a name="use-2.0-from-1.X"></a>
TensorFlow 2.0 can be enabled from recent 1.X installations using the following
code snippet.
```python
import tensorflow.compat.v2 as tf
tf.enable_v2_behavior()
```
### Build from source code <a name="latest_package"></a>
In order to run the latest version of the TensorFlow Lite Converter Python API,
either install the nightly build with
[pip](https://www.tensorflow.org/install/pip) (recommended) or
[Docker](https://www.tensorflow.org/install/docker), or
[build the pip package from source](https://www.tensorflow.org/install/source).

1
tensorflow/lite/g3doc/r2/images/convert/workflow.svg
View File

File diff suppressed because one or more lines are too long
Side by Side

Before

Width:  |  Height:  |  Size: 98 KiB