diff --git a/tensorflow/lite/g3doc/_book.yaml b/tensorflow/lite/g3doc/_book.yaml index 0919be9064b..5e0eeb6bd6d 100644 --- a/tensorflow/lite/g3doc/_book.yaml +++ b/tensorflow/lite/g3doc/_book.yaml @@ -36,22 +36,14 @@ upper_tabs: path: /lite/convert/ - title: "Python API" path: /lite/convert/python_api - - title: "Command line examples" - path: /lite/convert/cmdline_examples - - title: "Command line reference" - path: /lite/convert/cmdline_reference + - title: "Command line" + path: /lite/convert/cmdline - title: "Convert quantized models" path: /lite/convert/quantization - title: "Convert RNN models" path: /lite/convert/rnn - - - heading: "Convert a TensorFlow 2.0 model" - - title: "TensorFlow Lite converter and 2.0" - path: /lite/r2/convert/ - title: "Generate a concrete function" - path: /lite/r2/convert/concrete_function - - title: "Python API" - path: /lite/r2/convert/python_api + path: /lite/convert/concrete_function - heading: "Inference" - title: "Overview" diff --git a/tensorflow/lite/g3doc/r2/convert/cmdline.md b/tensorflow/lite/g3doc/convert/cmdline.md similarity index 95% rename from tensorflow/lite/g3doc/r2/convert/cmdline.md rename to tensorflow/lite/g3doc/convert/cmdline.md index 3155ee54230..bb6ecda3332 100644 --- a/tensorflow/lite/g3doc/r2/convert/cmdline.md +++ b/tensorflow/lite/g3doc/convert/cmdline.md @@ -41,7 +41,7 @@ install the nightly build using [pip](https://www.tensorflow.org/install/pip) or use `bazel`. An example can be seen below. ``` -bazel run //third_party/tensorflow/lite/python:tflite_convert -- \ +bazel run //tensorflow/lite/python:tflite_convert -- \ --saved_model_dir=/tmp/mobilenet_saved_model \ --output_file=/tmp/mobilenet.tflite ``` diff --git a/tensorflow/lite/g3doc/r2/convert/concrete_function.md b/tensorflow/lite/g3doc/convert/concrete_function.md similarity index 100% rename from tensorflow/lite/g3doc/r2/convert/concrete_function.md rename to tensorflow/lite/g3doc/convert/concrete_function.md diff --git a/tensorflow/lite/g3doc/convert/index.md b/tensorflow/lite/g3doc/convert/index.md index f9c6d9f6cfe..14445dbdc76 100644 --- a/tensorflow/lite/g3doc/convert/index.md +++ b/tensorflow/lite/g3doc/convert/index.md @@ -1,48 +1,28 @@ # TensorFlow Lite converter -The TensorFlow Lite converter is used to convert TensorFlow models into an -optimized [FlatBuffer](https://google.github.io/flatbuffers/) format, so that -they can be used by the TensorFlow Lite interpreter. +The TensorFlow Lite converter takes a TensorFlow model and generates a +TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/) file +(`.tflite`). The converter supports +[SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model), +[`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview), and +[concrete functions](concrete_function.md). -Note: This page contains documentation on the converter API for TensorFlow 1.x. -The API for TensorFlow 2.0 is available -[here](https://www.tensorflow.org/lite/r2/convert/). +Note: This page contains documentation on the converter API for TensorFlow 2.0. +The API for TensorFlow 1.X is available +[here](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/g3doc/r1/convert/index.md). -## FlatBuffers +## Device deployment -FlatBuffer is an efficient open-source cross-platform serialization library. It -is similar to -[protocol buffers](https://developers.google.com/protocol-buffers), with the -distinction that FlatBuffers do not need a parsing/unpacking step to a secondary -representation before data can be accessed, avoiding per-object memory -allocation. The code footprint of FlatBuffers is an order of magnitude smaller -than protocol buffers. - -## From model training to device deployment - -The TensorFlow Lite converter generates a TensorFlow Lite -[FlatBuffer](https://google.github.io/flatbuffers/) file (`.tflite`) from a -TensorFlow model. - -The converter supports the following input formats: - -* [SavedModels](https://www.tensorflow.org/guide/saved_model#using_savedmodel_with_estimators) -* Frozen `GraphDef`: Models generated by - [freeze_graph.py](https://www.tensorflow.org/code/tensorflow/python/tools/freeze_graph.py). -* `tf.keras` HDF5 models. -* Any model taken from a `tf.Session` (Python API only). - -The TensorFlow Lite `FlatBuffer` file is then deployed to a client device, and -the TensorFlow Lite interpreter uses the compressed model for on-device -inference. This conversion process is shown in the diagram below: +The TensorFlow Lite `FlatBuffer` file is then deployed to a client device (e.g. +mobile, embedded) and run locally using the TensorFlow Lite interpreter. This +conversion process is shown in the diagram below: ![TFLite converter workflow](../images/convert/workflow.svg) -## Options +## Converting models -The TensorFlow Lite Converter can be used from either of these two options: - -* [Python](python_api.md) (**Preferred**): Using the Python API makes it - easier to convert models as part of a model development pipeline, and helps - mitigate [compatibility](../tf_ops_compatibility.md) issues early on. -* [Command line](cmdline_examples.md) +The TensorFlow Lite converter should be used from the +[Python API](python_api.md). Using the Python API makes it easier to convert +models as part of a model development pipeline and helps mitigate +[compatibility](../guide/ops_compatibility.md) issues early on. Alternatively, +the [command line tool](cmdline.md) supports basic models. diff --git a/tensorflow/lite/g3doc/convert/python_api.md b/tensorflow/lite/g3doc/convert/python_api.md index 777c363e7fb..e27f1c83dbc 100644 --- a/tensorflow/lite/g3doc/convert/python_api.md +++ b/tensorflow/lite/g3doc/convert/python_api.md @@ -1,187 +1,280 @@ # Converter Python API guide -This page describes how to convert TensorFlow models into the TensorFlow Lite -format using the TensorFlow Lite Converter Python API. - -If you're looking for information about how to run a TensorFlow Lite model, -see [TensorFlow Lite inference](../guide/inference.md). - -Note: This page describes the converter in the TensorFlow nightly release, -installed using `pip install tf-nightly`. For docs describing older versions -reference ["Converting models from TensorFlow 1.12"](#pre_tensorflow_1.12). +This page provides examples on how to use the +[TensorFlow Lite converter](index.md) using the Python API in TensorFlow 2.0. [TOC] +## Python API -## High-level overview +The Python API for converting TensorFlow models to TensorFlow Lite in TensorFlow +2.0 is `tf.lite.TFLiteConverter`. `TFLiteConverter` provides the following +classmethods to convert a model based on the original model format: -While the TensorFlow Lite Converter can be used from the command line, it is -often convenient to use in a Python script as part of the model development -pipeline. This allows you to know early that you are designing a model that can -be targeted to devices with mobile. +* `TFLiteConverter.from_saved_model()`: Converts + [SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model). +* `TFLiteConverter.from_keras_model()`: Converts + [`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview). +* `TFLiteConverter.from_concrete_functions()`: Converts + [concrete functions](concrete_function.md). -## API +Note: The TensorFlow Lite 2.0 alpha had a different version of the +`TFLiteConverter` API which only contained the classmethod +[`from_concrete_function`](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/lite/TFLiteConverter#from_concrete_function). +The API detailed in this document can be installed using the +[`tf-nightly-2.0-preview`](#installing_the_tensorflow_20_nightly_) pip install. -The API for converting TensorFlow models to TensorFlow Lite is -`tf.lite.TFLiteConverter`, which provides class methods based on the original -format of the model. For example, `TFLiteConverter.from_session()` is available -for GraphDefs, `TFLiteConverter.from_saved_model()` is available for -SavedModels, and `TFLiteConverter.from_keras_model_file()` is available for -`tf.Keras` files. +This document contains [example usages](#examples) of the API, a detailed list +of [changes in the API between 1.X and 2.0](#differences), and +[instructions](#versioning) on running the different versions of TensorFlow. -Example usages for simple float-point models are shown in -[Basic Examples](#basic). Examples usages for more complex models is shown in -[Complex Examples](#complex). +## Examples -## Basic examples +### Converting a SavedModel -The following section shows examples of how to convert a basic float-point model -from each of the supported data formats into a TensorFlow Lite FlatBuffers. - -### Exporting a GraphDef from tf.Session - -The following example shows how to convert a TensorFlow GraphDef into a -TensorFlow Lite FlatBuffer from a `tf.Session` object. +The following example shows how to convert a +[SavedModel](https://www.tensorflow.org/alpha/guide/saved_model) into a +TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/). ```python import tensorflow as tf -img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3)) -var = tf.get_variable("weights", dtype=tf.float32, shape=(1, 64, 64, 3)) -val = img + var -out = tf.identity(val, name="out") +# Construct a basic model. +root = tf.train.Checkpoint() +root.v1 = tf.Variable(3.) +root.v2 = tf.Variable(2.) +root.f = tf.function(lambda x: root.v1 * root.v2 * x) -with tf.Session() as sess: - sess.run(tf.global_variables_initializer()) - converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out]) - tflite_model = converter.convert() - open("converted_model.tflite", "wb").write(tflite_model) -``` +# Save the model. +export_dir = "/tmp/test_saved_model" +input_data = tf.constant(1., shape=[1, 1]) +to_save = root.f.get_concrete_function(input_data) +tf.saved_model.save(root, export_dir, to_save) -### Exporting a GraphDef from file - -The following example shows how to convert a TensorFlow GraphDef into a -TensorFlow Lite FlatBuffer when the GraphDef is stored in a file. Both `.pb` and -`.pbtxt` files are accepted. - -The example uses -[Mobilenet_1.0_224](https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz). -The function only supports GraphDefs frozen using -[freeze_graph.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py). - -```python -import tensorflow as tf - -graph_def_file = "/path/to/Downloads/mobilenet_v1_1.0_224/frozen_graph.pb" -input_arrays = ["input"] -output_arrays = ["MobilenetV1/Predictions/Softmax"] - -converter = tf.lite.TFLiteConverter.from_frozen_graph( - graph_def_file, input_arrays, output_arrays) +# Convert the model. +converter = tf.lite.TFLiteConverter.from_saved_model(export_dir) tflite_model = converter.convert() -open("converted_model.tflite", "wb").write(tflite_model) ``` -### Exporting a SavedModel +This API does not have the option of specifying the input shape of any input +arrays. If your model requires specifying the input shape, use the +[`from_concrete_functions`](#concrete_function) classmethod instead. The code +looks similar to the following: -The following example shows how to convert a SavedModel into a TensorFlow Lite -FlatBuffer. +```python +model = tf.saved_model.load(export_dir) +concrete_func = model.signatures[ + tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY] +concrete_func.inputs[0].set_shape([1, 256, 256, 3]) +converter = TFLiteConverter.from_concrete_functions([concrete_func]) +``` + +### Converting a Keras model + +The following example shows how to convert a +[`tf.keras` model](https://www.tensorflow.org/alpha/guide/keras/overview) into a +TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/). ```python import tensorflow as tf -converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) +# Create a simple Keras model. +x = [-1, 0, 1, 2, 3, 4] +y = [-3, -1, 1, 3, 5, 7] + +model = tf.keras.models.Sequential( + [tf.keras.layers.Dense(units=1, input_shape=[1])]) +model.compile(optimizer='sgd', loss='mean_squared_error') +model.fit(x, y, epochs=50) + +# Convert the model. +converter = tf.lite.TFLiteConverter.from_keras_model(model) tflite_model = converter.convert() -open("converted_model.tflite", "wb").write(tflite_model) ``` -For more complex SavedModels, the optional parameters that can be passed into -`TFLiteConverter.from_saved_model()` are `input_arrays`, `input_shapes`, -`output_arrays`, `tag_set` and `signature_key`. Details of each parameter are -available by running `help(tf.lite.TFLiteConverter)`. +### Converting a concrete function -### Exporting a tf.keras File - -The following example shows how to convert a `tf.keras` model into a TensorFlow -Lite FlatBuffer. This example requires -[`h5py`](http://docs.h5py.org/en/latest/build.html) to be installed. +The following example shows how to convert a TensorFlow +[concrete function](concrete_function.md) into a TensorFlow Lite +[`FlatBuffer`](https://google.github.io/flatbuffers/). ```python import tensorflow as tf -converter = tf.lite.TFLiteConverter.from_keras_model_file("keras_model.h5") +# Construct a basic model. +root = tf.train.Checkpoint() +root.v1 = tf.Variable(3.) +root.v2 = tf.Variable(2.) +root.f = tf.function(lambda x: root.v1 * root.v2 * x) + +# Create the concrete function. +input_data = tf.constant(1., shape=[1, 1]) +concrete_func = root.f.get_concrete_function(input_data) + +# Convert the model. +# +# `from_concrete_function` takes in a list of concrete functions, however, +# currently only supports converting one function at a time. Converting multiple +# functions is under development. +converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func]) tflite_model = converter.convert() -open("converted_model.tflite", "wb").write(tflite_model) ``` -The `tf.keras` file must contain both the model and the weights. A comprehensive -example including model construction can be seen below. +### End-to-end MobileNet conversion + +The following example shows how to convert and run inference on a pre-trained +`tf.keras` MobileNet model to TensorFlow Lite. It compares the results of the +TensorFlow and TensorFlow Lite model on random data. In order to load the model +from file, use `model_path` instead of `model_content`. ```python import numpy as np import tensorflow as tf -# Generate tf.keras model. -model = tf.keras.models.Sequential() -model.add(tf.keras.layers.Dense(2, input_shape=(3,))) -model.add(tf.keras.layers.RepeatVector(3)) -model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(3))) -model.compile(loss=tf.keras.losses.MSE, - optimizer=tf.keras.optimizers.RMSprop(lr=0.0001), - metrics=[tf.keras.metrics.categorical_accuracy], - sample_weight_mode='temporal') +# Load the MobileNet tf.keras model. +model = tf.keras.applications.MobileNetV2( + weights="imagenet", input_shape=(224, 224, 3)) -x = np.random.random((1, 3)) -y = np.random.random((1, 3, 3)) -model.train_on_batch(x, y) -model.predict(x) - -# Save tf.keras model in HDF5 format. -keras_file = "keras_model.h5" -tf.keras.models.save_model(model, keras_file) - -# Convert to TensorFlow Lite model. -converter = tf.lite.TFLiteConverter.from_keras_model_file(keras_file) +# Convert the model. +converter = tf.lite.TFLiteConverter.from_keras_model(model) tflite_model = converter.convert() -open("converted_model.tflite", "wb").write(tflite_model) + +# Load TFLite model and allocate tensors. +interpreter = tf.lite.Interpreter(model_content=tflite_model) +interpreter.allocate_tensors() + +# Get input and output tensors. +input_details = interpreter.get_input_details() +output_details = interpreter.get_output_details() + +# Test the TensorFlow Lite model on random input data. +input_shape = input_details[0]['shape'] +input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32) +interpreter.set_tensor(input_details[0]['index'], input_data) + +interpreter.invoke() + +# The function `get_tensor()` returns a copy of the tensor data. +# Use `tensor()` in order to get a pointer to the tensor. +tflite_results = interpreter.get_tensor(output_details[0]['index']) + +# Test the TensorFlow model on random input data. +tf_results = model(tf.constant(input_data)) + +# Compare the result. +for tf_result, tflite_result in zip(tf_results, tflite_results): + np.testing.assert_almost_equal(tf_result, tflite_result, decimal=5) ``` -## Complex examples +## Summary of changes in Python API between 1.X and 2.0 -For models where the default value of the attributes is not sufficient, the -attribute's values should be set before calling `convert()`. In order to call -any constants use `tf.lite.constants.` as seen below with -`QUANTIZED_UINT8`. Run `help(tf.lite.TFLiteConverter)` in the Python -terminal for detailed documentation on the attributes. +The following section summarizes the changes in the Python API from 1.X to 2.0. +If any of the changes raise concerns, please file a +[GitHub issue](https://github.com/tensorflow/tensorflow/issues). -Although the examples are demonstrated on GraphDefs containing only constants. -The same logic can be applied irrespective of the input data format. +### Formats supported by `TFLiteConverter` -### Exporting a quantized GraphDef +`TFLiteConverter` in 2.0 supports SavedModels and Keras model files generated in +both 1.X and 2.0. However, the conversion process no longer supports frozen +`GraphDefs` generated in 1.X. Users who want to convert frozen `GraphDefs` to +TensorFlow Lite should use `tf.compat.v1.TFLiteConverter`. -The following example shows how to convert a quantized model into a TensorFlow -Lite FlatBuffer. +### Quantization-aware training + +The following attributes and methods associated with +[quantization-aware training](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize) +have been removed from `TFLiteConverter` in TensorFlow 2.0: + +* `inference_type` +* `inference_input_type` +* `quantized_input_stats` +* `default_ranges_stats` +* `reorder_across_fake_quant` +* `change_concat_input_ranges` +* `post_training_quantize` - Deprecated in the 1.X API +* `get_input_arrays()` + +The rewriter function that supports quantization-aware training does not support +models generated by TensorFlow 2.0. Additionally, TensorFlow Lite’s quantization +API is being reworked and streamlined in a direction that supports +quantization-aware training through the Keras API. These attributes will be +removed in the 2.0 API until the new quantization API is launched. Users who +want to convert models generated by the rewriter function can use +`tf.compat.v1.TFLiteConverter`. + +### Changes to `TFLiteConverter` attributes + +The `target_ops` attribute has become an attribute of `TargetSpec` and renamed +to `supported_ops` in line with future additions to the optimization framework. + +Additionally, the following attributes have been removed: + +* `drop_control_dependency` (default: `True`) - Control flow is currently not + supported by TFLite so it is always `True`. +* _Graph visualization_ - The recommended approach for visualizing a + TensorFlow Lite graph in TensorFlow 2.0 will be to use + [visualize.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/tools/visualize.py). + Unlike GraphViz, it enables users to visualize the graph after post training + quantization has occurred. The following attributes related to graph + visualization will be removed: + * `output_format` + * `dump_graphviz_dir` + * `dump_graphviz_video` + +### General API changes + +#### Conversion methods + +The following methods that were previously deprecated in 1.X will no longer be +exported in 2.0: + +* `lite.toco_convert` +* `lite.TocoConverter` + +#### `lite.constants` + +The `lite.constants` API was removed in 2.0 in order to decrease duplication +between TensorFlow and TensorFlow Lite. The following list maps the +`lite.constant` type to the TensorFlow type: + +* `lite.constants.FLOAT`: `tf.float32` +* `lite.constants.INT8`: `tf.int8` +* `lite.constants.INT32`: `tf.int32` +* `lite.constants.INT64`: `tf.int64` +* `lite.constants.STRING`: `tf.string` +* `lite.constants.QUANTIZED_UINT8`: `tf.uint8` + +Additionally, `lite.constants.TFLITE` and `lite.constants.GRAPHVIZ_DOT` were +removed due to the deprecation of the `output_format` flag in `TFLiteConverter`. + +#### `lite.OpHint` + +The `OpHint` API is currently not available in 2.0 due to an incompatibility +with the 2.0 APIs. This API enables conversion of LSTM based models. Support for +LSTMs in 2.0 is being investigated. All related `lite.experimental` APIs have +been removed due to this issue. + +## Installing TensorFlow + +### Installing the TensorFlow 2.0 nightly + +The TensorFlow 2.0 nightly can be installed using the following command: + +``` +pip install tf-nightly-2.0-preview +``` + +### Using TensorFlow 2.0 from a 1.X installation + +TensorFlow 2.0 can be enabled from recent 1.X installations using the following +code snippet. ```python -import tensorflow as tf +import tensorflow.compat.v2 as tf -img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3)) -const = tf.constant([1., 2., 3.]) + tf.constant([1., 4., 4.]) -val = img + const -out = tf.fake_quant_with_min_max_args(val, min=0., max=1., name="output") - -with tf.Session() as sess: - converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out]) - converter.inference_type = tf.lite.constants.QUANTIZED_UINT8 - input_arrays = converter.get_input_arrays() - converter.quantized_input_stats = {input_arrays[0] : (0., 1.)} # mean, std_dev - tflite_model = converter.convert() - open("converted_model.tflite", "wb").write(tflite_model) +tf.enable_v2_behavior() ``` - -## Additional instructions - ### Build from source code In order to run the latest version of the TensorFlow Lite Converter Python API, @@ -189,14 +282,3 @@ either install the nightly build with [pip](https://www.tensorflow.org/install/pip) (recommended) or [Docker](https://www.tensorflow.org/install/docker), or [build the pip package from source](https://www.tensorflow.org/install/source). - -### Converting models from TensorFlow 1.12 - -Reference the following table to convert TensorFlow models to TensorFlow Lite in -and before TensorFlow 1.12. Run `help()` to get details of each API. - -TensorFlow Version | Python API ------------------- | --------------------------------- -1.12 | `tf.contrib.lite.TFLiteConverter` -1.9-1.11 | `tf.contrib.lite.TocoConverter` -1.7-1.8 | `tf.contrib.lite.toco_convert` diff --git a/tensorflow/lite/g3doc/images/convert/workflow.svg b/tensorflow/lite/g3doc/images/convert/workflow.svg index 3dfcbd67d89..c0c45628952 100644 --- a/tensorflow/lite/g3doc/images/convert/workflow.svg +++ b/tensorflow/lite/g3doc/images/convert/workflow.svg @@ -1 +1 @@ - \ No newline at end of file + diff --git a/tensorflow/lite/g3doc/convert/cmdline_examples.md b/tensorflow/lite/g3doc/r1/convert/cmdline_examples.md similarity index 99% rename from tensorflow/lite/g3doc/convert/cmdline_examples.md rename to tensorflow/lite/g3doc/r1/convert/cmdline_examples.md index 139a6c9b521..fc8318a5585 100644 --- a/tensorflow/lite/g3doc/convert/cmdline_examples.md +++ b/tensorflow/lite/g3doc/r1/convert/cmdline_examples.md @@ -18,7 +18,7 @@ There are two approaches to running the converter in the command line. [clone the TensorFlow repository](https://www.tensorflow.org/install/source) and use `bazel`. * Example: `bazel run - //tensorflow/lite/python:tflite_convert -- + //third_party/tensorflow/lite/python:tflite_convert -- --output_file=...` ### Converting models prior to TensorFlow 1.9 diff --git a/tensorflow/lite/g3doc/convert/cmdline_reference.md b/tensorflow/lite/g3doc/r1/convert/cmdline_reference.md similarity index 100% rename from tensorflow/lite/g3doc/convert/cmdline_reference.md rename to tensorflow/lite/g3doc/r1/convert/cmdline_reference.md diff --git a/tensorflow/lite/g3doc/r1/convert/index.md b/tensorflow/lite/g3doc/r1/convert/index.md new file mode 100644 index 00000000000..f9c6d9f6cfe --- /dev/null +++ b/tensorflow/lite/g3doc/r1/convert/index.md @@ -0,0 +1,48 @@ +# TensorFlow Lite converter + +The TensorFlow Lite converter is used to convert TensorFlow models into an +optimized [FlatBuffer](https://google.github.io/flatbuffers/) format, so that +they can be used by the TensorFlow Lite interpreter. + +Note: This page contains documentation on the converter API for TensorFlow 1.x. +The API for TensorFlow 2.0 is available +[here](https://www.tensorflow.org/lite/r2/convert/). + +## FlatBuffers + +FlatBuffer is an efficient open-source cross-platform serialization library. It +is similar to +[protocol buffers](https://developers.google.com/protocol-buffers), with the +distinction that FlatBuffers do not need a parsing/unpacking step to a secondary +representation before data can be accessed, avoiding per-object memory +allocation. The code footprint of FlatBuffers is an order of magnitude smaller +than protocol buffers. + +## From model training to device deployment + +The TensorFlow Lite converter generates a TensorFlow Lite +[FlatBuffer](https://google.github.io/flatbuffers/) file (`.tflite`) from a +TensorFlow model. + +The converter supports the following input formats: + +* [SavedModels](https://www.tensorflow.org/guide/saved_model#using_savedmodel_with_estimators) +* Frozen `GraphDef`: Models generated by + [freeze_graph.py](https://www.tensorflow.org/code/tensorflow/python/tools/freeze_graph.py). +* `tf.keras` HDF5 models. +* Any model taken from a `tf.Session` (Python API only). + +The TensorFlow Lite `FlatBuffer` file is then deployed to a client device, and +the TensorFlow Lite interpreter uses the compressed model for on-device +inference. This conversion process is shown in the diagram below: + +![TFLite converter workflow](../images/convert/workflow.svg) + +## Options + +The TensorFlow Lite Converter can be used from either of these two options: + +* [Python](python_api.md) (**Preferred**): Using the Python API makes it + easier to convert models as part of a model development pipeline, and helps + mitigate [compatibility](../tf_ops_compatibility.md) issues early on. +* [Command line](cmdline_examples.md) diff --git a/tensorflow/lite/g3doc/r1/convert/python_api.md b/tensorflow/lite/g3doc/r1/convert/python_api.md new file mode 100644 index 00000000000..777c363e7fb --- /dev/null +++ b/tensorflow/lite/g3doc/r1/convert/python_api.md @@ -0,0 +1,202 @@ +# Converter Python API guide + +This page describes how to convert TensorFlow models into the TensorFlow Lite +format using the TensorFlow Lite Converter Python API. + +If you're looking for information about how to run a TensorFlow Lite model, +see [TensorFlow Lite inference](../guide/inference.md). + +Note: This page describes the converter in the TensorFlow nightly release, +installed using `pip install tf-nightly`. For docs describing older versions +reference ["Converting models from TensorFlow 1.12"](#pre_tensorflow_1.12). + +[TOC] + + +## High-level overview + +While the TensorFlow Lite Converter can be used from the command line, it is +often convenient to use in a Python script as part of the model development +pipeline. This allows you to know early that you are designing a model that can +be targeted to devices with mobile. + +## API + +The API for converting TensorFlow models to TensorFlow Lite is +`tf.lite.TFLiteConverter`, which provides class methods based on the original +format of the model. For example, `TFLiteConverter.from_session()` is available +for GraphDefs, `TFLiteConverter.from_saved_model()` is available for +SavedModels, and `TFLiteConverter.from_keras_model_file()` is available for +`tf.Keras` files. + +Example usages for simple float-point models are shown in +[Basic Examples](#basic). Examples usages for more complex models is shown in +[Complex Examples](#complex). + +## Basic examples + +The following section shows examples of how to convert a basic float-point model +from each of the supported data formats into a TensorFlow Lite FlatBuffers. + +### Exporting a GraphDef from tf.Session + +The following example shows how to convert a TensorFlow GraphDef into a +TensorFlow Lite FlatBuffer from a `tf.Session` object. + +```python +import tensorflow as tf + +img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3)) +var = tf.get_variable("weights", dtype=tf.float32, shape=(1, 64, 64, 3)) +val = img + var +out = tf.identity(val, name="out") + +with tf.Session() as sess: + sess.run(tf.global_variables_initializer()) + converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out]) + tflite_model = converter.convert() + open("converted_model.tflite", "wb").write(tflite_model) +``` + +### Exporting a GraphDef from file + +The following example shows how to convert a TensorFlow GraphDef into a +TensorFlow Lite FlatBuffer when the GraphDef is stored in a file. Both `.pb` and +`.pbtxt` files are accepted. + +The example uses +[Mobilenet_1.0_224](https://storage.googleapis.com/download.tensorflow.org/models/mobilenet_v1_1.0_224_frozen.tgz). +The function only supports GraphDefs frozen using +[freeze_graph.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/python/tools/freeze_graph.py). + +```python +import tensorflow as tf + +graph_def_file = "/path/to/Downloads/mobilenet_v1_1.0_224/frozen_graph.pb" +input_arrays = ["input"] +output_arrays = ["MobilenetV1/Predictions/Softmax"] + +converter = tf.lite.TFLiteConverter.from_frozen_graph( + graph_def_file, input_arrays, output_arrays) +tflite_model = converter.convert() +open("converted_model.tflite", "wb").write(tflite_model) +``` + +### Exporting a SavedModel + +The following example shows how to convert a SavedModel into a TensorFlow Lite +FlatBuffer. + +```python +import tensorflow as tf + +converter = tf.lite.TFLiteConverter.from_saved_model(saved_model_dir) +tflite_model = converter.convert() +open("converted_model.tflite", "wb").write(tflite_model) +``` + +For more complex SavedModels, the optional parameters that can be passed into +`TFLiteConverter.from_saved_model()` are `input_arrays`, `input_shapes`, +`output_arrays`, `tag_set` and `signature_key`. Details of each parameter are +available by running `help(tf.lite.TFLiteConverter)`. + +### Exporting a tf.keras File + +The following example shows how to convert a `tf.keras` model into a TensorFlow +Lite FlatBuffer. This example requires +[`h5py`](http://docs.h5py.org/en/latest/build.html) to be installed. + +```python +import tensorflow as tf + +converter = tf.lite.TFLiteConverter.from_keras_model_file("keras_model.h5") +tflite_model = converter.convert() +open("converted_model.tflite", "wb").write(tflite_model) +``` + +The `tf.keras` file must contain both the model and the weights. A comprehensive +example including model construction can be seen below. + +```python +import numpy as np +import tensorflow as tf + +# Generate tf.keras model. +model = tf.keras.models.Sequential() +model.add(tf.keras.layers.Dense(2, input_shape=(3,))) +model.add(tf.keras.layers.RepeatVector(3)) +model.add(tf.keras.layers.TimeDistributed(tf.keras.layers.Dense(3))) +model.compile(loss=tf.keras.losses.MSE, + optimizer=tf.keras.optimizers.RMSprop(lr=0.0001), + metrics=[tf.keras.metrics.categorical_accuracy], + sample_weight_mode='temporal') + +x = np.random.random((1, 3)) +y = np.random.random((1, 3, 3)) +model.train_on_batch(x, y) +model.predict(x) + +# Save tf.keras model in HDF5 format. +keras_file = "keras_model.h5" +tf.keras.models.save_model(model, keras_file) + +# Convert to TensorFlow Lite model. +converter = tf.lite.TFLiteConverter.from_keras_model_file(keras_file) +tflite_model = converter.convert() +open("converted_model.tflite", "wb").write(tflite_model) +``` + +## Complex examples + +For models where the default value of the attributes is not sufficient, the +attribute's values should be set before calling `convert()`. In order to call +any constants use `tf.lite.constants.` as seen below with +`QUANTIZED_UINT8`. Run `help(tf.lite.TFLiteConverter)` in the Python +terminal for detailed documentation on the attributes. + +Although the examples are demonstrated on GraphDefs containing only constants. +The same logic can be applied irrespective of the input data format. + +### Exporting a quantized GraphDef + +The following example shows how to convert a quantized model into a TensorFlow +Lite FlatBuffer. + +```python +import tensorflow as tf + +img = tf.placeholder(name="img", dtype=tf.float32, shape=(1, 64, 64, 3)) +const = tf.constant([1., 2., 3.]) + tf.constant([1., 4., 4.]) +val = img + const +out = tf.fake_quant_with_min_max_args(val, min=0., max=1., name="output") + +with tf.Session() as sess: + converter = tf.lite.TFLiteConverter.from_session(sess, [img], [out]) + converter.inference_type = tf.lite.constants.QUANTIZED_UINT8 + input_arrays = converter.get_input_arrays() + converter.quantized_input_stats = {input_arrays[0] : (0., 1.)} # mean, std_dev + tflite_model = converter.convert() + open("converted_model.tflite", "wb").write(tflite_model) +``` + + +## Additional instructions + +### Build from source code + +In order to run the latest version of the TensorFlow Lite Converter Python API, +either install the nightly build with +[pip](https://www.tensorflow.org/install/pip) (recommended) or +[Docker](https://www.tensorflow.org/install/docker), or +[build the pip package from source](https://www.tensorflow.org/install/source). + +### Converting models from TensorFlow 1.12 + +Reference the following table to convert TensorFlow models to TensorFlow Lite in +and before TensorFlow 1.12. Run `help()` to get details of each API. + +TensorFlow Version | Python API +------------------ | --------------------------------- +1.12 | `tf.contrib.lite.TFLiteConverter` +1.9-1.11 | `tf.contrib.lite.TocoConverter` +1.7-1.8 | `tf.contrib.lite.toco_convert` diff --git a/tensorflow/lite/g3doc/r1/images/convert/workflow.svg b/tensorflow/lite/g3doc/r1/images/convert/workflow.svg new file mode 100644 index 00000000000..3dfcbd67d89 --- /dev/null +++ b/tensorflow/lite/g3doc/r1/images/convert/workflow.svg @@ -0,0 +1 @@ + \ No newline at end of file diff --git a/tensorflow/lite/g3doc/r2/convert/index.md b/tensorflow/lite/g3doc/r2/convert/index.md deleted file mode 100644 index 15755dd4e34..00000000000 --- a/tensorflow/lite/g3doc/r2/convert/index.md +++ /dev/null @@ -1,28 +0,0 @@ -# TensorFlow Lite converter - -The TensorFlow Lite converter takes a TensorFlow model and generates a -TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/) file -(`.tflite`). The converter supports -[SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model), -[`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview), and -[concrete functions](concrete_function.md). - -Note: This page contains documentation on the converter API for TensorFlow 2.0. -The API for TensorFlow 1.X is available -[here](https://www.tensorflow.org/lite/convert/). - -## Device deployment - -The TensorFlow Lite `FlatBuffer` file is then deployed to a client device (e.g. -mobile, embedded) and run locally using the TensorFlow Lite interpreter. This -conversion process is shown in the diagram below: - -![TFLite converter workflow](../images/convert/workflow.svg) - -## Converting models - -The TensorFlow Lite converter should be used from the -[Python API](python_api.md). Using the Python API makes it easier to convert -models as part of a model development pipeline and helps mitigate -[compatibility](../../guide/ops_compatibility.md) issues early on. -Alternatively, the [command line tool](cmdline.md) supports basic models. diff --git a/tensorflow/lite/g3doc/r2/convert/python_api.md b/tensorflow/lite/g3doc/r2/convert/python_api.md deleted file mode 100644 index e27f1c83dbc..00000000000 --- a/tensorflow/lite/g3doc/r2/convert/python_api.md +++ /dev/null @@ -1,284 +0,0 @@ -# Converter Python API guide - -This page provides examples on how to use the -[TensorFlow Lite converter](index.md) using the Python API in TensorFlow 2.0. - -[TOC] - -## Python API - -The Python API for converting TensorFlow models to TensorFlow Lite in TensorFlow -2.0 is `tf.lite.TFLiteConverter`. `TFLiteConverter` provides the following -classmethods to convert a model based on the original model format: - -* `TFLiteConverter.from_saved_model()`: Converts - [SavedModel directories](https://www.tensorflow.org/alpha/guide/saved_model). -* `TFLiteConverter.from_keras_model()`: Converts - [`tf.keras` models](https://www.tensorflow.org/alpha/guide/keras/overview). -* `TFLiteConverter.from_concrete_functions()`: Converts - [concrete functions](concrete_function.md). - -Note: The TensorFlow Lite 2.0 alpha had a different version of the -`TFLiteConverter` API which only contained the classmethod -[`from_concrete_function`](https://www.tensorflow.org/versions/r2.0/api_docs/python/tf/lite/TFLiteConverter#from_concrete_function). -The API detailed in this document can be installed using the -[`tf-nightly-2.0-preview`](#installing_the_tensorflow_20_nightly_) pip install. - -This document contains [example usages](#examples) of the API, a detailed list -of [changes in the API between 1.X and 2.0](#differences), and -[instructions](#versioning) on running the different versions of TensorFlow. - -## Examples - -### Converting a SavedModel - -The following example shows how to convert a -[SavedModel](https://www.tensorflow.org/alpha/guide/saved_model) into a -TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/). - -```python -import tensorflow as tf - -# Construct a basic model. -root = tf.train.Checkpoint() -root.v1 = tf.Variable(3.) -root.v2 = tf.Variable(2.) -root.f = tf.function(lambda x: root.v1 * root.v2 * x) - -# Save the model. -export_dir = "/tmp/test_saved_model" -input_data = tf.constant(1., shape=[1, 1]) -to_save = root.f.get_concrete_function(input_data) -tf.saved_model.save(root, export_dir, to_save) - -# Convert the model. -converter = tf.lite.TFLiteConverter.from_saved_model(export_dir) -tflite_model = converter.convert() -``` - -This API does not have the option of specifying the input shape of any input -arrays. If your model requires specifying the input shape, use the -[`from_concrete_functions`](#concrete_function) classmethod instead. The code -looks similar to the following: - -```python -model = tf.saved_model.load(export_dir) -concrete_func = model.signatures[ - tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY] -concrete_func.inputs[0].set_shape([1, 256, 256, 3]) -converter = TFLiteConverter.from_concrete_functions([concrete_func]) -``` - -### Converting a Keras model - -The following example shows how to convert a -[`tf.keras` model](https://www.tensorflow.org/alpha/guide/keras/overview) into a -TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/). - -```python -import tensorflow as tf - -# Create a simple Keras model. -x = [-1, 0, 1, 2, 3, 4] -y = [-3, -1, 1, 3, 5, 7] - -model = tf.keras.models.Sequential( - [tf.keras.layers.Dense(units=1, input_shape=[1])]) -model.compile(optimizer='sgd', loss='mean_squared_error') -model.fit(x, y, epochs=50) - -# Convert the model. -converter = tf.lite.TFLiteConverter.from_keras_model(model) -tflite_model = converter.convert() -``` - -### Converting a concrete function - -The following example shows how to convert a TensorFlow -[concrete function](concrete_function.md) into a TensorFlow Lite -[`FlatBuffer`](https://google.github.io/flatbuffers/). - -```python -import tensorflow as tf - -# Construct a basic model. -root = tf.train.Checkpoint() -root.v1 = tf.Variable(3.) -root.v2 = tf.Variable(2.) -root.f = tf.function(lambda x: root.v1 * root.v2 * x) - -# Create the concrete function. -input_data = tf.constant(1., shape=[1, 1]) -concrete_func = root.f.get_concrete_function(input_data) - -# Convert the model. -# -# `from_concrete_function` takes in a list of concrete functions, however, -# currently only supports converting one function at a time. Converting multiple -# functions is under development. -converter = tf.lite.TFLiteConverter.from_concrete_functions([concrete_func]) -tflite_model = converter.convert() -``` - -### End-to-end MobileNet conversion - -The following example shows how to convert and run inference on a pre-trained -`tf.keras` MobileNet model to TensorFlow Lite. It compares the results of the -TensorFlow and TensorFlow Lite model on random data. In order to load the model -from file, use `model_path` instead of `model_content`. - -```python -import numpy as np -import tensorflow as tf - -# Load the MobileNet tf.keras model. -model = tf.keras.applications.MobileNetV2( - weights="imagenet", input_shape=(224, 224, 3)) - -# Convert the model. -converter = tf.lite.TFLiteConverter.from_keras_model(model) -tflite_model = converter.convert() - -# Load TFLite model and allocate tensors. -interpreter = tf.lite.Interpreter(model_content=tflite_model) -interpreter.allocate_tensors() - -# Get input and output tensors. -input_details = interpreter.get_input_details() -output_details = interpreter.get_output_details() - -# Test the TensorFlow Lite model on random input data. -input_shape = input_details[0]['shape'] -input_data = np.array(np.random.random_sample(input_shape), dtype=np.float32) -interpreter.set_tensor(input_details[0]['index'], input_data) - -interpreter.invoke() - -# The function `get_tensor()` returns a copy of the tensor data. -# Use `tensor()` in order to get a pointer to the tensor. -tflite_results = interpreter.get_tensor(output_details[0]['index']) - -# Test the TensorFlow model on random input data. -tf_results = model(tf.constant(input_data)) - -# Compare the result. -for tf_result, tflite_result in zip(tf_results, tflite_results): - np.testing.assert_almost_equal(tf_result, tflite_result, decimal=5) -``` - -## Summary of changes in Python API between 1.X and 2.0 - -The following section summarizes the changes in the Python API from 1.X to 2.0. -If any of the changes raise concerns, please file a -[GitHub issue](https://github.com/tensorflow/tensorflow/issues). - -### Formats supported by `TFLiteConverter` - -`TFLiteConverter` in 2.0 supports SavedModels and Keras model files generated in -both 1.X and 2.0. However, the conversion process no longer supports frozen -`GraphDefs` generated in 1.X. Users who want to convert frozen `GraphDefs` to -TensorFlow Lite should use `tf.compat.v1.TFLiteConverter`. - -### Quantization-aware training - -The following attributes and methods associated with -[quantization-aware training](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/contrib/quantize) -have been removed from `TFLiteConverter` in TensorFlow 2.0: - -* `inference_type` -* `inference_input_type` -* `quantized_input_stats` -* `default_ranges_stats` -* `reorder_across_fake_quant` -* `change_concat_input_ranges` -* `post_training_quantize` - Deprecated in the 1.X API -* `get_input_arrays()` - -The rewriter function that supports quantization-aware training does not support -models generated by TensorFlow 2.0. Additionally, TensorFlow Lite’s quantization -API is being reworked and streamlined in a direction that supports -quantization-aware training through the Keras API. These attributes will be -removed in the 2.0 API until the new quantization API is launched. Users who -want to convert models generated by the rewriter function can use -`tf.compat.v1.TFLiteConverter`. - -### Changes to `TFLiteConverter` attributes - -The `target_ops` attribute has become an attribute of `TargetSpec` and renamed -to `supported_ops` in line with future additions to the optimization framework. - -Additionally, the following attributes have been removed: - -* `drop_control_dependency` (default: `True`) - Control flow is currently not - supported by TFLite so it is always `True`. -* _Graph visualization_ - The recommended approach for visualizing a - TensorFlow Lite graph in TensorFlow 2.0 will be to use - [visualize.py](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/tools/visualize.py). - Unlike GraphViz, it enables users to visualize the graph after post training - quantization has occurred. The following attributes related to graph - visualization will be removed: - * `output_format` - * `dump_graphviz_dir` - * `dump_graphviz_video` - -### General API changes - -#### Conversion methods - -The following methods that were previously deprecated in 1.X will no longer be -exported in 2.0: - -* `lite.toco_convert` -* `lite.TocoConverter` - -#### `lite.constants` - -The `lite.constants` API was removed in 2.0 in order to decrease duplication -between TensorFlow and TensorFlow Lite. The following list maps the -`lite.constant` type to the TensorFlow type: - -* `lite.constants.FLOAT`: `tf.float32` -* `lite.constants.INT8`: `tf.int8` -* `lite.constants.INT32`: `tf.int32` -* `lite.constants.INT64`: `tf.int64` -* `lite.constants.STRING`: `tf.string` -* `lite.constants.QUANTIZED_UINT8`: `tf.uint8` - -Additionally, `lite.constants.TFLITE` and `lite.constants.GRAPHVIZ_DOT` were -removed due to the deprecation of the `output_format` flag in `TFLiteConverter`. - -#### `lite.OpHint` - -The `OpHint` API is currently not available in 2.0 due to an incompatibility -with the 2.0 APIs. This API enables conversion of LSTM based models. Support for -LSTMs in 2.0 is being investigated. All related `lite.experimental` APIs have -been removed due to this issue. - -## Installing TensorFlow - -### Installing the TensorFlow 2.0 nightly - -The TensorFlow 2.0 nightly can be installed using the following command: - -``` -pip install tf-nightly-2.0-preview -``` - -### Using TensorFlow 2.0 from a 1.X installation - -TensorFlow 2.0 can be enabled from recent 1.X installations using the following -code snippet. - -```python -import tensorflow.compat.v2 as tf - -tf.enable_v2_behavior() -``` - -### Build from source code - -In order to run the latest version of the TensorFlow Lite Converter Python API, -either install the nightly build with -[pip](https://www.tensorflow.org/install/pip) (recommended) or -[Docker](https://www.tensorflow.org/install/docker), or -[build the pip package from source](https://www.tensorflow.org/install/source). diff --git a/tensorflow/lite/g3doc/r2/images/convert/workflow.svg b/tensorflow/lite/g3doc/r2/images/convert/workflow.svg deleted file mode 100644 index c0c45628952..00000000000 --- a/tensorflow/lite/g3doc/r2/images/convert/workflow.svg +++ /dev/null @@ -1 +0,0 @@ -