diff --git a/tensorflow/lite/g3doc/_book.yaml b/tensorflow/lite/g3doc/_book.yaml index a8c9750bbc0..f1d20403103 100644 --- a/tensorflow/lite/g3doc/_book.yaml +++ b/tensorflow/lite/g3doc/_book.yaml @@ -42,8 +42,6 @@ upper_tabs: path: /lite/convert/quantization - title: "Convert RNN models" path: /lite/convert/rnn - - title: "Generate a concrete function" - path: /lite/convert/concrete_function - heading: "Inference" - title: "Overview" diff --git a/tensorflow/lite/g3doc/convert/concrete_function.md b/tensorflow/lite/g3doc/convert/concrete_function.md deleted file mode 100644 index 32a86346399..00000000000 --- a/tensorflow/lite/g3doc/convert/concrete_function.md +++ /dev/null @@ -1,208 +0,0 @@ -# Generate a concrete function - -In order to convert TensorFlow 2.0 models to TensorFlow Lite, the model needs to -be exported as a concrete function. This document outlines what a concrete -function is and how to generate one for an existing model. - -[TOC] - -## Background - -In TensorFlow 2.0, eager execution is on by default. TensorFlow's eager -execution is an imperative programming environment that evaluates operations -immediately, without building graphs. Operations return concrete values instead -of constructing a computational graph to run later. A detailed guide on eager -execution is available -[here](https://www.tensorflow.org/guide/eager). - -While running imperatively with eager execution makes development and debugging -more interactive, it doesn't allow for deploying on-device. The `tf.function` -API makes it possible to save models as graphs, which is required to run -TensorFlow Lite in 2.0. All operations wrapped in the `tf.function` decorator -can be exported as a graph which can then be converted to the TensorFlow Lite -FlatBuffer format. - -## Terminology - -The following terminology is used in this document: - -* **Signature** - The inputs and outputs for a set of operations. -* **Concrete function** - Graph with a single signature. -* **Polymorphic function** - Python callable that encapsulates several - concrete function graphs behind one API. - -## Methodology - -This section describes how to export a concrete function. - -### Annotate functions with `tf.function` - -Annotating a function with `tf.function` generates a *polymorphic function* -containing those operations. All operations that are not annotated with -`tf.function` will be evaluated with eager execution. The examples below show -how to use `tf.function`. - -```python -@tf.function -def pow(x): - return x ** 2 -``` - -```python -tf.function(lambda x : x ** 2) -``` - -### Create an object to save - -The `tf.function` can be optionally stored as part of a `tf.Module` object. -Variables should only be defined once within the `tf.Module`. The examples below -show two different approaches for creating a class that derives `Checkpoint`. - -```python -class BasicModel(tf.Module): - - def __init__(self): - self.const = None - - @tf.function - def pow(self, x): - if self.const is None: - self.const = tf.Variable(2.) - return x ** self.const - -root = BasicModel() -``` - -```python -root = tf.Module() -root.const = tf.Variable(2.) -root.pow = tf.function(lambda x : x ** root.const) -``` - -### Exporting the concrete function - -The concrete function defines a graph that can be converted to TensorFlow Lite -model or be exported to a SavedModel. In order to export a concrete function -from the polymorphic function, the signature needs to be defined. The signature -can be defined the following ways: - -* Define `input_signature` parameter in `tf.function`. -* Pass in `tf.TensorSpec` into `get_concrete_function`: e.g. - `tf.TensorSpec(shape=[1], dtype=tf.float32)`. -* Pass in a sample input tensor into `get_concrete_function`: e.g. - `tf.constant(1., shape=[1])`. - -The follow example shows how to define the `input_signature` parameter for -`tf.function`. - -```python -class BasicModel(tf.Module): - - def __init__(self): - self.const = None - - @tf.function(input_signature=[tf.TensorSpec(shape=[1], dtype=tf.float32)]) - def pow(self, x): - if self.const is None: - self.const = tf.Variable(2.) - return x ** self.const - -# Create the tf.Module object. -root = BasicModel() - -# Get the concrete function. -concrete_func = root.pow.get_concrete_function() -``` - -The example below passes in a sample input tensor into `get_concrete_function`. - -```python -# Create the tf.Module object. -root = tf.Module() -root.const = tf.Variable(2.) -root.pow = tf.function(lambda x : x ** root.const) - -# Get the concrete function. -input_data = tf.constant(1., shape=[1]) -concrete_func = root.pow.get_concrete_function(input_data) -``` - -## Example program - -```python -import tensorflow as tf - -# Initialize the tf.Module object. -root = tf.Module() - -# Instantiate the variable once. -root.var = None - -# Define a function so that the operations aren't computed in advance. -@tf.function -def exported_function(x): - # Each variable can only be defined once. The variable can be defined within - # the function but needs to contain a reference outside of the function. - if root.var is None: - root.var = tf.Variable(tf.random.uniform([2, 2])) - root.const = tf.constant([[37.0, -23.0], [1.0, 4.0]]) - root.mult = tf.matmul(root.const, root.var) - return root.mult * x - -# Save the function as part of the tf.Module object. -root.func = exported_function - -# Get the concrete function. -concrete_func = root.func.get_concrete_function( - tf.TensorSpec([1, 1], tf.float32)) -``` - -## Common Questions - -### How do I save a concrete function as a SavedModel? - -Users who want to save their TensorFlow model before converting it to TensorFlow -Lite should save it as a SavedModel. After getting the concrete function, call -`tf.saved_model.save` to save the model. The example above can be saved using -the following instruction. - -```python -tf.saved_model.save(root, export_dir, concrete_func) -``` - -Reference the -[SavedModel guide](https://www.tensorflow.org/guide/saved_model) -for detailed instructions on using SavedModels. - -### How do I get a concrete function from the SavedModel? - -Each concrete function within a SavedModel can be identified by a signature key. -The default signature key is `tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY`. -The example below shows how to get the concrete function from a model. - -```python -model = tf.saved_model.load(export_dir) -concrete_func = model.signatures[ - tf.saved_model.DEFAULT_SERVING_SIGNATURE_DEF_KEY] -``` - -### How do I get a concrete function for a `tf.Keras` model? - -There are two approaches that you can use: - -1. Save the model as a SavedModel. A concrete function will be generated during - the saving process, which can be accessed upon loading the model. -2. Annotate the model with `tf.function` as seen below. - -```python -model = tf.keras.Sequential([tf.keras.layers.Dense(units=1, input_shape=[1])]) -model.compile(optimizer='sgd', loss='mean_squared_error') -model.fit(x=[-1, 0, 1, 2, 3, 4], y=[-3, -1, 1, 3, 5, 7], epochs=50) - -# Get the concrete function from the Keras model. -run_model = tf.function(lambda x : model(x)) - -# Save the concrete function. -concrete_func = run_model.get_concrete_function( - tf.TensorSpec(model.inputs[0].shape, model.inputs[0].dtype)) -``` diff --git a/tensorflow/lite/g3doc/convert/index.md b/tensorflow/lite/g3doc/convert/index.md index f9870a6d1d6..c445361d6b9 100644 --- a/tensorflow/lite/g3doc/convert/index.md +++ b/tensorflow/lite/g3doc/convert/index.md @@ -5,7 +5,7 @@ TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/) file (`.tflite`). The converter supports [SavedModel directories](https://www.tensorflow.org/guide/saved_model), [`tf.keras` models](https://www.tensorflow.org/guide/keras/overview), and -[concrete functions](concrete_function.md). +[concrete functions](https://tensorflow.org/guide/concrete_function). Note: This page contains documentation on the converter API for TensorFlow 2.0. The API for TensorFlow 1.X is available diff --git a/tensorflow/lite/g3doc/convert/python_api.md b/tensorflow/lite/g3doc/convert/python_api.md index 5cc97ff414e..b8f7312c3fc 100644 --- a/tensorflow/lite/g3doc/convert/python_api.md +++ b/tensorflow/lite/g3doc/convert/python_api.md @@ -20,7 +20,7 @@ to convert a model based on the original model format: * `TFLiteConverter.from_keras_model()`: Converts [`tf.keras` models](https://www.tensorflow.org/guide/keras/overview). * `TFLiteConverter.from_concrete_functions()`: Converts - [concrete functions](concrete_function.md). + [concrete functions](https://tensorflow.org/guide/concrete_function). This document contains [example usages](#examples) of the API, a detailed list of [changes in the API between Tensorflow 1 and TensorFlow 2](#differences), and @@ -93,8 +93,8 @@ tflite_model = converter.convert() ### Converting a concrete function The following example shows how to convert a TensorFlow -[concrete function](concrete_function.md) into a TensorFlow Lite -[`FlatBuffer`](https://google.github.io/flatbuffers/). +[concrete function](https://tensorflow.org/guide/concrete_function) into a +TensorFlow Lite [`FlatBuffer`](https://google.github.io/flatbuffers/). ```python import tensorflow as tf diff --git a/tensorflow/lite/g3doc/guide/get_started.md b/tensorflow/lite/g3doc/guide/get_started.md index 50f6d03a5ba..61a4be7ae3e 100644 --- a/tensorflow/lite/g3doc/guide/get_started.md +++ b/tensorflow/lite/g3doc/guide/get_started.md @@ -127,7 +127,7 @@ When [converting TensorFlow 2.x models](../convert/python_api.md), these are: * [SavedModel directories](https://www.tensorflow.org/guide/saved_model) * [`tf.keras` models](https://www.tensorflow.org/guide/keras/overview) -* [Concrete functions](../convert/concrete_function.md) +* [Concrete functions](https://tensorflow.org/guide/concrete_function) The converter can be configured to apply various optimizations that can improve performance or reduce file size. This is covered in section 4,