Updated TensorFlow Lite examples.
PiperOrigin-RevId: 244418529
This commit is contained in:
A. Unique TensorFlower
TensorFlower Gardener
parent
c75ecf3c93
commit
4b4d920790
@ -1,146 +1,87 @@
|
||||
# Android quickstart
|
||||
|
||||
An example Android application using TensorFLow Lite is available
|
||||
[on GitHub](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/java/demo).
|
||||
The demo is a sample camera app that classifies images continuously
|
||||
using either a quantized Mobilenet model or a floating point Inception-v3 model.
|
||||
To run the demo, a device running Android 5.0 ( API 21) or higher is required.
|
||||
To get started with TensorFlow Lite on Android, we recommend exploring the
|
||||
following example.
|
||||
|
||||
In the demo app, inference is done using the TensorFlow Lite Java API. The demo
|
||||
app classifies frames in real-time, displaying the top most probable
|
||||
classifications. It also displays the time taken to detect the object.
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android">Android
|
||||
image classification example</a>
|
||||
|
||||
There are three ways to get the demo app to your device:
|
||||
For an explanation of the source code, you should also read
|
||||
[TensorFlow Lite Android image classification](https://www.tensorflow.org/lite/models/image_classification/android).
|
||||
|
||||
* Download the [prebuilt binary APK](http://download.tensorflow.org/deps/tflite/TfLiteCameraDemo.apk).
|
||||
* Use Android Studio to build the application.
|
||||
* Download the source code for TensorFlow Lite and the demo and build it using
|
||||
bazel.
|
||||
This example app uses
|
||||
[image classification](https://www.tensorflow.org/lite/models/image_classification/overview)
|
||||
to continuously classify whatever it sees from the device's rear-facing camera.
|
||||
The application can run either on device or emulator.
|
||||
|
||||
Inference is performed using the TensorFlow Lite Java API. The demo app
|
||||
classifies frames in real-time, displaying the top most probable
|
||||
classifications. It allows the user to choose between a floating point or
|
||||
[quantized](https://www.tensorflow.org/lite/performance/post_training_quantization)
|
||||
model, select the thread count, and decide whether to run on CPU, GPU, or via
|
||||
[NNAPI](https://developer.android.com/ndk/guides/neuralnetworks).
|
||||
|
||||
## Download the pre-built binary
|
||||
Note: Additional Android applications demonstrating TensorFlow Lite in a variety
|
||||
of use cases are available in
|
||||
[Examples](https://www.tensorflow.org/lite/examples).
|
||||
|
||||
The easiest way to try the demo is to download the
|
||||
[pre-built binary APK](https://storage.googleapis.com/download.tensorflow.org/deps/tflite/TfLiteCameraDemo.apk)
|
||||
## Build in Android Studio
|
||||
|
||||
Once the APK is installed, click the app icon to start the program. The first
|
||||
time the app is opened, it asks for runtime permissions to access the device
|
||||
camera. The demo app opens the back-camera of the device and recognizes objects
|
||||
in the camera's field of view. At the bottom of the image (or at the left
|
||||
of the image if the device is in landscape mode), it displays top three objects
|
||||
classified and the classification latency.
|
||||
To build the example in Android Studio, follow the instructions in
|
||||
[README.md](https://github.com/tensorflow/examples/blob/master/lite/examples/image_classification/android/README.md).
|
||||
|
||||
## Create your own Android app
|
||||
|
||||
## Build in Android Studio with TensorFlow Lite AAR from JCenter
|
||||
To get started quickly writing your own Android code, we recommend using our
|
||||
[Android image classification example](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android)
|
||||
as a starting point.
|
||||
|
||||
Use Android Studio to try out changes in the project code and compile the demo
|
||||
app:
|
||||
The following sections contain some useful information for working with
|
||||
TensorFlow Lite on Android.
|
||||
|
||||
* Install the latest version of
|
||||
[Android Studio](https://developer.android.com/studio/index.html).
|
||||
* Make sure the Android SDK version is greater than 26 and NDK version is greater
|
||||
than 14 (in the Android Studio settings).
|
||||
* Import the `tensorflow/lite/java/demo` directory as a new
|
||||
Android Studio project.
|
||||
* Install all the Gradle extensions it requests.
|
||||
### Use the TensorFlow Lite AAR from JCenter
|
||||
|
||||
Now you can build and run the demo app.
|
||||
To use TensorFlow Lite in your Android app, we recommend using the
|
||||
[TensorFlow Lite AAR hosted at JCenter](https://bintray.com/google/tensorflow/tensorflow-lite).
|
||||
|
||||
The build process downloads the quantized [Mobilenet TensorFlow Lite model](https://storage.googleapis.com/download.tensorflow.org/models/tflite/mobilenet_v1_224_android_quant_2017_11_08.zip), and unzips it into the assets directory: `tensorflow/lite/java/demo/app/src/main/assets/`.
|
||||
You can specify this in your `build.gradle` dependencies as follows:
|
||||
|
||||
Some additional details are available on the
|
||||
[TF Lite Android App page](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/java/demo/README.md).
|
||||
|
||||
### Using other models
|
||||
|
||||
To use a different model:
|
||||
* Download the floating point [Inception-v3 model](https://storage.googleapis.com/download.tensorflow.org/models/tflite/inception_v3_slim_2016_android_2017_11_10.zip).
|
||||
* Unzip and copy `inceptionv3_non_slim_2015.tflite` to the assets directory.
|
||||
* Change the chosen classifier in [Camera2BasicFragment.java](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/java/demo/app/src/main/java/com/example/android/tflitecamerademo/Camera2BasicFragment.java)<br>
|
||||
from: `classifier = new ImageClassifierQuantizedMobileNet(getActivity());`<br>
|
||||
to: `classifier = new ImageClassifierFloatInception(getActivity());`.
|
||||
|
||||
|
||||
## Build TensorFlow Lite and the demo app from source
|
||||
|
||||
### Clone the TensorFlow repo
|
||||
|
||||
```sh
|
||||
git clone https://github.com/tensorflow/tensorflow
|
||||
```build
|
||||
dependencies {
|
||||
implementation 'org.tensorflow:tensorflow-lite:0.0.0-nightly'
|
||||
}
|
||||
```
|
||||
|
||||
### Install Bazel
|
||||
This AAR includes binaries for all of the
|
||||
[Android ABIs](https://developer.android.com/ndk/guides/abis). You can reduce
|
||||
the size of your application's binary by only including the ABIs you need to
|
||||
support.
|
||||
|
||||
If `bazel` is not installed on your system, see
|
||||
[Installing Bazel](https://bazel.build/versions/master/docs/install.html).
|
||||
We recommend most developers omit the `x86`, `x86_64`, and `arm32` ABIs. This
|
||||
can be achieved with the following Gradle configuration, which specifically
|
||||
includes only `armeabi-v7a` and `arm64-v8a`, which should cover most modern
|
||||
Android devices.
|
||||
|
||||
Note: Bazel does not currently support Android builds on Windows. Windows users
|
||||
should download the
|
||||
[prebuilt binary](https://storage.googleapis.com/download.tensorflow.org/deps/tflite/TfLiteCameraDemo.apk).
|
||||
|
||||
### Install Android NDK and SDK
|
||||
|
||||
The Android NDK is required to build the native (C/C++) TensorFlow Lite code. The
|
||||
current recommended version is *14b* and can be found on the
|
||||
[NDK Archives](https://developer.android.com/ndk/downloads/older_releases.html#ndk-14b-downloads)
|
||||
page.
|
||||
|
||||
The Android SDK and build tools can be
|
||||
[downloaded separately](https://developer.android.com/tools/revisions/build-tools.html)
|
||||
or used as part of
|
||||
[Android Studio](https://developer.android.com/studio/index.html). To build the
|
||||
TensorFlow Lite Android demo, build tools require API >= 23 (but it will run on
|
||||
devices with API >= 21).
|
||||
|
||||
In the root of the TensorFlow repository, update the `WORKSPACE` file with the
|
||||
`api_level` and location of the SDK and NDK. If you installed it with
|
||||
Android Studio, the SDK path can be found in the SDK manager. The default NDK
|
||||
path is:`{SDK path}/ndk-bundle.` For example:
|
||||
|
||||
```
|
||||
android_sdk_repository (
|
||||
name = "androidsdk",
|
||||
api_level = 23,
|
||||
build_tools_version = "23.0.2",
|
||||
path = "/home/xxxx/android-sdk-linux/",
|
||||
)
|
||||
|
||||
android_ndk_repository(
|
||||
name = "androidndk",
|
||||
path = "/home/xxxx/android-ndk-r10e/",
|
||||
api_level = 19,
|
||||
)
|
||||
```build
|
||||
android {
|
||||
defaultConfig {
|
||||
ndk {
|
||||
abiFilters 'armeabi-v7a', 'arm64-v8a'
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Some additional details are available on the
|
||||
[TF Lite Android App page](https://github.com/tensorflow/tensorflow/tree/master/tensorflow/lite/java/demo/README.md).
|
||||
To learn more about `abiFilters`, see
|
||||
[`NdkOptions`](https://google.github.io/android-gradle-dsl/current/com.android.build.gradle.internal.dsl.NdkOptions.html)
|
||||
in the Android Gradle documentation.
|
||||
|
||||
### Build the source code
|
||||
### Build TensorFlow Lite locally
|
||||
|
||||
To build the demo app, run `bazel`:
|
||||
In some cases, you might wish to use a local build of TensorFlow Lite. For
|
||||
example, you may be building a custom binary that includes
|
||||
[operations selected from TensorFlow](https://www.tensorflow.org/lite/guide/ops_select).
|
||||
|
||||
```
|
||||
bazel build --cxxopt=--std=c++11 //tensorflow/lite/java/demo/app/src/main:TfLiteCameraDemo
|
||||
```
|
||||
|
||||
Caution: Because of an bazel bug, we only support building the Android demo app
|
||||
within a Python 2 environment.
|
||||
|
||||
|
||||
## About the demo
|
||||
|
||||
The demo app is resizing each camera image frame (224 width * 224 height) to
|
||||
match the quantized MobileNets model (299 * 299 for Inception-v3). The resized
|
||||
image is converted—row by row—into a
|
||||
[ByteBuffer](https://developer.android.com/reference/java/nio/ByteBuffer.html).
|
||||
Its size is 1 * 224 * 224 * 3 bytes, where 1 is the number of images in a batch.
|
||||
224 * 224 (299 * 299) is the width and height of the image. 3 bytes represents
|
||||
the 3 colors of a pixel.
|
||||
|
||||
This demo uses the TensorFlow Lite Java inference API
|
||||
for models which take a single input and provide a single output. This outputs a
|
||||
two-dimensional array, with the first dimension being the category index and the
|
||||
second dimension being the confidence of classification. Both models have 1001
|
||||
unique categories and the app sorts the probabilities of all the categories and
|
||||
displays the top three. The model file must be downloaded and bundled within the
|
||||
assets directory of the app.
|
||||
In this case, follow the
|
||||
[custom AAR build instructions](https://www.tensorflow.org/lite/guide/ops_select#android_aar)
|
||||
to create your own AAR and include it in your app.
|
||||
|
||||
@ -1,229 +1,98 @@
|
||||
# iOS quickstart
|
||||
|
||||
This tutorial provides a simple iOS mobile application to classify images using
|
||||
the iOS device camera. In this tutorial, you will download the demo application
|
||||
from the Tensorflow repository, build it on your computer, and install it on
|
||||
your iOS Device. You will also learn how to customize the application to suit
|
||||
your requirements.
|
||||
To get started with TensorFlow Lite on iOS, we recommend exploring the following
|
||||
example.
|
||||
|
||||
## Prerequisites
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios">iOS
|
||||
image classification example</a>
|
||||
|
||||
* You must have [Xcode](https://developer.apple.com/xcode/) installed and have
|
||||
a valid Apple Developer ID, and have an iOS device set up and linked to your
|
||||
developer account with all of the appropriate certificates. For these
|
||||
instructions, we assume that you have already been able to build and deploy
|
||||
an app to an iOS device with your current developer environment.
|
||||
For an explanation of the source code, you should also read
|
||||
[TensorFlow Lite iOS image classification](https://www.tensorflow.org/lite/models/image_classification/ios).
|
||||
|
||||
* The demo app requires a camera and must be executed on a real iOS device.
|
||||
You can build it and run with the iPhone Simulator but it won't have any
|
||||
camera information to classify.
|
||||
This example app uses
|
||||
[image classification](https://www.tensorflow.org/lite/models/image_classification/overview)
|
||||
to continuously classify whatever it sees from the device's rear-facing camera.
|
||||
The application must be run on an iOS device.
|
||||
|
||||
* You don't need to build the entire TensorFlow library to run the demo, but
|
||||
you will need to clone the TensorFlow repository if you haven't already:
|
||||
Inference is performed using the TensorFlow Lite C++ API. The demo app
|
||||
classifies frames in real-time, displaying the top most probable
|
||||
classifications. It allows the user to choose between a floating point or
|
||||
[quantized](https://www.tensorflow.org/lite/performance/post_training_quantization)
|
||||
model, select the thread count, and decide whether to run on CPU, GPU, or via
|
||||
[NNAPI](https://developer.android.com/ndk/guides/neuralnetworks).
|
||||
|
||||
git clone https://github.com/tensorflow/tensorflow
|
||||
cd tensorflow
|
||||
Note: Additional iOS applications demonstrating TensorFlow Lite in a variety of
|
||||
use cases are available in [Examples](https://www.tensorflow.org/lite/examples).
|
||||
|
||||
* You'll also need the Xcode command-line tools:
|
||||
## Build in Xcode
|
||||
|
||||
xcode-select --install
|
||||
To build the example in Xcode, follow the instructions in
|
||||
[README.md](https://github.com/tensorflow/examples/blob/master/lite/examples/image_classification/ios/README.md).
|
||||
|
||||
If this is a new install, you will need to run the Xcode application once to
|
||||
agree to the license before continuing.
|
||||
## Create your own Android app
|
||||
|
||||
* Install CocoaPods if you don't have it:
|
||||
To get started quickly writing your own iOS code, we recommend using our
|
||||
[iOS image classification example](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios)
|
||||
as a starting point.
|
||||
|
||||
sudo gem install cocoapods
|
||||
The following sections contain some useful information for working with
|
||||
TensorFlow Lite on iOS.
|
||||
|
||||
### Step 1. Clone the TensorFlow source code
|
||||
### Use TensorFlow Lite from Objective-C and Swift
|
||||
|
||||
First, we clone the GitHub repository on the computer in a folder to get the
|
||||
demo application.
|
||||
The example app provides an Objective-C wrapper on top of the C++ Tensorflow
|
||||
Lite library. This wrapper is required because currently there is no
|
||||
interoperability between Swift and C++. The wrapper is exposed to Swift via
|
||||
bridging so that the Tensorflow Lite methods can be called from Swift.
|
||||
|
||||
```
|
||||
git clone https://github.com/tensorflow/tensorflow
|
||||
The wrapper is located in
|
||||
[TensorflowLiteWrapper](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios/ImageClassification/TensorflowLiteWrapper).
|
||||
It is not tightly coupled with the example code, so you can use it in your own
|
||||
iOS apps. It exposes the following interface:
|
||||
|
||||
```objectivec
|
||||
@interface TfliteWrapper : NSObject
|
||||
|
||||
/**
|
||||
This method initializes the TfliteWrapper with the specified model file.
|
||||
*/
|
||||
- (instancetype)initWithModelFileName:(NSString *)fileName;
|
||||
|
||||
/**
|
||||
This method initializes the interpreter of TensorflowLite library with the specified model file
|
||||
that performs the inference.
|
||||
*/
|
||||
- (BOOL)setUpModelAndInterpreter;
|
||||
|
||||
/**
|
||||
This method gets a reference to the input tensor at an index.
|
||||
*/
|
||||
- (uint8_t *)inputTensorAtIndex:(int)index;
|
||||
|
||||
/**
|
||||
This method performs the inference by invoking the interpreter.
|
||||
*/
|
||||
- (BOOL)invokeInterpreter;
|
||||
|
||||
/**
|
||||
This method gets the output tensor at a specified index.
|
||||
*/
|
||||
- (uint8_t *)outputTensorAtIndex:(int)index;
|
||||
|
||||
/**
|
||||
This method sets the number of threads used by the interpreter to perform inference.
|
||||
*/
|
||||
- (void)setNumberOfThreads:(int)threadCount;
|
||||
|
||||
@end
|
||||
```
|
||||
|
||||
### Step 2. Download required dependencies
|
||||
To use these files in your own iOS app, copy them into your Xcode project.
|
||||
|
||||
Execute the shell script to download the model files used by the demo app (this
|
||||
is done from inside the cloned directory):
|
||||
|
||||
```
|
||||
tensorflow/lite/examples/ios/download_models.sh
|
||||
```
|
||||
|
||||
Run the following command to install TensorFlow Lite pod:
|
||||
|
||||
```
|
||||
cd tensorflow/lite/examples/ios/camera
|
||||
pod install
|
||||
```
|
||||
|
||||
If you have installed this pod before and that command doesn't work, try
|
||||
|
||||
```
|
||||
pod repo update
|
||||
```
|
||||
|
||||
### Step 3. Build the XCode project
|
||||
|
||||
Open the `tflite_camera_example.xcworkspace` project file generated in the last
|
||||
step:
|
||||
|
||||
```
|
||||
open tflite_camera_example.xcworkspace
|
||||
```
|
||||
|
||||
Under `Project navigator -> tflite_camera_example -> Targets ->
|
||||
tflite_camera_example -> General` change the bundle identifier by pre-pending
|
||||
your name:
|
||||
|
||||
|
||||
|
||||
Plug in your iOS device. Note the app must be executed with a real device with
|
||||
camera. Select the iOS device from the drop-down menu.
|
||||
|
||||
|
||||
|
||||
Click the "Run" button to build and run the app
|
||||
|
||||
|
||||
|
||||
Note that as mentioned earlier, you must already have a device set up and linked
|
||||
to your Apple Developer account in order to deploy the app on a device.
|
||||
|
||||
You'll have to grant permissions for the app to use the device's camera. Point
|
||||
the camera at various objects and enjoy seeing how the model classifies things!
|
||||
|
||||
## Understanding iOS App Code
|
||||
|
||||
### Get camera input
|
||||
|
||||
The main logic of this app is in the Objective C++ source file
|
||||
`tensorflow/lite/examples/ios/camera/CameraExampleViewController.mm`.
|
||||
|
||||
The `setupAVCapture` method constructs a `AVCaptureSession` and set itself as a
|
||||
delegate. The `captureOutput:didOutputSampleBuffer:fromConnection:` method is
|
||||
called for every captured frame. It calls `runModelOnFrame` to run the model for
|
||||
every frame.
|
||||
|
||||
### Create an interpreter
|
||||
|
||||
To create the interpreter, we need to load the model file. The following code
|
||||
will load a model and create an interpreter.
|
||||
|
||||
```
|
||||
model = tflite::FlatBufferModel::BuildFromFile([graph_path UTF8String]);
|
||||
```
|
||||
|
||||
Behind the scenes, the model is loaded as a memory-mapped file. It offers faster
|
||||
load times and reduce the dirty pages in memory.
|
||||
|
||||
Construct a `BuiltinOpResolver` to use the TensorFlow Lite buildin ops. Then,
|
||||
create the interpreter object using `InterpreterBuilder` that takes the model
|
||||
file as argument as shown below.
|
||||
|
||||
```
|
||||
tflite::ops::builtin::BuiltinOpResolver resolver;
|
||||
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
|
||||
```
|
||||
|
||||
### Obtain the input buffer
|
||||
|
||||
By default, the app uses quantized model since it's smaller and faster. The
|
||||
buffer is a raw pointer to an array of 8 bit unsigned integers (`uint8_t`). The
|
||||
following code obtains the input buffer from the interpreter:
|
||||
|
||||
```
|
||||
// Get the index of first input tensor.
|
||||
int input_tensor_index = interpreter->inputs()[0];
|
||||
// Get the pointer to the input buffer.
|
||||
uint8_t* buffer = interpreter->typed_tensor<uint8_t>(input_tensor_index);
|
||||
```
|
||||
|
||||
Throughout this document, it's assumed a quantized model is used.
|
||||
|
||||
### Pre-process of bitmap image
|
||||
|
||||
The MobileNet model we're using takes 224x224x3 inputs, where the dimensions are
|
||||
width, height, and colors (RGB). The images returned from `AVCaptureSession` is
|
||||
bigger, and has 4 color channels (RGBA).
|
||||
|
||||
Many image classification models (like MobileNet) take fixe-sized inputs. It's
|
||||
required to scale or crop the image before feeding it into the model, and change
|
||||
the channels from RGBA to RGB.
|
||||
|
||||
The code to pre-process the images is in `ProcessInputWithQuantizedModel`
|
||||
function in
|
||||
`tensorflow/lite/examples/ios/camera/CameraExampleViewController.mm`. It's a
|
||||
simple implementation for nearest neighbor color sampling, and it only copies
|
||||
the first 3 bytes for each pixel.
|
||||
|
||||
```
|
||||
void ProcessInputWithQuantizedModel(
|
||||
uint8_t* input, uint8_t* output, int image_width, int image_height, int image_channels) {
|
||||
for (int y = 0; y < wanted_input_height; ++y) {
|
||||
uint8_t* out_row = output + (y * wanted_input_width * wanted_input_channels);
|
||||
for (int x = 0; x < wanted_input_width; ++x) {
|
||||
const int in_x = (y * image_width) / wanted_input_width;
|
||||
const int in_y = (x * image_height) / wanted_input_height;
|
||||
uint8_t* in_pixel = input + (in_y * image_width * image_channels) + (in_x * image_channels);
|
||||
uint8_t* out_pixel = out_row + (x * wanted_input_channels);
|
||||
for (int c = 0; c < wanted_input_channels; ++c) {
|
||||
out_pixel[c] = in_pixel[c];
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note the code is preprocessing and preparing the model input from the camera
|
||||
data. Therefore the first parameter `input` should be the camera buffer. The
|
||||
second parameter `output` should be the buffer of model input.
|
||||
|
||||
### Run inference and obtain output buffer
|
||||
|
||||
After preprocessing and filling the data into the input buffer of the
|
||||
interpreter, it's really easy to run the interpreter:
|
||||
|
||||
```
|
||||
if (interpreter->Invoke() != kTfLiteOk) {
|
||||
NSLog("Failed to invoke!");
|
||||
}
|
||||
```
|
||||
|
||||
The result is stored in the output tensor buffer of the interpreter. The
|
||||
following code obtains the pointer to the buffer:
|
||||
|
||||
```
|
||||
// Get the index of first output tensor.
|
||||
const int output_tensor_index = interpreter->outputs()[0];
|
||||
// Get the pointer to the output buffer.
|
||||
uint8_t* buffer = interpreter->typed_tensor<uint8_t>(output_tensor_index);
|
||||
```
|
||||
|
||||
### Post-process values
|
||||
|
||||
The output buffer contains an array of `uint8_t`, and the value range is 0-255.
|
||||
We need to convert the value to float to get the probabilities with value range
|
||||
0.0-1.0. The formula of the quantization value mapping is:
|
||||
|
||||
float_value = (quantized_value - zero_point) * scale
|
||||
|
||||
The following code converts quantized values back to float values, using the
|
||||
quantizaiton parameters in tensors:
|
||||
|
||||
```
|
||||
uint8_t* quantized_output = interpreter->typed_output_tensor<uint8_t>(0);
|
||||
int32_t zero_point = input_tensor->params.zero_point;
|
||||
float scale = input_tensor->params.scale;
|
||||
float output[output_size];
|
||||
for (int i = 0; i < output_size; ++i) {
|
||||
output[i] = (quantized_output[i] - zero_point) * scale;
|
||||
}
|
||||
```
|
||||
|
||||
Finally, we find the best set of classifications by storing them in a priority
|
||||
queue based on their confidence scores. See the `GetTopN` function in
|
||||
`tensorflow/lite/examples/ios/camera/CameraExampleViewController.mm`.
|
||||
Note: When you add an Objective-C file to an existing Swift app (or vice versa),
|
||||
Xcode will prompt you to create a *bridging header* file to expose the files to
|
||||
Swift. In the example project, this file is named
|
||||
[`ImageClassification-Bridging-Header.h`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios/ImageClassification/TensorflowLiteWrapper/ImageClassification-Bridging-Header.h).
|
||||
For more information, see Apple's
|
||||
[Importing Objective-C into Swift](https://developer.apple.com/documentation/swift/imported_c_and_objective-c_apis/importing_objective-c_into_swift){: .external}
|
||||
documentation.
|
||||
|
||||
@ -1,207 +1,317 @@
|
||||
# TensorFlow Lite Android Image Classifier App Example
|
||||
# TensorFlow Lite Android image classification example
|
||||
|
||||
This tutorial provides a simple Android mobile application to classify images
|
||||
using the Android device camera. In this tutorial, you will download the demo
|
||||
application from the Tensorflow examples repository, build it on your computer,
|
||||
and install it on your Android device. You will also learn how to customize the
|
||||
application to suit your requirements.
|
||||
This document walks through the code of a simple Android mobile application that
|
||||
demonstrates [image classification](overview.md) using the device camera.
|
||||
|
||||
### Prerequisites
|
||||
The application code is located in the
|
||||
[Tensorflow examples](https://github.com/tensorflow/examples) repository, along
|
||||
with instructions for building and deploying the app.
|
||||
|
||||
* Android Studio 3.2 (installed on a Linux, Mac or Windows machine)
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android">Example
|
||||
application</a>
|
||||
|
||||
* Android device
|
||||
## Explore the code
|
||||
|
||||
* USB cable (to connect Android device to your computer)
|
||||
|
||||
### Step 1. Clone the TensorFlow source code
|
||||
|
||||
Clone the TensorFlow examples GitHub repository to your computer to get the demo
|
||||
application.
|
||||
|
||||
```
|
||||
|
||||
git clone https://github.com/tensorflow/examples
|
||||
|
||||
```
|
||||
|
||||
Open the TensorFlow source code in Android Studio. To do this, open Android
|
||||
Studio and select `Open an existing project` setting the folder to
|
||||
`examples/lite/examples/image_classification/android`
|
||||
|
||||
<img src="images/classifydemo_img1.png" />
|
||||
|
||||
This folder contains the demo application for image classification, object
|
||||
detection, and speech hotword detection.
|
||||
|
||||
### Step 2. Build the Android Studio project
|
||||
|
||||
Select `Build -> Make Project` and check that the project builds
|
||||
successfully. You will need Android SDK configured in the settings. You'll need
|
||||
at least SDK version 23. The gradle file will prompt you to download any missing
|
||||
libraries.
|
||||
|
||||
<img src="images/classifydemo_img4.png" style="width: 40%" />
|
||||
|
||||
<img src="images/classifydemo_img2.png" style="width: 60%" />
|
||||
|
||||
#### TensorFlow Lite AAR from JCenter:
|
||||
|
||||
Note that the `build.gradle` is configured to use TensorFlow Lite's nightly
|
||||
build.
|
||||
|
||||
If you see a build error related to compatibility with Tensorflow Lite's Java
|
||||
API (example: method X is undefined for type Interpreter), there has likely been
|
||||
a backwards compatible change to the API. You will need to pull new app code
|
||||
that's compatible with the nightly build by running `git pull`.
|
||||
|
||||
### Step 3. Install and run the app
|
||||
|
||||
Connect the Android device to the computer and be sure to approve any ADB
|
||||
permission prompts that appear on your phone. Select `Run -> Run app.` Select
|
||||
the deployment target in the connected devices to the device on which the app will
|
||||
be installed. This will install the app on the device.
|
||||
|
||||
<img src="images/classifydemo_img5.png" style="width: 60%" />
|
||||
|
||||
<img src="images/classifydemo_img6.png" style="width: 70%" />
|
||||
|
||||
<img src="images/classifydemo_img7.png" style="width: 40%" />
|
||||
|
||||
<img src="images/classifydemo_img8.png" style="width: 80%" />
|
||||
|
||||
To test the app, open the app called `TFL Classify` on your device. When you run
|
||||
the app the first time, the app will request permission to access the camera.
|
||||
Re-installing the app may require you to uninstall the previous installations.
|
||||
|
||||
## Understanding Android App Code
|
||||
We're now going to walk through the most important parts of the sample code.
|
||||
|
||||
### Get camera input
|
||||
|
||||
This mobile application gets the camera input using the functions defined in the
|
||||
file CameraActivity.java in the folder
|
||||
`examples/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/CameraActivity.java.`
|
||||
This file depends on `AndroidManifest.xml` in the folder
|
||||
`examples/lite/examples/image_classification/android/app/src/main` to set the
|
||||
camera orientation.
|
||||
file
|
||||
[`CameraActivity.java`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/CameraActivity.java).
|
||||
This file depends on
|
||||
[`AndroidManifest.xml`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android/app/src/main/AndroidManifest.xml)
|
||||
to set the camera orientation.
|
||||
|
||||
### Pre-process bitmap image
|
||||
`CameraActivity` also contains code to capture user preferences from the UI and
|
||||
make them available to other classes via convenience methods.
|
||||
|
||||
The mobile application code that pre-processes the images and runs inference is
|
||||
in
|
||||
`examples/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/tflite/Classifier.java.`
|
||||
Here, we take the input camera bitmap image and convert it to a Bytebuffer
|
||||
format for efficient processing. We pre-allocate the memory for ByteBuffer
|
||||
object based on the image dimensions because Bytebuffer objects can't infer the
|
||||
object shape.
|
||||
|
||||
```
|
||||
c.imgData =
|
||||
ByteBuffer.allocateDirect( DIM_BATCH_SIZE * DIM_IMG_SIZE_X * DIM_IMG_SIZE_Y *
|
||||
DIM_PIXEL_SIZE);
|
||||
c.imgData.order(ByteOrder.nativeOrder());
|
||||
```java
|
||||
model = Model.valueOf(modelSpinner.getSelectedItem().toString().toUpperCase());
|
||||
device = Device.valueOf(deviceSpinner.getSelectedItem().toString());
|
||||
numThreads = Integer.parseInt(threadsTextView.getText().toString().trim());
|
||||
```
|
||||
|
||||
While running the application, we pre-process the incoming bitmap images from the
|
||||
camera to a Bytebuffer. Since this model is quantized 8-bit, we will put a
|
||||
single byte for each channel. `imgData` will contain an encoded `Color` for each
|
||||
pixel in ARGB format, so we need to mask the least significant 8 bits to get
|
||||
blue, and next 8 bits to get green and next 8 bits to get blue, and we have an
|
||||
opaque image so alpha can be ignored.
|
||||
### Classifier
|
||||
|
||||
The file
|
||||
[`Classifier.java`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/tflite/Classifier.java)
|
||||
contains most of the complex logic for processing the camera input and running
|
||||
inference.
|
||||
|
||||
Two subclasses of the file exist, in
|
||||
[`ClassifierFloatMobileNet.java`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/tflite/ClassifierFloatMobileNet.java)
|
||||
and
|
||||
[`ClassifierQuantizedMobileNet.java`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/tflite/ClassifierQuantizedMobileNet.java),
|
||||
to demonstrate the use of both floating point and
|
||||
[quantized](https://www.tensorflow.org/lite/performance/post_training_quantization)
|
||||
models.
|
||||
|
||||
The `Classifier` class implements a static method, `create`, which is used to
|
||||
instantiate the appropriate subclass based on the supplied model type (quantized
|
||||
vs floating point).
|
||||
|
||||
#### Load model and create interpreter
|
||||
|
||||
To perform inference, we need to load a model file and instantiate an
|
||||
`Interpreter`. This happens in the constructor of the `Classifier` class, along
|
||||
with loading the list of class labels. Information about the device type and
|
||||
number of threads is used to configure the `Interpreter` via the
|
||||
`Interpreter.Options` instance passed into its constructor. Note how that in the
|
||||
case of a GPU being available, a
|
||||
[`Delegate`](https://www.tensorflow.org/lite/performance/gpu) is created using
|
||||
`GpuDelegateHelper`.
|
||||
|
||||
```java
|
||||
protected Classifier(Activity activity, Device device, int numThreads) throws IOException {
|
||||
tfliteModel = loadModelFile(activity);
|
||||
switch (device) {
|
||||
case NNAPI:
|
||||
tfliteOptions.setUseNNAPI(true);
|
||||
break;
|
||||
case GPU:
|
||||
gpuDelegate = GpuDelegateHelper.createGpuDelegate();
|
||||
tfliteOptions.addDelegate(gpuDelegate);
|
||||
break;
|
||||
case CPU:
|
||||
break;
|
||||
}
|
||||
tfliteOptions.setNumThreads(numThreads);
|
||||
tflite = new Interpreter(tfliteModel, tfliteOptions);
|
||||
labels = loadLabelList(activity);
|
||||
...
|
||||
```
|
||||
imgData.rewind();
|
||||
bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());
|
||||
// Convert the image to floating point.
|
||||
int pixel = 0;
|
||||
for (int i = 0; i < DIM_IMG_SIZE_X; ++i) {
|
||||
for (int j = 0; j < DIM_IMG_SIZE_Y; ++j) {
|
||||
final int val = intValues[pixel++];
|
||||
imgData.put((byte) ((val >> 16) & 0xFF));
|
||||
imgData.put((byte) ((val >> 8) & 0xFF));
|
||||
imgData.put((byte) (val & 0xFF));
|
||||
}
|
||||
|
||||
For Android devices, we recommend pre-loading and memory mapping the model file
|
||||
to offer faster load times and reduce the dirty pages in memory. The method
|
||||
`loadModelFile` does this, returning a `MappedByteBuffer` containing the model.
|
||||
|
||||
```java
|
||||
private MappedByteBuffer loadModelFile(Activity activity) throws IOException {
|
||||
AssetFileDescriptor fileDescriptor = activity.getAssets().openFd(getModelPath());
|
||||
FileInputStream inputStream = new FileInputStream(fileDescriptor.getFileDescriptor());
|
||||
FileChannel fileChannel = inputStream.getChannel();
|
||||
long startOffset = fileDescriptor.getStartOffset();
|
||||
long declaredLength = fileDescriptor.getDeclaredLength();
|
||||
return fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
|
||||
}
|
||||
```
|
||||
|
||||
Note: If your model file is compressed then you will have to load the model as a
|
||||
`File`, as it cannot be directly mapped and used from memory.
|
||||
|
||||
The `MappedByteBuffer` is passed into the `Interpreter` constructor, along with
|
||||
an `Interpreter.Options` object. This object can be used to configure the
|
||||
interpreter, for example by setting the number of threads (`.setNumThreads(1)`)
|
||||
or enabling [NNAPI](https://developer.android.com/ndk/guides/neuralnetworks)
|
||||
(`.setUseNNAPI(true)`).
|
||||
|
||||
#### Pre-process bitmap image
|
||||
|
||||
Next in the `Classifier` constructor, we take the input camera bitmap image and
|
||||
convert it to a `ByteBuffer` format for efficient processing. We pre-allocate
|
||||
the memory for the `ByteBuffer` object based on the image dimensions because
|
||||
Bytebuffer objects can't infer the object shape.
|
||||
|
||||
The `ByteBuffer` represents the image as a 1D array with three bytes per channel
|
||||
(red, green, and blue). We call `order(ByteOrder.nativeOrder())` to ensure bits
|
||||
are stored in the device's native order.
|
||||
|
||||
```java
|
||||
imgData =
|
||||
ByteBuffer.allocateDirect(
|
||||
DIM_BATCH_SIZE
|
||||
* getImageSizeX()
|
||||
* getImageSizeY()
|
||||
* DIM_PIXEL_SIZE
|
||||
* getNumBytesPerChannel());
|
||||
imgData.order(ByteOrder.nativeOrder());
|
||||
```
|
||||
|
||||
The code in `convertBitmapToByteBuffer` pre-processes the incoming bitmap images
|
||||
from the camera to this `ByteBuffer`. It calls the method `addPixelValue` to add
|
||||
each set of pixel values to the `ByteBuffer` sequentially.
|
||||
|
||||
```java
|
||||
imgData.rewind();
|
||||
bitmap.getPixels(intValues, 0, bitmap.getWidth(), 0, 0, bitmap.getWidth(), bitmap.getHeight());
|
||||
// Convert the image to floating point.
|
||||
int pixel = 0;
|
||||
for (int i = 0; i < getImageSizeX(); ++i) {
|
||||
for (int j = 0; j < getImageSizeY(); ++j) {
|
||||
final int val = intValues[pixel++];
|
||||
addPixelValue(val);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
In `ClassifierQuantizedMobileNet`, `addPixelValue` is overridden to put a single
|
||||
byte for each channel. The bitmap contains an encoded color for each pixel in
|
||||
ARGB format, so we need to mask the least significant 8 bits to get blue, and
|
||||
next 8 bits to get green and next 8 bits to get blue. Since we have an opaque
|
||||
image, alpha can be ignored.
|
||||
|
||||
```java
|
||||
@Override
|
||||
protected void addPixelValue(int pixelValue) {
|
||||
imgData.put((byte) ((pixelValue >> 16) & 0xFF));
|
||||
imgData.put((byte) ((pixelValue >> 8) & 0xFF));
|
||||
imgData.put((byte) (pixelValue & 0xFF));
|
||||
}
|
||||
```
|
||||
|
||||
For `ClassifierFloatMobileNet`, we must provide a floating point number for each
|
||||
channel where the value is between `0` and `1`. To do this, we mask out each
|
||||
color channel as before, but then divide each resulting value by `255.f`.
|
||||
|
||||
```java
|
||||
@Override
|
||||
protected void addPixelValue(int pixelValue) {
|
||||
imgData.putFloat(((pixelValue >> 16) & 0xFF) / 255.f);
|
||||
imgData.putFloat(((pixelValue >> 8) & 0xFF) / 255.f);
|
||||
imgData.putFloat((pixelValue & 0xFF) / 255.f);
|
||||
}
|
||||
```
|
||||
|
||||
#### Run inference
|
||||
|
||||
The method that runs inference, `runInference`, is implemented by each subclass
|
||||
of `Classifier`. In `ClassifierQuantizedMobileNet`, the method looks as follows:
|
||||
|
||||
```java
|
||||
protected void runInference() {
|
||||
tflite.run(imgData, labelProbArray);
|
||||
}
|
||||
```
|
||||
|
||||
The output of the inference is stored in a byte array `labelProbArray`, which is
|
||||
allocated in the subclass's constructor. It consists of a single outer element,
|
||||
containing one innner element for each label in the classification model.
|
||||
|
||||
To run inference, we call `run()` on the interpreter instance, passing the input
|
||||
and output buffers as arguments.
|
||||
|
||||
#### Recognize image
|
||||
|
||||
Rather than call `runInference` directly, the method `recognizeImage` is used.
|
||||
It accepts a bitmap, runs inference, and returns a sorted `List` of
|
||||
`Recognition` instances, each corresponding to a label. The method will return a
|
||||
number of results bounded by `MAX_RESULTS`, which is 3 by default.
|
||||
|
||||
`Recognition` is a simple class that contains information about a specific
|
||||
recognition result, including its `title` and `confidence`.
|
||||
|
||||
A `PriorityQueue` is used for sorting. Each `Classifier` subclass has a
|
||||
`getNormalizedProbability` method, which is expected to return a probability
|
||||
between 0 and 1 of a given class being represented by the image.
|
||||
|
||||
```java
|
||||
PriorityQueue<Recognition> pq =
|
||||
new PriorityQueue<Recognition>(
|
||||
3,
|
||||
new Comparator<Recognition>() {
|
||||
@Override
|
||||
public int compare(Recognition lhs, Recognition rhs) {
|
||||
// Intentionally reversed to put high confidence at the head of the queue.
|
||||
return Float.compare(rhs.getConfidence(), lhs.getConfidence());
|
||||
}
|
||||
});
|
||||
for (int i = 0; i < labels.size(); ++i) {
|
||||
pq.add(
|
||||
new Recognition(
|
||||
"" + i,
|
||||
labels.size() > i ? labels.get(i) : "unknown",
|
||||
getNormalizedProbability(i),
|
||||
null));
|
||||
}
|
||||
```
|
||||
|
||||
### Display results
|
||||
|
||||
The classifier is invoked and inference results are displayed by the
|
||||
`processImage()` function in
|
||||
[`ClassifierActivity.java`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android/app/src/main/java/org/tensorflow/lite/examples/classification/ClassifierActivity.java).
|
||||
|
||||
`ClassifierActivity` is a subclass of `CameraActivity` that contains method
|
||||
implementations that render the camera image, run classification, and display
|
||||
the results. The method `processImage()` runs classification on a background
|
||||
thread as fast as possible, rendering information on the UI thread to avoid
|
||||
blocking inference and creating latency.
|
||||
|
||||
```java
|
||||
protected void processImage() {
|
||||
rgbFrameBitmap.setPixels(getRgbBytes(), 0, previewWidth, 0, 0, previewWidth, previewHeight);
|
||||
final Canvas canvas = new Canvas(croppedBitmap);
|
||||
canvas.drawBitmap(rgbFrameBitmap, frameToCropTransform, null);
|
||||
|
||||
runInBackground(
|
||||
new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
if (classifier != null) {
|
||||
final long startTime = SystemClock.uptimeMillis();
|
||||
final List<Classifier.Recognition> results = classifier.recognizeImage(croppedBitmap);
|
||||
lastProcessingTimeMs = SystemClock.uptimeMillis() - startTime;
|
||||
LOGGER.v("Detect: %s", results);
|
||||
cropCopyBitmap = Bitmap.createBitmap(croppedBitmap);
|
||||
|
||||
runOnUiThread(
|
||||
new Runnable() {
|
||||
@Override
|
||||
public void run() {
|
||||
showResultsInBottomSheet(results);
|
||||
showFrameInfo(previewWidth + "x" + previewHeight);
|
||||
showCropInfo(cropCopyBitmap.getWidth() + "x" + cropCopyBitmap.getHeight());
|
||||
showCameraResolution(canvas.getWidth() + "x" + canvas.getHeight());
|
||||
showRotationInfo(String.valueOf(sensorOrientation));
|
||||
showInference(lastProcessingTimeMs + "ms");
|
||||
}
|
||||
});
|
||||
}
|
||||
readyForNextImage();
|
||||
}
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
Another important role of `ClassifierActivity` is to determine user preferences
|
||||
(by interrogating `CameraActivity`), and instantiate the appropriately
|
||||
configured `Classifier` subclass. This happens when the video feed begins (via
|
||||
`onPreviewSizeChosen()`) and when options are changed in the UI (via
|
||||
`onInferenceConfigurationChanged()`).
|
||||
|
||||
```java
|
||||
private void recreateClassifier(Model model, Device device, int numThreads) {
|
||||
if (classifier != null) {
|
||||
LOGGER.d("Closing classifier.");
|
||||
classifier.close();
|
||||
classifier = null;
|
||||
}
|
||||
if (device == Device.GPU) {
|
||||
if (!GpuDelegateHelper.isGpuDelegateAvailable()) {
|
||||
LOGGER.d("Not creating classifier: GPU support unavailable.");
|
||||
runOnUiThread(
|
||||
() -> {
|
||||
Toast.makeText(this, "GPU acceleration unavailable.", Toast.LENGTH_LONG).show();
|
||||
});
|
||||
return;
|
||||
} else if (model == Model.QUANTIZED && device == Device.GPU) {
|
||||
LOGGER.d("Not creating classifier: GPU doesn't support quantized models.");
|
||||
runOnUiThread(
|
||||
() -> {
|
||||
Toast.makeText(
|
||||
this, "GPU does not yet supported quantized models.", Toast.LENGTH_LONG)
|
||||
.show();
|
||||
});
|
||||
return;
|
||||
}
|
||||
}
|
||||
try {
|
||||
LOGGER.d(
|
||||
"Creating classifier (model=%s, device=%s, numThreads=%d)", model, device, numThreads);
|
||||
classifier = Classifier.create(this, model, device, numThreads);
|
||||
} catch (IOException e) {
|
||||
LOGGER.e(e, "Failed to create classifier.");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### Create interpreter
|
||||
|
||||
To create the interpreter, we need to load the model file. In Android devices,
|
||||
we recommend pre-loading and memory mapping the model file as shown below to
|
||||
offer faster load times and reduce the dirty pages in memory. If your model file
|
||||
is compressed, then you will have to load the model as a `File`, as it cannot be
|
||||
directly mapped and used from memory.
|
||||
|
||||
```
|
||||
// Memory-map the model file
|
||||
AssetFileDescriptor fileDescriptor = assets.openFd(modelFilename);
|
||||
FileInputStream inputStream = new
|
||||
FileInputStream(fileDescriptor.getFileDescriptor()); FileChannel fileChannel =
|
||||
inputStream.getChannel(); long startOffset = fileDescriptor.getStartOffset();
|
||||
long declaredLength = fileDescriptor.getDeclaredLength(); return
|
||||
fileChannel.map(FileChannel.MapMode.READ_ONLY, startOffset, declaredLength);
|
||||
```
|
||||
|
||||
Then, create the interpreter object using `new Interpreter()` that takes the
|
||||
model file as argument as shown below.
|
||||
|
||||
```
|
||||
// Create Interpreter
|
||||
c.tfLite = new Interpreter(loadModelFile(assetManager, modelFilename));
|
||||
```
|
||||
|
||||
### Run inference
|
||||
|
||||
The output of the inference is stored in a byte array `labelprob.` We
|
||||
pre-allocate the memory for the output buffer. Then, we run inference on the
|
||||
interpreter object using function `run()` that takes input and output buffers as
|
||||
arguments.
|
||||
|
||||
```
|
||||
// Pre-allocate output buffers.
|
||||
c.labelProb = new byte[1][c.labels.size()];
|
||||
// Run Inference
|
||||
tfLite.run(imgData, labelProb);
|
||||
```
|
||||
|
||||
### Post-process values
|
||||
|
||||
Finally, we find the best set of classifications by storing them in a priority
|
||||
queue based on their confidence scores.
|
||||
|
||||
```
|
||||
// Find the best classifications
|
||||
PriorityQueue<Recognition> pq = ...
|
||||
for (int i = 0; i < labels.size(); ++i)
|
||||
{
|
||||
pq.add( new Recognition( ' '+ i,
|
||||
labels.size() > i ? labels.get(i) : unknown,
|
||||
(float) labelProb[0][i], null));
|
||||
}
|
||||
```
|
||||
|
||||
And we display up to MAX_RESULTS number of classifications in the application,
|
||||
where Recognition is a generic class defined in `Classifier.java` that contains
|
||||
the following information of the classified object: id, title, label, and its
|
||||
location when the model is an object detection model.
|
||||
|
||||
```
|
||||
// Display the best classifications
|
||||
final ArrayList<Recognition> recognitions =
|
||||
new ArrayList<Recognition>();
|
||||
int recognitionsSize = Math.min(pq.size(), MAX_RESULTS);
|
||||
for (int i = 0; i < recognitionsSize; ++i) {
|
||||
recognitions.add(pq.poll());
|
||||
}
|
||||
```
|
||||
|
||||
### Load onto display
|
||||
|
||||
We render the results on the Android device screen using the following lines in
|
||||
`processImage()` function in `ClassifierActivity.java` which uses the UI defined
|
||||
in `RecognitionScoreView.java.`
|
||||
|
||||
```
|
||||
resultsView.setResults(results);
|
||||
requestRender();
|
||||
```
|
||||
|
||||
Binary file not shown.
|
Before Width: | Height: | Size: 137 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 86 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 91 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 178 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 135 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 25 KiB |
Binary file not shown.
|
Before Width: | Height: | Size: 17 KiB |
@ -1,229 +1,221 @@
|
||||
# TensorFlow Lite iOS Image Classifier App Example
|
||||
# TensorFlow Lite iOS image classification example
|
||||
|
||||
This tutorial provides a simple iOS mobile application to classify images using
|
||||
the iOS device camera. In this tutorial, you will download the demo application
|
||||
from the Tensorflow repository, build it on your computer, and install it on
|
||||
your iOS Device. You will also learn how to customize the application to suit
|
||||
your needs.
|
||||
This document walks through the code of a simple iOS mobile application that
|
||||
demonstrates [image classification](overview.md) using the device camera.
|
||||
|
||||
## Prerequisites
|
||||
The application code is located in the
|
||||
[Tensorflow examples](https://github.com/tensorflow/examples) repository, along
|
||||
with instructions for building and deploying the app.
|
||||
|
||||
* You must have [Xcode](https://developer.apple.com/xcode/) installed and have
|
||||
a valid Apple Developer ID, and have an iOS device set up and linked to your
|
||||
developer account with all of the appropriate certificates. For these
|
||||
instructions, we assume that you have already been able to build and deploy
|
||||
an app to an iOS device with your current developer environment.
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios">Example
|
||||
application</a>
|
||||
|
||||
* The demo app requires a camera and must be executed on a real iOS device.
|
||||
You can build it and run with the iPhone Simulator but it won't have any
|
||||
camera information to classify.
|
||||
## Explore the code
|
||||
|
||||
* You don't need to build the entire TensorFlow library to run the demo, but
|
||||
you will need to clone the TensorFlow repository if you haven't already:
|
||||
We're now going to walk through the most important parts of the sample code.
|
||||
|
||||
git clone https://github.com/tensorflow/tensorflow
|
||||
cd tensorflow
|
||||
|
||||
* You'll also need the Xcode command-line tools:
|
||||
|
||||
xcode-select --install
|
||||
|
||||
If this is a new install, you will need to run the Xcode application once to
|
||||
agree to the license before continuing.
|
||||
|
||||
* Install CocoaPods if you don't have it:
|
||||
|
||||
sudo gem install cocoapods
|
||||
|
||||
### Step 1. Clone the TensorFlow source code
|
||||
|
||||
lone the GitHub repository onto your computer to get the
|
||||
demo application.
|
||||
|
||||
```
|
||||
git clone https://github.com/tensorflow/tensorflow
|
||||
```
|
||||
|
||||
### Step 2. Download required dependencies
|
||||
|
||||
Execute the shell script to download the model files used by the demo app (this
|
||||
is done from inside the cloned directory):
|
||||
|
||||
```
|
||||
tensorflow/lite/examples/ios/download_models.sh
|
||||
```
|
||||
|
||||
Run the following command to install TensorFlow Lite pod:
|
||||
|
||||
```
|
||||
cd tensorflow/lite/examples/ios/camera
|
||||
pod install
|
||||
```
|
||||
|
||||
If you have installed this pod before and that command doesn't work, try
|
||||
|
||||
```
|
||||
pod repo update
|
||||
```
|
||||
|
||||
### Step 3. Build the XCode project
|
||||
|
||||
Open the `tflite_camera_example.xcworkspace` project file generated in the last
|
||||
step:
|
||||
|
||||
```
|
||||
open tflite_camera_example.xcworkspace
|
||||
```
|
||||
|
||||
Under `Project navigator -> tflite_camera_example -> Targets ->
|
||||
tflite_camera_example -> General` change the bundle identifier by pre-pending
|
||||
your name:
|
||||
|
||||
|
||||
|
||||
Plug in your iOS device. Note that the app must be executed with a real device with
|
||||
a camera. Select the iOS device from the drop-down menu.
|
||||
|
||||
|
||||
|
||||
Click the "Run" button to build and run the app
|
||||
|
||||
|
||||
|
||||
Note that, as mentioned earlier, you must already have a device set up and linked
|
||||
to your Apple Developer account in order to deploy the app onto a device.
|
||||
|
||||
You'll have to grant permissions for the app to use the device's camera. Point
|
||||
the camera at various objects and enjoy seeing how the model classifies things!
|
||||
|
||||
## Understanding iOS App Code
|
||||
This example is written in both Swift and Objective-C. All application
|
||||
functionality, image processing, and results formatting is developed in Swift.
|
||||
Objective-C is used via
|
||||
[bridging](https://developer.apple.com/documentation/swift/imported_c_and_objective-c_apis/importing_objective-c_into_swift)
|
||||
to make the TensorFlow Lite C++ framework calls.
|
||||
|
||||
### Get camera input
|
||||
|
||||
The main logic of this app is in the Objective C++ source file
|
||||
`tensorflow/lite/examples/ios/camera/CameraExampleViewController.mm`.
|
||||
The main logic of this app is in the Swift source file
|
||||
[`ViewController.swift`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios/ImageClassification/ViewControllers/ViewController.swift).
|
||||
|
||||
The `setupAVCapture` method constructs a `AVCaptureSession` and set itself as a
|
||||
delegate. The `captureOutput:didOutputSampleBuffer:fromConnection:` method is
|
||||
called for every captured frame. It calls `runModelOnFrame` to run the model for
|
||||
every frame.
|
||||
The app's main view is represented by the `ViewController` class, which we
|
||||
extend with functionality from `CameraFeedManagerDelegate`, a class created to
|
||||
handle a camera feed. To run inference on the feed, we implement the `didOutput`
|
||||
method, which is called whenever a frame is available from the camera.
|
||||
|
||||
### Create an interpreter
|
||||
Our implementation of `didOutput` includes a call to the `runModel` method of a
|
||||
`ModelDataHandler` instance. As we will see below, this class gives us access to
|
||||
the TensorFlow Lite interpreter and the image classification model we are using.
|
||||
|
||||
To create the interpreter, we need to load the model file. The following code
|
||||
will load a model and create an interpreter.
|
||||
```swift
|
||||
extension ViewController: CameraFeedManagerDelegate {
|
||||
|
||||
```
|
||||
model = tflite::FlatBufferModel::BuildFromFile([graph_path UTF8String]);
|
||||
func didOutput(pixelBuffer: CVPixelBuffer) {
|
||||
|
||||
// Run the live camera pixelBuffer through TensorFlow to get the result
|
||||
let currentTimeMs = Date().timeIntervalSince1970 * 1000
|
||||
|
||||
guard (currentTimeMs - previousInferenceTimeMs) >= delayBetweenInferencesMs else {
|
||||
return
|
||||
}
|
||||
|
||||
previousInferenceTimeMs = currentTimeMs
|
||||
result = modelDataHandler?.runModel(onFrame: pixelBuffer)
|
||||
|
||||
DispatchQueue.main.async {
|
||||
|
||||
let resolution = CGSize(width: CVPixelBufferGetWidth(pixelBuffer), height: CVPixelBufferGetHeight(pixelBuffer))
|
||||
|
||||
// Display results by handing off to the InferenceViewController
|
||||
self.inferenceViewController?.inferenceResult = self.result
|
||||
self.inferenceViewController?.resolution = resolution
|
||||
self.inferenceViewController?.tableView.reloadData()
|
||||
|
||||
}
|
||||
}
|
||||
...
|
||||
```
|
||||
|
||||
Behind the scenes, the model is loaded as a memory-mapped file. It offers faster
|
||||
load times and reduce the dirty pages in memory.
|
||||
### TensorFlow Lite wrapper
|
||||
|
||||
Construct a `BuiltinOpResolver` to use the TensorFliw Lite buildin ops. Then,
|
||||
create the interpreter object using `InterpreterBuilder` that takes the model
|
||||
file as argument as shown below.
|
||||
The app uses TensorFlow Lite's C++ library via an Objective-C wrapper defined in
|
||||
[`TfliteWrapper.h`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios/ImageClassification/TensorFlowLiteWrapper/TfliteWrapper.h)
|
||||
and
|
||||
[`TfliteWrapper.mm`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios/ImageClassification/TensorFlowLiteWrapper/TfliteWrapper.mm).
|
||||
|
||||
```
|
||||
tflite::ops::builtin::BuiltinOpResolver resolver;
|
||||
tflite::InterpreterBuilder(*model, resolver)(&interpreter);
|
||||
This wrapper is required because currently there is no interoperability between
|
||||
Swift and C++. The wrapper is exposed to Swift via bridging so that the
|
||||
Tensorflow Lite methods can be called from Swift.
|
||||
|
||||
### ModelDataHandler
|
||||
|
||||
The Swift class `ModelDataHandler`, defined by
|
||||
[`ModelDataHandler.swift`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios/ImageClassification/ModelDataHandler/ModelDataHandler.swift),
|
||||
handles all data preprocessing and makes calls to run inference on a given frame
|
||||
through the TfliteWrapper. It then formats the inferences obtained and returns
|
||||
the top N results for a successful inference.
|
||||
|
||||
The following sections show how this works.
|
||||
|
||||
#### Initialization
|
||||
|
||||
The method `init` instantiates a `TfliteWrapper` and loads the supplied model
|
||||
and labels files from disk.
|
||||
|
||||
```swift
|
||||
init?(modelFileName: String, labelsFileName: String, labelsFileExtension: String) {
|
||||
|
||||
// Initializes TFliteWrapper and based on the setup result of interpreter, initializes the object of this class
|
||||
self.tfLiteWrapper = TfliteWrapper(modelFileName: modelFileName)
|
||||
guard self.tfLiteWrapper.setUpModelAndInterpreter() else {
|
||||
return nil
|
||||
}
|
||||
|
||||
super.init()
|
||||
|
||||
tfLiteWrapper.setNumberOfThreads(threadCount)
|
||||
|
||||
// Opens and loads the classes listed in the labels file
|
||||
loadLabels(fromFileName: labelsFileName, fileExtension: labelsFileExtension)
|
||||
}
|
||||
```
|
||||
|
||||
### Obtain the input buffer
|
||||
#### Process input
|
||||
|
||||
By default, the app uses a quantized model since it's smaller and faster. The
|
||||
buffer is a raw pointer to an array of 8 bit unsigned integers (`uint8_t`). The
|
||||
following code obtains the input buffer from the interpreter:
|
||||
The method `runModel` accepts a `CVPixelBuffer` of camera data, which can be
|
||||
obtained from the `didOutput` method defined in `ViewController`.
|
||||
|
||||
```
|
||||
// Get the index of first input tensor.
|
||||
int input_tensor_index = interpreter->inputs()[0];
|
||||
// Get the pointer to the input buffer.
|
||||
uint8_t* buffer = interpreter->typed_tensor<uint8_t>(input_tensor_index);
|
||||
We crop the image, call `CVPixelBufferLockBaseAddress` to prepare the buffer to
|
||||
be read by the CPU, and then create an input tensor using the TensorFlow Lite
|
||||
wrapper:
|
||||
|
||||
```swift
|
||||
guard let tensorInputBaseAddress = tfLiteWrapper.inputTensor(at: 0) else {
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
Throughout this document, it's assumed that a quantized model is used.
|
||||
The image buffer contains an encoded color for each pixel in `BGRA` format
|
||||
(where `A` represents Alpha, or transparency), and our model expects it in `RGB`
|
||||
format. We now step through the buffer four bytes at a time, copying the three
|
||||
bytes we care about (`R`, `G`, and `B`) to the input tensor.
|
||||
|
||||
### Pre-process bitmap image
|
||||
Note: Since we are using a quantized model, we can directly use the `UInt8`
|
||||
values from the buffer. If we were using a float model, we would have to convert
|
||||
them to floating point by dividing by 255.
|
||||
|
||||
The MobileNet model that we're using takes 224x224x3 inputs, where the dimensions are
|
||||
width, height, and colors (RGB). The images returned from `AVCaptureSession` is
|
||||
bigger and has 4 color channels (RGBA).
|
||||
```swift
|
||||
let inputImageBaseAddress = sourceStartAddrss.assumingMemoryBound(to: UInt8.self)
|
||||
|
||||
Many image classification models (like MobileNet) take fixe-sized inputs. It's
|
||||
required to scale or crop the image before feeding it into the model and change
|
||||
the channels from RGBA to RGB.
|
||||
for y in 0...wantedInputHeight - 1 {
|
||||
let tensorInputRow = tensorInputBaseAddress.advanced(by: (y * wantedInputWidth * wantedInputChannels))
|
||||
let inputImageRow = inputImageBaseAddress.advanced(by: y * wantedInputWidth * imageChannels)
|
||||
|
||||
The code to pre-process the images is in `ProcessInputWithQuantizedModel`
|
||||
function in
|
||||
`tensorflow/lite/examples/ios/camera/CameraExampleViewController.mm`. It's a
|
||||
simple implementation for nearest neighbor color sampling and it only copies
|
||||
the first 3 bytes for each pixel.
|
||||
for x in 0...wantedInputWidth - 1 {
|
||||
|
||||
```
|
||||
void ProcessInputWithQuantizedModel(
|
||||
uint8_t* input, uint8_t* output, int image_width, int image_height, int image_channels) {
|
||||
for (int y = 0; y < wanted_input_height; ++y) {
|
||||
uint8_t* out_row = output + (y * wanted_input_width * wanted_input_channels);
|
||||
for (int x = 0; x < wanted_input_width; ++x) {
|
||||
const int in_x = (y * image_width) / wanted_input_width;
|
||||
const int in_y = (x * image_height) / wanted_input_height;
|
||||
uint8_t* in_pixel = input + (in_y * image_width * image_channels) + (in_x * image_channels);
|
||||
uint8_t* out_pixel = out_row + (x * wanted_input_channels);
|
||||
for (int c = 0; c < wanted_input_channels; ++c) {
|
||||
out_pixel[c] = in_pixel[c];
|
||||
}
|
||||
let out_pixel = tensorInputRow.advanced(by: x * wantedInputChannels)
|
||||
let in_pixel = inputImageRow.advanced(by: x * imageChannels)
|
||||
|
||||
var b = 2
|
||||
for c in 0...(wantedInputChannels) - 1 {
|
||||
|
||||
// We are reversing the order of pixels since the source pixel format is BGRA, but the model requires RGB format.
|
||||
out_pixel[c] = in_pixel[b]
|
||||
b = b - 1
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
Note that the code pre-processes and prepares the model input from the camera
|
||||
data. Therefore, the first parameter `input` should be the camera buffer. The
|
||||
second parameter `output` should be the buffer of model input.
|
||||
#### Run inference
|
||||
|
||||
### Run inference and obtain output buffer
|
||||
Running inference is a simple call to `tfLiteWrapper.invokeInterpreter()`. The
|
||||
result of this synchronous call can be obtained by calling
|
||||
`tfLiteWrapper.outputTensor()`.
|
||||
|
||||
After pre-processing and filling the data into the input buffer of the
|
||||
interpreter, it's really easy to run the interpreter:
|
||||
```swift
|
||||
guard tfLiteWrapper.invokeInterpreter() else {
|
||||
return nil
|
||||
}
|
||||
|
||||
```
|
||||
if (interpreter->Invoke() != kTfLiteOk) {
|
||||
NSLog("Failed to invoke!");
|
||||
guard let outputTensor = tfLiteWrapper.outputTensor(at: 0) else {
|
||||
return nil
|
||||
}
|
||||
```
|
||||
|
||||
The result is stored in the output tensor buffer of the interpreter. The
|
||||
following code obtains the pointer to the buffer:
|
||||
#### Process results
|
||||
|
||||
```
|
||||
// Get the index of first output tensor.
|
||||
const int output_tensor_index = interpreter->outputs()[0];
|
||||
// Get the pointer to the output buffer.
|
||||
uint8_t* buffer = interpreter->typed_tensor<uint8_t>(output_tensor_index);
|
||||
```
|
||||
The `getTopN` method, also declared in `ModelDataHandler.swift`, interprets the
|
||||
contents of the output tensor. It returns a list of the top N predictions,
|
||||
ordered by confidence.
|
||||
|
||||
### Post-process values
|
||||
The output tensor contains one `UInt8` value per class label, with a value
|
||||
between 0 and 255 corresponding to a confidence of 0 to 100% that each label is
|
||||
present in the image.
|
||||
|
||||
The output buffer contains an array of `uint8_t`, and the value range is from 0-255.
|
||||
We need to convert the value to float to get the probabilities with a value range from
|
||||
0.0-1.0. The formula of the quantization value mapping is:
|
||||
First, the results are mapped into an array of `Inference` instances, each with
|
||||
a `confidence` between 0 and 1 and a `className` representing the label.
|
||||
|
||||
float_value = (quantized_value - zero_point) * scale
|
||||
```swift
|
||||
for i in 0...predictionSize - 1 {
|
||||
let value = Double(prediction[i]) / 255.0
|
||||
|
||||
The following code converts quantized values back to float values, using the
|
||||
quantizaiton parameters in tensors:
|
||||
guard i < labels.count else {
|
||||
continue
|
||||
}
|
||||
|
||||
```
|
||||
uint8_t* quantized_output = interpreter->typed_output_tensor<uint8_t>(0);
|
||||
int32_t zero_point = input_tensor->params.zero_point;
|
||||
float scale = input_tensor->params.scale;
|
||||
float output[output_size];
|
||||
for (int i = 0; i < output_size; ++i) {
|
||||
output[i] = (quantized_output[i] - zero_point) * scale;
|
||||
let inference = Inference(confidence: value, className: labels[i])
|
||||
resultsArray.append(inference)
|
||||
}
|
||||
```
|
||||
|
||||
Finally, we find the best set of classifications by storing them in a priority
|
||||
queue based on their confidence scores. See the `GetTopN` function in
|
||||
`tensorflow/lite/examples/ios/camera/CameraExampleViewController.mm`.
|
||||
Next, the results are sorted, and we return the top `N` (where N is
|
||||
`resultCount`).
|
||||
|
||||
```swift
|
||||
resultsArray.sort { (first, second) -> Bool in
|
||||
return first.confidence > second.confidence
|
||||
}
|
||||
|
||||
guard resultsArray.count > resultCount else {
|
||||
return resultsArray
|
||||
}
|
||||
let finalArray = resultsArray[0..<resultCount]
|
||||
|
||||
return Array(finalArray)
|
||||
```
|
||||
|
||||
### Display results
|
||||
|
||||
The file
|
||||
[`InferenceViewController.swift`](https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios/ImageClassification/ViewControllers/InferenceViewController.swift)
|
||||
defines the app's UI.
|
||||
|
||||
A `UITableView` instance, `tableView`, is used to display the results.
|
||||
|
||||
@ -11,21 +11,12 @@ If you are unfamiliar with the concept of image classification, you should start
|
||||
by reading <a href="#what_is_image_classification">What is image
|
||||
classification?</a>
|
||||
|
||||
If you understand image classification, you’re new to TensorFlow Lite, and
|
||||
you’re working with Android or iOS, we recommend following the corresponding
|
||||
tutorial that will walk you through our sample code.
|
||||
|
||||
<a class="button button-primary" href="android.md">Android</a>
|
||||
<a class="button button-primary" href="ios.md">iOS</a>
|
||||
|
||||
We also provide <a href="example_applications">example applications</a> you can
|
||||
use to get started.
|
||||
To learn how to use image classification in a mobile app, we recommend exploring
|
||||
our <a href="example_applications">Example applications and guides</a>.
|
||||
|
||||
If you are using a platform other than Android or iOS, or you are already
|
||||
familiar with the
|
||||
<a href="https://www.tensorflow.org/api_docs/python/tf/lite">TensorFlow Lite
|
||||
APIs</a>, you can download our starter image classification model and the
|
||||
accompanying labels.
|
||||
familiar with the TensorFlow Lite APIs, you can download our starter image
|
||||
classification model and the accompanying labels.
|
||||
|
||||
<a class="button button-primary" href="https://storage.googleapis.com/download.tensorflow.org/models/tflite/mobilenet_v1_1.0_224_quant_and_labels.zip">Download
|
||||
starter model and labels</a>
|
||||
@ -35,16 +26,28 @@ experiment with different models to find the optimal balance between
|
||||
performance, accuracy, and model size. For guidance, see
|
||||
<a href="#choose_a_different_model">Choose a different model</a>.
|
||||
|
||||
### Example applications
|
||||
### Example applications and guides
|
||||
|
||||
We have example applications for image classification for both Android and iOS.
|
||||
For each example, we provide a guide that explains how it works.
|
||||
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android">Android
|
||||
example</a>
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios.md">iOS
|
||||
example</a>
|
||||
#### Android
|
||||
|
||||
The following screenshot shows the Android image classification example:
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/android">View
|
||||
Android example</a>
|
||||
|
||||
Read the [Android example guide](android.md) to learn how the app works.
|
||||
|
||||
#### iOS
|
||||
|
||||
<a class="button button-primary" href="https://github.com/tensorflow/examples/tree/master/lite/examples/image_classification/ios.md">View
|
||||
iOS example</a>
|
||||
|
||||
Read the [iOS example guide](ios.md) to learn how the app works.
|
||||
|
||||
#### Screenshot
|
||||
|
||||
The following screenshot shows the Android image classification example.
|
||||
|
||||
<img src="images/android_banana.png" alt="Screenshot of Android example" width="30%">
|
||||
|
||||
@ -199,8 +202,8 @@ If you want to train a model to recognize new classes, see
|
||||
For the following use cases, you should use a different type of model:
|
||||
|
||||
<ul>
|
||||
<li>Predicting the type and position of one or more objects within an image (see <a href="../object_detection/overview.md">object detection</a>)</li>
|
||||
<li>Predicting the composition of an image, for example subject versus background (see <a href="../segmentation/overview.md">segmentation</a>)</li>
|
||||
<li>Predicting the type and position of one or more objects within an image (see <a href="../object_detection/overview.md">Object detection</a>)</li>
|
||||
<li>Predicting the composition of an image, for example subject versus background (see <a href="../segmentation/overview.md">Segmentation</a>)</li>
|
||||
</ul>
|
||||
|
||||
Once you have the starter model running on your target device, you can
|
||||
@ -226,7 +229,7 @@ analyze each frame in the time before the next frame is drawn (e.g. inference
|
||||
must be faster than 33ms to perform real-time inference on a 30fps video
|
||||
stream).
|
||||
|
||||
Our quantized Mobilenet models’ performance ranges from 3.7ms to 80.3 ms.
|
||||
Our quantized MobileNet models’ performance ranges from 3.7ms to 80.3 ms.
|
||||
|
||||
### Accuracy
|
||||
|
||||
@ -240,7 +243,7 @@ appears as the label with the highest probability in the model’s output. Top-5
|
||||
refers to how often the correct label appears in the top 5 highest probabilities
|
||||
in the model’s output.
|
||||
|
||||
Our quantized Mobilenet models’ Top-5 accuracy ranges from 64.4 to 89.9%.
|
||||
Our quantized MobileNet models’ Top-5 accuracy ranges from 64.4 to 89.9%.
|
||||
|
||||
### Size
|
||||
|
||||
@ -248,13 +251,13 @@ The size of a model on-disk varies with its performance and accuracy. Size may
|
||||
be important for mobile development (where it might impact app download sizes)
|
||||
or when working with hardware (where available storage might be limited).
|
||||
|
||||
Our quantized Mobilenet models’ size ranges from 0.5 to 3.4 Mb.
|
||||
Our quantized MobileNet models’ size ranges from 0.5 to 3.4 Mb.
|
||||
|
||||
### Architecture
|
||||
|
||||
There are several different architectures of models available on
|
||||
<a href="../../guide/hosted_models.md">List of hosted models</a>, indicated by
|
||||
the model’s name. For example, you can choose between Mobilenet, Inception, and
|
||||
the model’s name. For example, you can choose between MobileNet, Inception, and
|
||||
others.
|
||||
|
||||
The architecture of a model impacts its performance, accuracy, and size. All of
|
||||
|
||||
Loading…
Reference in New Issue
Block a user