experiments/STT-tensorflow
1
0
Fork 0
Code Issues Pull Requests Projects Releases Wiki Activity

Update iOS documentations with Objective C API.

Browse Source 
Consolidated language usages with `devsite-selector`s.

PiperOrigin-RevId: 350522544
Change-Id: I0dc37803eb86fe39eef16b70de426c3cedb483ab
This commit is contained in:
Taehee Jeong 2021-01-07 03:07:14 -08:00 committed by TensorFlower Gardener
parent 4cb1278b33
commit e30e1d1aed
4 changed files with 349 additions and 244 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

2
tensorflow/lite/g3doc/guide/ios.md
View File

@ -72,7 +72,7 @@ builds, you can write:
pod 'TensorFlowLiteSwift', '~> 0.0.1-nightly'
```
For nightly version, by default
From 2.4.0 version and latest nightly releases, by default
[GPU](https://www.tensorflow.org/lite/performance/gpu) and
[Core ML delegates](https://www.tensorflow.org/lite/performance/coreml_delegate)
are excluded from the pod to reduce the binary size. You can include them by

245
tensorflow/lite/g3doc/performance/coreml_delegate.md
View File

@ -4,7 +4,8 @@ The TensorFlow Lite Core ML delegate enables running TensorFlow Lite models on
[Core ML framework](https://developer.apple.com/documentation/coreml), which
results in faster model inference on iOS devices.
Note: This delegate is in experimental (beta) phase.
Note: This delegate is in experimental (beta) phase. It is available from
TensorFlow Lite 2.4.0 and latest nightly releases.
Note: Core ML delegate supports Core ML version 2 and later.
@ -24,88 +25,108 @@ The Core ML delegate currently supports float (FP32 and FP16) models.
## Trying the Core ML delegate on your own model
The Core ML delegate is already included in nightly release of TensorFlow lite
CocoaPods. To use Core ML delegate, change your TensorFlow lite pod
(`TensorflowLiteC` for C API, and `TensorFlowLiteSwift` for Swift) version to
`0.0.1-nightly` in your `Podfile`, and include subspec `CoreML`
CocoaPods. To use Core ML delegate, change your TensorFlow lite pod to include
subspec `CoreML` in your `Podfile`.
Note: If you want to use C API instead of Objective-C API, you can include
`TensorFlowLiteC/CoreML` pod to do so.
```
target 'YourProjectName'
# pod 'TensorFlowLiteSwift'
pod 'TensorFlowLiteSwift/CoreML', '~> 0.0.1-nightly'
pod 'TensorFlowLiteSwift/CoreML', '~> 2.4.0' # Or TensorFlowLiteObjC/CoreML
```
OR
```
# Particularily useful when you also want to include 'Metal' subspec.
target 'YourProjectName'
# pod 'TensorFlowLiteSwift'
pod 'TensorFlowLiteSwift', '~> 0.0.1-nightly', :subspecs => ['CoreML']
pod 'TensorFlowLiteSwift', '~> 2.4.0', :subspecs => ['CoreML']
```
Note: After updating `Podfile`, you should run `pod update` to reflect changes.
If you can't see the latest `CoreMLDelegate.swift` file, try running `pod cache
clean TensorFlowLiteSwift`.
Note: Core ML delegate can also use C API for Objective-C code. Prior to
TensorFlow Lite 2.4.0 release, this was the only option.
### Swift
<div>
<devsite-selector>
<section>
<h3>Swift</h3>
<p><pre class="prettyprint lang-swift">
let coreMLDelegate = CoreMLDelegate()
var interpreter: Interpreter
Initialize TensorFlow Lite interpreter with the Core ML delegate.
// Core ML delegate will only be created for devices with Neural Engine
if coreMLDelegate != nil {
interpreter = try Interpreter(modelPath: modelPath,
delegates: [coreMLDelegate!])
} else {
interpreter = try Interpreter(modelPath: modelPath)
}
</pre></p>
</section>
<section>
<h3>Objective-C</h3>
<p><pre class="prettyprint lang-objc">
```swift
let coreMLDelegate = CoreMLDelegate()
var interpreter: Interpreter
// Import module when using CocoaPods with module support
@import TFLTensorFlowLite;
// Core ML delegate will only be created for devices with Neural Engine
if coreMLDelegate != nil {
interpreter = try Interpreter(modelPath: modelPath,
delegates: [coreMLDelegate!])
} else {
interpreter = try Interpreter(modelPath: modelPath)
}
```
// Or import following headers manually
# import "tensorflow/lite/objc/apis/TFLCoreMLDelegate.h"
# import "tensorflow/lite/objc/apis/TFLTensorFlowLite.h"
### Objective-C
// Initialize Core ML delegate
TFLCoreMLDelegate* coreMLDelegate = [[TFLCoreMLDelegate alloc] init];
The Core ML delegate uses C API for Objective-C codes.
// Initialize interpreter with model path and Core ML delegate
TFLInterpreterOptions* options = [[TFLInterpreterOptions alloc] init];
NSError* error = nil;
TFLInterpreter* interpreter = [[TFLInterpreter alloc]
initWithModelPath:modelPath
options:options
delegates:@[ coreMLDelegate ]
error:&amp;error];
if (error != nil) { /* Error handling... */ }
#### Step 1. Include `coreml_delegate.h`.
if (![interpreter allocateTensorsWithError:&amp;error]) { /* Error handling... */ }
if (error != nil) { /* Error handling... */ }
```c
#include "tensorflow/lite/delegates/coreml/coreml_delegate.h"
```
// Run inference ...
</pre></p>
</section>
<section>
<h3>C (Until 2.3.0)</h3>
<p><pre class="prettyprint lang-c">
#include "tensorflow/lite/delegates/coreml/coreml_delegate.h"
#### Step 2. Create a delegate and initialize a TensorFlow Lite Interpreter
// Initialize interpreter with model
TfLiteModel* model = TfLiteModelCreateFromFile(model_path);
After initializing the interpreter options, call
`TfLiteInterpreterOptionsAddDelegate` with initialized Core ML delegate to apply
the delegate. Then initialize the interpreter with the created option.
// Initialize interpreter with Core ML delegate
TfLiteInterpreterOptions* options = TfLiteInterpreterOptionsCreate();
TfLiteDelegate* delegate = TfLiteCoreMlDelegateCreate(NULL); // default config
TfLiteInterpreterOptionsAddDelegate(options, delegate);
TfLiteInterpreterOptionsDelete(options);
```c
// Initialize interpreter with model
TfLiteModel* model = TfLiteModelCreateFromFile(model_path);
TfLiteInterpreter* interpreter = TfLiteInterpreterCreate(model, options);
// Initialize interpreter with Core ML delegate
TfLiteInterpreterOptions* options = TfLiteInterpreterOptionsCreate();
TfLiteDelegate* delegate = TfLiteCoreMlDelegateCreate(NULL); // default config
TfLiteInterpreterOptionsAddDelegate(options, delegate);
TfLiteInterpreterOptionsDelete(options);
TfLiteInterpreterAllocateTensors(interpreter);
TfLiteInterpreter* interpreter = TfLiteInterpreterCreate(model, options);
// Run inference ...
TfLiteInterpreterAllocateTensors(interpreter);
/* ... */
// Run inference ...
```
// Dispose resources when it is no longer used.
// Add following code to the section where you dispose of the delegate
// (e.g. `dealloc` of class).
#### Step 3. Dispose resources when it is no longer used.
Add this code to the section where you dispose of the delegate (e.g. `dealloc`
of class).
```c
TfLiteInterpreterDelete(interpreter);
TfLiteCoreMlDelegateDelete(delegate);
TfLiteModelDelete(model);
```
TfLiteInterpreterDelete(interpreter);
TfLiteCoreMlDelegateDelete(delegate);
TfLiteModelDelete(model);
</pre></p>
</section>
</devsite-selector>
</div>
## Best practices
@ -117,24 +138,40 @@ run Core ML delegate on other environments (for example, simulator), pass `.all`
as an option while creating delegate in Swift. On C++ (and Objective-C), you can
pass `TfLiteCoreMlDelegateAllDevices`. Following example shows how to do this:
#### Swift
<div>
<devsite-selector>
<section>
<h3>Swift</h3>
<p><pre class="prettyprint lang-swift">
var options = CoreMLDelegate.Options()
options.enabledDevices = .all
let coreMLDelegate = CoreMLDelegate(options: options)!
let interpreter = try Interpreter(modelPath: modelPath,
delegates: [coreMLDelegate])
</pre></p>
</section>
<section>
<h3>Objective-C</h3>
<p><pre class="prettyprint lang-objc">
TFLCoreMLDelegateOptions* coreMLOptions = [[TFLCoreMLDelegateOptions alloc] init];
coreMLOptions.enabledDevices = TFLCoreMLDelegateEnabledDevicesAll;
TFLCoreMLDelegate* coreMLDelegate = [[TFLCoreMLDelegate alloc]
initWithOptions:coreMLOptions];
```swift
var options = CoreMLDelegate.Options()
options.enabledDevices = .all
let coreMLDelegate = CoreMLDelegate(options: options)!
let interpreter = try Interpreter(modelPath: modelPath,
delegates: [coreMLDelegate])
```
#### Objective-C
```c
TfLiteCoreMlDelegateOptions options;
options.enabled_devices = TfLiteCoreMlDelegateAllDevices;
TfLiteDelegate* delegate = TfLiteCoreMlDelegateCreate(&options);
// Initialize interpreter with delegate
```
// Initialize interpreter with delegate
</pre></p>
</section>
<section>
<h3>C</h3>
<p><pre class="prettyprint lang-c">
TfLiteCoreMlDelegateOptions options;
options.enabled_devices = TfLiteCoreMlDelegateAllDevices;
TfLiteDelegate* delegate = TfLiteCoreMlDelegateCreate(&amp;options);
// Initialize interpreter with delegate
</pre></p>
</section>
</devsite-selector>
</div>
### Using Metal(GPU) delegate as a fallback.
@ -142,29 +179,45 @@ When the Core ML delegate is not created, alternatively you can still use
[Metal delegate](https://www.tensorflow.org/lite/performance/gpu#ios) to get
performance benefits. Following example shows how to do this:
#### Swift
<div>
<devsite-selector>
<section>
<h3>Swift</h3>
<p><pre class="prettyprint lang-swift">
var delegate = CoreMLDelegate()
if delegate == nil {
delegate = MetalDelegate() // Add Metal delegate options if necessary.
}
```swift
var delegate = CoreMLDelegate()
if delegate == nil {
delegate = MetalDelegate() // Add Metal delegate options if necessary.
}
let interpreter = try Interpreter(modelPath: modelPath,
delegates: [delegate!])
```
#### Objective-C
```c
TfLiteCoreMlDelegateOptions options = {};
delegate = TfLiteCoreMlDelegateCreate(&options);
if (delegate == NULL) {
// Add Metal delegate options if necessary
delegate = TFLGpuDelegateCreate(NULL);
}
// Initialize interpreter with delegate
```
let interpreter = try Interpreter(modelPath: modelPath,
delegates: [delegate!])
</pre></p>
</section>
<section>
<h3>Objective-C</h3>
<p><pre class="prettyprint lang-objc">
TFLDelegate* delegate = [[TFLCoreMLDelegate alloc] init];
if (!delegate) {
// Add Metal delegate options if necessary
delegate = [[TFLMetalDelegate alloc] init];
}
// Initialize interpreter with delegate
</pre></p>
</section>
<section>
<h3>C</h3>
<p><pre class="prettyprint lang-c">
TfLiteCoreMlDelegateOptions options = {};
delegate = TfLiteCoreMlDelegateCreate(&amp;options);
if (delegate == NULL) {
// Add Metal delegate options if necessary
delegate = TFLGpuDelegateCreate(NULL);
}
// Initialize interpreter with delegate
</pre></p>
</section>
</devsite-selector>
</div>
The delegate creation logic reads device's machine id (e.g. iPhone11,1) to
determine its Neural Engine availability. See the
@ -249,4 +302,4 @@ issue with all the necessary details to reproduce.
* [Core ML delegate Swift API](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/swift/Sources/CoreMLDelegate.swift)
* [Core ML delegate C API](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/coreml/coreml_delegate.h)
* This can be used for Objective-C codes.
* This can be used for Objective-C codes. ~~~

171
tensorflow/lite/g3doc/performance/gpu.md
View File

@ -77,32 +77,9 @@ your phone.
#### Step 2. Modify the Podfile to use the TensorFlow Lite GPU CocoaPod
<section class="zippy">
Until TensorFlow Lite 2.0.0
We have built a binary CocoaPod that includes the GPU delegate. To switch the
project to use it, modify the
`tensorflow/tensorflow/lite/examples/ios/camera/Podfile` file to use the
`TensorFlowLiteGpuExperimental` pod instead of `TensorFlowLite`.
```
target 'YourProjectName'
# pod 'TensorFlowLite', '1.12.0'
pod 'TensorFlowLiteGpuExperimental'
```
</section>
From TensorFlow Lite 2.1.0, GPU delegate is included in the `TensorFlowLiteC`
pod. You can choose between `TensorFlowLiteC` and `TensorFlowLiteSwift`
depending on the language.
Note: This behavior will be changed in 2.3.0 and latest nightly releases
For nightly version and upcoming 2.3.0 release, by default GPU delegate is
excluded from the pod to reduce the binary size. You can include them by
specifying subspec. For `TensorFlowLiteSwift` pod:
From 2.3.0 release, by default GPU delegate is excluded from the pod to reduce
the binary size. You can include them by specifying subspec. For
`TensorFlowLiteSwift` pod:
```ruby
pod 'TensorFlowLiteSwift/Metal', '~> 0.0.1-nightly',
@ -114,7 +91,32 @@ OR
pod 'TensorFlowLiteSwift', '~> 0.0.1-nightly', :subspecs => ['Metal']
```
You can do similarly for `TensorFlowLiteC` if you want to use the C API.
You can do similarly for `TensorFlowLiteObjC` or `TensorFlowLitC` if you want to
use the Objective-C (from 2.4.0 release) or C API.
<div>
<devsite-expandable>
<h4 class="showalways">Before 2.3.0 release</h4>
<h4>Until TensorFlow Lite 2.0.0</h4>
<p>
We have built a binary CocoaPod that includes the GPU delegate. To switch
the project to use it, modify the
`tensorflow/tensorflow/lite/examples/ios/camera/Podfile` file to use the
`TensorFlowLiteGpuExperimental` pod instead of `TensorFlowLite`.
</p>
<pre class="prettyprint lang-ruby notranslate" translate="no"><code>
target 'YourProjectName'
# pod 'TensorFlowLite', '1.12.0'
pod 'TensorFlowLiteGpuExperimental'
</code></pre>
<h4>Until TensorFlow Lite 2.2.0</h4>
<p>
From TensorFlow Lite 2.1.0 to 2.2.0, GPU delegate is included in the
`TensorFlowLiteC` pod. You can choose between `TensorFlowLiteC` and
`TensorFlowLiteSwift` depending on the language.
</p>
</devsite-expandable>
</div>
#### Step 3. Enable the GPU delegate
@ -232,63 +234,92 @@ the`addDelegate` function to register the GPU delegate to the interpreter:
### iOS
#### Swift
Note: GPU delegate can also use C API for Objective-C code. Prior to TensorFlow
Lite 2.4.0 release, this was the only option.
Initialize TensorFlow Lite interpreter with the GPU delegate.
<div>
<devsite-selector>
<section>
<h3>Swift</h3>
<p><pre class="prettyprint lang-swift">
import TensorFlowLite
```swift
import TensorFlowLite
// Load model ...
// Load model ...
// Initialize TensorFlow Lite interpreter with the GPU delegate.
let delegate = MetalDelegate()
if let interpreter = try Interpreter(modelPath: modelPath,
delegates: [delegate]) {
// Run inference ...
}
</pre></p>
</section>
<section>
<h3>Objective-C</h3>
<p><pre class="prettyprint lang-objc">
// Import module when using CocoaPods with module support
@import TFLTensorFlowLite;
let delegate = MetalDelegate()
// Or import following headers manually
#import "tensorflow/lite/objc/apis/TFLMetalDelegate.h"
#import "tensorflow/lite/objc/apis/TFLTensorFlowLite.h"
if let interpreter = try Interpreter(modelPath: modelPath,
delegates: [delegate]) {
// Run inference ...
}
// Initialize GPU delegate
TFLMetalDelegate* metalDelegate = [[TFLMetalDelegate alloc] init];
```
// Initialize interpreter with model path and GPU delegate
TFLInterpreterOptions* options = [[TFLInterpreterOptions alloc] init];
NSError* error = nil;
TFLInterpreter* interpreter = [[TFLInterpreter alloc]
initWithModelPath:modelPath
options:options
delegates:@[ metalDelegate ]
error:&amp;error];
if (error != nil) { /* Error handling... */ }
#### Objective-C
if (![interpreter allocateTensorsWithError:&amp;error]) { /* Error handling... */ }
if (error != nil) { /* Error handling... */ }
Note: For Objective-C, GPU delegate is provided via C API.
// Run inference ...
```
</pre></p>
</section>
<section>
<h3>C (Until 2.3.0)</h3>
<p><pre class="prettyprint lang-c">
#include "tensorflow/lite/c/c_api.h"
#include "tensorflow/lite/delegates/gpu/metal_delegate.h"
In your application code, include the GPU delegate header and call the
`Interpreter::ModifyGraphWithDelegate` function to register the GPU delegate to
the interpreter:
// Initialize model
TfLiteModel* model = TfLiteModelCreateFromFile(model_path);
```objc
#include "tensorflow/lite/c/c_api.h"
#include "tensorflow/lite/delegates/gpu/metal_delegate.h"
// Initialize interpreter with GPU delegate
TfLiteInterpreterOptions* options = TfLiteInterpreterOptionsCreate();
TfLiteDelegate* delegate = TFLGPUDelegateCreate(nil); // default config
TfLiteInterpreterOptionsAddDelegate(options, metal_delegate);
TfLiteInterpreter* interpreter = TfLiteInterpreterCreate(model, options);
TfLiteInterpreterOptionsDelete(options);
// Initialize model
TfLiteModel* model = TfLiteModelCreateFromFile(model_path);
TfLiteInterpreterAllocateTensors(interpreter);
// Initialize interpreter with GPU delegate
TfLiteInterpreterOptions* options = TfLiteInterpreterOptionsCreate();
TfLiteDelegate* delegate = TFLGPUDelegateCreate(nil); // default config
TfLiteInterpreterOptionsAddDelegate(options, metal_delegate);
TfLiteInterpreter* interpreter = TfLiteInterpreterCreate(model, options);
TfLiteInterpreterOptionsDelete(options);
NSMutableData *input_data = [NSMutableData dataWithLength:input_size * sizeof(float)];
NSMutableData *output_data = [NSMutableData dataWithLength:output_size * sizeof(float)];
TfLiteTensor* input = TfLiteInterpreterGetInputTensor(interpreter, 0);
const TfLiteTensor* output = TfLiteInterpreterGetOutputTensor(interpreter, 0);
TfLiteInterpreterAllocateTensors(interpreter);
// Run inference
TfLiteTensorCopyFromBuffer(input, inputData.bytes, inputData.length);
TfLiteInterpreterInvoke(interpreter);
TfLiteTensorCopyToBuffer(output, outputData.mutableBytes, outputData.length);
NSMutableData *input_data = [NSMutableData dataWithLength:input_size * sizeof(float)];
NSMutableData *output_data = [NSMutableData dataWithLength:output_size * sizeof(float)];
TfLiteTensor* input = TfLiteInterpreterGetInputTensor(interpreter, 0);
const TfLiteTensor* output = TfLiteInterpreterGetOutputTensor(interpreter, 0);
// Run inference
TfLiteTensorCopyFromBuffer(input, inputData.bytes, inputData.length);
TfLiteInterpreterInvoke(interpreter);
TfLiteTensorCopyToBuffer(output, outputData.mutableBytes, outputData.length);
// Clean up
TfLiteInterpreterDelete(interpreter);
TFLGpuDelegateDelete(metal_delegate);
TfLiteModelDelete(model);
```
// Clean up
TfLiteInterpreterDelete(interpreter);
TFLGpuDelegateDelete(metal_delegate);
TfLiteModelDelete(model);
</pre></p>
</section>
</devsite-selector>
</div>
## Supported Models and Ops

175
tensorflow/lite/g3doc/performance/gpu_advanced.md
View File

@ -187,25 +187,21 @@ bazel build -c opt --config android_arm64 tensorflow/lite/delegates/gpu:delegate
bazel build -c opt --config android_arm64 tensorflow/lite/delegates/gpu:libtensorflowlite_gpu_delegate.so # for dynamic library
```
### iOS (Swift)
Note: When calling `Interpreter::ModifyGraphWithDelegate()` or
`Interpreter::Invoke()`, the caller must have an `EGLContext` in the current
thread and `Interpreter::Invoke()` must be called from the same `EGLContext`. If
an `EGLContext` does not exist, the delegate will internally create one, but
then the developer must ensure that `Interpreter::Invoke()` is always called
from the same thread in which `Interpreter::ModifyGraphWithDelegate()` was
called.
Initialize TensorFlow Lite interpreter with the GPU delegate.
### iOS (C++)
```swift
import TensorFlowLite
Note: For Swift/Objective-C/C use cases, please refer to
[GPU delegate guide](gpu#ios)
let delegate = MetalDelegate()
if let interpreter = try Interpreter(modelPath: modelPath,
delegates: [delegate]) {
// Run inference ...
}
```
### iOS (Objective-C)
Note: For Objective-C, GPU delegate is provided via C API.
Note: This is only available when you are using bazel or build TensorFlow Lite
by yourself. C++ API can't be used with CocoaPods.
To use TensorFlow Lite on GPU, get the GPU delegate via `TFLGpuDelegateCreate()`
and then pass it to `Interpreter::ModifyGraphWithDelegate()` (instead of calling
@ -233,62 +229,72 @@ ReadFromOutputTensor(interpreter->typed_output_tensor<float>(0));
TFLGpuDelegateDelete(delegate);
```
Note: When calling `Interpreter::ModifyGraphWithDelegate()` or
`Interpreter::Invoke()`, the caller must have an `EGLContext` in the current
thread and `Interpreter::Invoke()` must be called from the same `EGLContext`. If
an `EGLContext` does not exist, the delegate will internally create one, but
then the developer must ensure that `Interpreter::Invoke()` is always called
from the same thread in which `Interpreter::ModifyGraphWithDelegate()` was
called.
## Advanced usage
### Delegate Options for iOS
`TFLGpuDelegateCreate()` accepts a `struct` of options.
([C API](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/gpu/metal_delegate.h),
[Swift API](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/swift/Sources/MetalDelegate.swift))
Constructor for GPU delegate accepts a `struct` of options.
([Swift API](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/swift/Sources/MetalDelegate.swift),
[Objective-C API](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/objc/apis/TFLMetalDelegate.h),
[C API](https://github.com/tensorflow/tensorflow/blob/master/tensorflow/lite/delegates/gpu/metal_delegate.h))
Passing `nullptr`(C API) or nothing (Swift API) to the initializer sets the
default options (which are explicated in the Basic Usage example above).
Passing `nullptr` (C API) or nothing (Objective-C and Swift API) to the
initializer sets the default options (which are explicated in the Basic Usage
example above).
**Swift API**
<div>
<devsite-selector>
<section>
<h3>Swift</h3>
<p><pre class="prettyprint lang-swift">
// THIS:
var options = MetalDelegate.Options()
options.isPrecisionLossAllowed = false
options.waitType = .passive
options.isQuantizationEnabled = true
let delegate = MetalDelegate(options: options)
```swift
// IS THE SAME AS THIS:
let delegate = MetalDelegate()
</pre></p>
</section>
<section>
<h3>Objective-C</h3>
<p><pre class="prettyprint lang-objc">
// THIS:
TFLMetalDelegateOptions* options = [[TFLMetalDelegateOptions alloc] init];
options.precisionLossAllowed = false;
options.waitType = TFLMetalDelegateThreadWaitTypePassive;
options.quantizationEnabled = true;
// THIS:
var options = MetalDelegate.Options()
options.isPrecisionLossAllowed = false
options.waitType = .passive
options.isQuantizationEnabled = false
let delegate = MetalDelegate(options: options)
TFLMetalDelegate* delegate = [[TFLMetalDelegate alloc] initWithOptions:options];
// IS THE SAME AS THIS:
let delegate = MetalDelegate()
// IS THE SAME AS THIS:
TFLMetalDelegate* delegate = [[TFLMetalDelegate alloc] init];
</pre></p>
</section>
<section>
<h3>C</h3>
<p><pre class="prettyprint lang-c">
// THIS:
const TFLGpuDelegateOptions options = {
.allow_precision_loss = false,
.wait_type = TFLGpuDelegateWaitType::TFLGpuDelegateWaitTypePassive,
.enable_quantization = true,
};
```
TfLiteDelegate* delegate = TFLGpuDelegateCreate(options);
**C API (also used for Objective-C)**
// IS THE SAME AS THIS:
TfLiteDelegate* delegate = TFLGpuDelegateCreate(nullptr);
</pre></p>
</section>
</devsite-selector>
</div>
```c++
// THIS:
const TFLGpuDelegateOptions options = {
.allow_precision_loss = false,
.wait_type = TFLGpuDelegateWaitType::TFLGpuDelegateWaitTypePassive,
.enable_quantization = false,
};
auto* delegate = TFLGpuDelegateCreate(options);
// IS THE SAME AS THIS:
auto* delegate = TFLGpuDelegateCreate(nullptr);
```
While it is convenient to use `nullptr`, we recommend that you explicitly set
the options, to avoid any unexpected behavior if default values are changed in
the future.
While it is convenient to use `nullptr` or default constructors, we recommend
that you explicitly set the options, to avoid any unexpected behavior if default
values are changed in the future.
### Running quantized models on GPU
@ -347,26 +353,41 @@ Interpreter.Options options = (new Interpreter.Options()).addDelegate(delegate);
#### iOS
iOD APIs support quantized models by default. To disable, do the following:
iOS APIs support quantized models by default. To disable, do the following:
**Swift API**
<div>
<devsite-selector>
<section>
<h3>Swift</h3>
<p><pre class="prettyprint lang-swift">
var options = MetalDelegate.Options()
options.isQuantizationEnabled = false
let delegate = MetalDelegate(options: options)
</pre></p>
</section>
<section>
<h3>Objective-C</h3>
<p><pre class="prettyprint lang-objc">
TFLMetalDelegateOptions* options = [[TFLMetalDelegateOptions alloc] init];
options.quantizationEnabled = false;
</pre></p>
</section>
<section>
<h3>C</h3>
<p><pre class="prettyprint lang-c">
TFLGpuDelegateOptions options = TFLGpuDelegateOptionsDefault();
options.enable_quantization = false;
```swift
var options = MetalDelegate.Options()
options.isQuantizationEnabled = false
let delegate = MetalDelegate(options: options)
```
TfLiteDelegate* delegate = TFLGpuDelegateCreate(options);
</pre></p>
</section>
</devsite-selector>
</div>
**C API (also used for Objective-C)**
### Input/Output Buffers (iOS, C++ API only)
```c
TFLGpuDelegateOptions options = TFLGpuDelegateOptionsDefault();
options.enable_quantization = false;
auto* delegate = TFLGpuDelegateCreate(options);
```
### Input/Output Buffers (iOS only)
Note: This is only available when you are using bazel or build TensorFlow Lite
by yourself. C++ API can't be used with CocoaPods.
To do computation on the GPU, data must be made available to the GPU. This often
requires performing a memory copy. It is desirable not to cross the CPU/GPU
@ -378,7 +399,7 @@ If the network's input is an image already loaded in the GPU memory (for
example, a GPU texture containing the camera feed) it can stay in the GPU memory
without ever entering the CPU memory. Similarly, if the network's output is in
the form of a renderable image (for example,
[image style transfer](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf)_)
[image style transfer](https://www.cv-foundation.org/openaccess/content_cvpr_2016/papers/Gatys_Image_Style_Transfer_CVPR_2016_paper.pdf))
it can be directly displayed on the screen.
To achieve best performance, TensorFlow Lite makes it possible for users to