From ae3319352934dff5e6dc55de8a913e2df46c72fc Mon Sep 17 00:00:00 2001 From: Georgiy Manuilov Date: Sat, 21 Nov 2020 15:04:59 +0300 Subject: [PATCH] Updated XNNPACK delegeate readme Added some info on XNNPACK delegate to avoid confusion cause by XNNPACK engine being single-threaded by default. Further details are available in the description of the following issue: https://github.com/tensorflow/tensorflow/issues/42277 --- tensorflow/lite/delegates/xnnpack/README.md | 21 +++++++++++++++++++++ 1 file changed, 21 insertions(+) diff --git a/tensorflow/lite/delegates/xnnpack/README.md b/tensorflow/lite/delegates/xnnpack/README.md index 6f597006c1b..49a86a52bd4 100644 --- a/tensorflow/lite/delegates/xnnpack/README.md +++ b/tensorflow/lite/delegates/xnnpack/README.md @@ -63,6 +63,27 @@ bazel build -c opt --fat_apk_cpu=x86,x86_64,arm64-v8a,armeabi-v7a \ //tensorflow/lite/java:tensorflow-lite ``` +Note that in this case `Interpreter::SetNumThreads` invocation does not take +effect on number of threads used by XNNPACK engine. In order to specify number +of threads available for XNNPACK engine you should manually pass the value when +constructing the interpreter. The snippet below illustrates this assuming you are +using `InterpreterBuilder` to construct the interpreter: + +```c++ +// Load model +tflite::Model* model; +... + +// Construct the interprepter +tflite::ops::builtin::BuiltinOpResolver resolver; +std::unique_ptr interpreter; + +TfLiteStatus res = tflite::InterpreterBuilder(model, resolver, num_threads); +``` + +**XNNPACK engine used by TensorFlow Lite interpreter uses a single thread for +inference by default.** + ### Enable XNNPACK via additional dependency Another way to enable XNNPACK is to build and link the