66531e40a3
Also, the x86 test behavior has changed:
* removed the unnecessary testing script for x86 with Make and bazel.
* This change means that we no longer need a special skylark rule for tflite_micro_cc_test and can instead directly use cc_test.
* All the logs from the test are now visible on the terminal (previously we would only see logs on errors, which can be annoying
for debugging)
Tested that the following commands:
```
make -f tensorflow/lite/micro/tools/make/Makefile TARGET=xtensa OPTIMIZED_KERNEL_DIR=xtensa TARGET_ARCH=fusion_f1 XTENSA_CORE=F1_190305_swupgrade run_keyword_benchmark -j8
make -f tensorflow/lite/micro/tools/make/Makefile run_keyword_benchmark -j8
```
execute without any error.
The output is someting like:
```
InitializeKeywordRunner() took 39 ticks (0 ms)
KeywordRunNIerations(1) took 42 ticks (0 ms)
KeywordRunNIerations(10) took 187 ticks (0 ms)
```
Prior to this change, benchmarks would have to be run with
`make test_keyword_benchmark` which would give a confusing output:
```
InitializeKeywordRunner() took 85 ticks (0 ms)
KeywordRunNIerations(1) took 27 ticks (0 ms)
KeywordRunNIerations(10) took 276 ticks (0 ms)
make: *** [tensorflow/lite/micro/benchmarks/Makefile.inc:28: test_keyword_benchmark] Error 1
```
Fixes http://b/168123200
Another relevant bug (for the asan failures): http://b/179930607
2.3 KiB
TFLite for Microcontrollers Benchmarks
These benchmarks are for measuring the performance of key models and workloads. They are meant to be used as part of the model optimization process for a given platform.
Table of contents
- Keyword Benchmark
- Person Detection Benchmark
- Run on x86
- Run on Xtensa XPG Simulator
- Run on Sparkfun Edge
Keyword benchmark
The keyword benchmark contains a model for keyword detection with scrambled weights and biases. This model is meant to test performance on a platform only. Since the weights are scrambled, the output is meaningless. In order to validate the accuracy of optimized kernels, please run the kernel tests.
Person detection benchmark
The keyword benchmark provides a way to evaluate the performance of the 250KB visual wakewords model.
Run on x86
To run the keyword benchmark on x86, run
make -f tensorflow/lite/micro/tools/make/Makefile run_keyword_benchmark
To run the person detection benchmark on x86, run
make -f tensorflow/lite/micro/tools/make/Makefile run_person_detection_benchmark
Run on Xtensa XPG Simulator
To run the keyword benchmark on the Xtensa XPG simulator, you will need a valid Xtensa toolchain and license. With these set up, run:
make -f tensorflow/lite/micro/tools/make/Makefile TARGET=xtensa OPTIMIZED_KERNEL_DIR=xtensa TARGET_ARCH=<target architecture> XTENSA_CORE=<xtensa core> run_keyword_benchmark -j18
Run on Sparkfun Edge
The following instructions will help you build and deploy this benchmark on the SparkFun Edge development board.
If you're new to using this board, we recommend walking through the AI on a microcontroller with TensorFlow Lite and SparkFun Edge codelab to get an understanding of the workflow.
Build binary using
make -f tensorflow/lite/micro/tools/make/Makefile TARGET=sparkfun_edge person_detection_benchmark_bin
Refer to flashing instructions in the Person Detection Example.