Revision 35ef1861ee824f3c59163ce3800d0318b632032f authored by Steven Johnson on 27 January 2021, 01:11:35 UTC, committed by Steven Johnson on 27 January 2021, 01:11:35 UTC
For the TFLite Delegate, Tensors which are inputs or outputs don't need their own allocation (TFLite will do that for us); we should just point at the shared memory, which will save memory allocation, plus the time to copy between TFLite and our memory. Note that the 'read-only' inputs work basically the same as before, but for normal inputs and outputs, we must update the pointers in the Eval() method of the delegate, as they aren't guaranteed to be valid prior to then. (This is still substantially cheaper than copying the memory entirely.) I've left the code in the delegate marked with USE_EXTERNAL_TENSORS for now, to make the change a bit clearer. It's not intended to be a long-term flag. From local benchmarking on my Mac laptop, I don't see a meaningful performance difference for mobilenet_v2. (I haven't measured memory usage but it definitely will be lower.)
1 parent 607aaa3
| File | Mode | Size |
|---|---|---|
| .github | ||
| apps | ||
| cmake | ||
| dependencies | ||
| doc | ||
| packaging | ||
| python_bindings | ||
| src | ||
| test | ||
| tools | ||
| tutorial | ||
| util | ||
| .clang-format | -rw-r--r-- | 1.4 KB |
| .clang-format-ignore | -rw-r--r-- | 265 bytes |
| .clang-tidy | -rw-r--r-- | 1.8 KB |
| .gitattributes | -rw-r--r-- | 342 bytes |
| .gitignore | -rw-r--r-- | 1.1 KB |
| .gitmodules | -rw-r--r-- | 0 bytes |
| CMakeLists.txt | -rw-r--r-- | 4.2 KB |
| CMakePresets.json | -rw-r--r-- | 2.4 KB |
| CODE_OF_CONDUCT.md | -rw-r--r-- | 3.5 KB |
| LICENSE.txt | -rw-r--r-- | 3.2 KB |
| Makefile | -rw-r--r-- | 97.4 KB |
| README.md | -rw-r--r-- | 14.8 KB |
| README_cmake.md | -rw-r--r-- | 69.0 KB |
| README_rungen.md | -rw-r--r-- | 12.1 KB |
| README_webassembly.md | -rw-r--r-- | 7.5 KB |
| run-clang-format.sh | -rwxr-xr-x | 1.4 KB |
| run-clang-tidy.sh | -rwxr-xr-x | 3.0 KB |
Computing file changes ...
