Skip to main content

Browse the archive

Revision 35ef1861ee824f3c59163ce3800d0318b632032f authored by Steven Johnson on 27 January 2021, 01:11:35 UTC, committed by Steven Johnson on 27 January 2021, 01:11:35 UTC 
Allow Tensor to use an external memory allocation
 
For the TFLite Delegate, Tensors which are inputs or outputs don't need their own allocation (TFLite will do that for us); we should just point at the shared memory, which will save memory allocation, plus the time to copy between TFLite and our memory.

Note that the 'read-only' inputs work basically the same as before, but for normal inputs and outputs, we must update the pointers in the Eval() method of the delegate, as they aren't guaranteed to be valid prior to then. (This is still substantially cheaper than copying the memory entirely.)

I've left the code in the delegate marked with USE_EXTERNAL_TENSORS for now, to make the change a bit clearer. It's not intended to be a long-term flag.

From local benchmarking on my Mac laptop, I don't see a meaningful performance difference for mobilenet_v2. (I haven't measured memory usage but it definitely will be lower.)
1 parent 607aaa3
Raw File
run-clang-format.sh 
#!/bin/bash

set -e

ROOT_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" >/dev/null 2>&1 && pwd )"

# We are currently standardized on using LLVM/Clang10 for this script.
# Note that this is totally independent of the version of LLVM that you
# are using to build Halide itself. If you don't have LLVM10 installed,
# you can usually install what you need easily via:
#
# sudo apt-get install llvm-10 clang-10 libclang-10-dev clang-tidy-10
# export CLANG_FORMAT_LLVM_INSTALL_DIR=/usr/lib/llvm-10

[ -z "$CLANG_FORMAT_LLVM_INSTALL_DIR" ] && echo "CLANG_FORMAT_LLVM_INSTALL_DIR must point to an LLVM installation dir for this script." && exit
echo CLANG_FORMAT_LLVM_INSTALL_DIR = ${CLANG_FORMAT_LLVM_INSTALL_DIR}

VERSION=$(${CLANG_FORMAT_LLVM_INSTALL_DIR}/bin/clang-format --version)
if [[ ${VERSION} =~ .*version\ 10.* ]]
then
    echo "clang-format version 10 found."
else
    echo "CLANG_FORMAT_LLVM_INSTALL_DIR must point to an LLVM 10 install!"
    exit 1
fi

# Note that we specifically exclude files starting with . in order
# to avoid finding emacs backup files
find "${ROOT_DIR}/apps" \
     "${ROOT_DIR}/src" \
     "${ROOT_DIR}/tools" \
     "${ROOT_DIR}/test" \
     "${ROOT_DIR}/util" \
     "${ROOT_DIR}/python_bindings" \
     \( -name "*.cpp" -o -name "*.h" -o -name "*.c" \) -and -not -wholename "*/.*" | \
     xargs ${CLANG_FORMAT_LLVM_INSTALL_DIR}/bin/clang-format -i -style=file
back to top