Develop Reference

Mediapipe installation on Raspberry pi 4b+

During the installation I followed the https://github.com/superuser789/MediaPipe-on-RaspberryPi guide on my Raspberry Pi 4b+ 4gb and running Raspberry Pi OS 32bit. All my packages are up to date.
I ran python3 setup.py gen_protos && python3 setup.py bdist_wheel and got this result:
ERROR: Traceback (most recent call last):
File "/home/diegos-pi/.cache/bazel/_bazel_diegos-pi/7f3a9a61ae329ba025208f264fd9a815/external/rules_cc/cc/private/rules_impl/compiler_flag.bzl", line 17, column 76, in <toplevel>
load("#bazel_tools//tools/cpp:toolchain_utils.bzl", "find_cpp_toolchain", "use_cpp_toolchain")
Error: file '#bazel_tools//tools/cpp:toolchain_utils.bzl' does not contain symbol 'use_cpp_toolchain' (did you mean 'find_cpp_toolchain'?)
ERROR: While resolving toolchains for target //mediapipe/modules/face_detection:face_detection_short_range_cpu: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '#bazel_tools//tools/cpp': in /home/diegos-pi/.cache/bazel/_bazel_diegos-pi/7f3a9a61ae329ba025208f264fd9a815/external/rules_cc/cc/defs.bzl: Extension file 'cc/private/rules_impl/compiler_flag.bzl' has errors
ERROR: Analysis of target '//mediapipe/modules/face_detection:face_detection_short_range_cpu' failed; build aborted: com.google.devtools.build.lib.packages.BuildFileContainsErrorsException: error loading package '#bazel_tools//tools/cpp': in /home/diegos-pi/.cache/bazel/_bazel_diegos-pi/7f3a9a61ae329ba025208f264fd9a815/external/rules_cc/cc/defs.bzl: Extension file 'cc/private/rules_impl/compiler_flag.bzl' has errors
INFO: Elapsed time: 16.113s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (2 packages loaded, 0 targets conf\
igured)

Bazel project + Tensorflow dependency trouble

I've stuck while trying to connect Tensorflow package with Bazel build system.
I have such configurations
BUILD
cc_binary(
name = "basic_neural",
srcs = ["src/main.cc"],
deps = [
"#org_tensorflow//tensorflow:libtensorflow_cc"
],
)
WORKSPACE
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "org_tensorflow",
sha256 = "59912218ca58daf114f72a3aa45e2c8c47781a7bfa4f8811b685f4ac67d1d244",
url = "https://github.com/tensorflow/serving/archive/r2.7.tar.gz",
)
Error log
ERROR: /path/to/project/basic_neural/BUILD:1:10: no such package '#org_tensorflow//tensorflow': BUILD file not found in directory 'tensorflow' of external repository #org_tensorflow. Add a BUILD file to a directory to mark it as a package. and referenced by '//:basic_neural'
ERROR: Analysis of target '//:basic_neural' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.095s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (11 packages loaded, 18 targets configured)
FAILED: Build did NOT complete successfully (11 packages loaded, 18 targets configured)

detect.tflite failure converting TF model to TFlite

I am using my mac, running Catalina 10.15.5, with Python 2.7.17, running Tensorflow 1.15.3 from source, installed using pip, not in Anaconda, and my bazel version is 0.26.1.
I am trying to convert my custom model on ssd_mobilenet_v3_small_coco to TFlite, following this tutorial: On step 3 but cannot convert my tflite_graph.pb to detect.tflite using this line:
bazel run --config=opt tensorflow/lite/toco:toco -- --input_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/tflite_graph.pb --output_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/detect.tflite --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 --inference_type=FLOAT --allow_custom_ops
I keep running into this error (I only included the last bit because it's quite lengthy):
WARNING: /Users/jp3spinelli/tensorflow/tensorflow/core/BUILD:2455:12: in srcs attribute of cc_library rule //tensorflow/core:lib_internal_impl: please do not import '//tensorflow/core/lib/strings:proto_text_util.cc' directly. You should either move the file to this package or depend on an appropriate rule there
WARNING: /Users/jp3spinelli/tensorflow/tensorflow/core/BUILD:2455:12: in srcs attribute of cc_library rule //tensorflow/core:lib_internal_impl: please do not import '//tensorflow/core/lib/strings:scanner.cc' directly. You should either move the file to this package or depend on an appropriate rule there
WARNING: /Users/jp3spinelli/tensorflow/tensorflow/core/BUILD:2455:12: in srcs attribute of cc_library rule //tensorflow/core:lib_internal_impl: please do not import '//tensorflow/core/lib/strings:strcat.cc' directly. You should either move the file to this package or depend on an appropriate rule there
INFO: Analyzed target //tensorflow/lite/toco:toco (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //tensorflow/lite/toco:toco up-to-date:
bazel-bin/tensorflow/lite/toco/toco
INFO: Elapsed time: 0.472s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/tensorflow/lite/toco/toco '--input_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/tflite_graph.pb' '--output_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/detect.tflite' '--input_shapes=1,300,300,3' '--input_arrays=normalized_input_image_tensor' '--output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_DetectioINFO: Build completed successfully, 1 total action
2020-07-21 18:03:47.675656: I tensorflow/lite/toco/import_tensorflow.cc:659] Converting unsupported operation: TFLite_Detection_PostProcess
2020-07-21 18:03:47.691874: F tensorflow/lite/toco/tooling_util.cc:1669] Check failed: input_array_dims[i] == input_array_proto.shape().dims(i) (320 vs. 300)
Abort trap: 6
It is creating a file, but it has zero bytes so I know something is wrong.
I am working out of my tensorflow directory, which is in my home directory. One weird thing I noticed is that the path to my "models" folder on my Desktop says it starts in "iCloud Drive" not "Users."
Please let me know how to fix this, I am new with coding so I need some step-by-step instructions. Thanks!

I think that you should set output_arrays between '' as a String, try to run something like this:
bazel run --config=opt tensorflow/lite/toco:toco -- \
--input_file=$OUTPUT_DIR/tflite_graph.pb \
--output_file=$OUTPUT_DIR/detect.tflite \
--input_shapes=1,300,300,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
--inference_type=FLOAT \
--allow_custom_ops

Bazel build and test after Tensorflow contribution

I have added functionality to tensorflow/tensorflow/python/ops/image_ops_impl.py and corresponding unit tests in tensorflow/tensorflow/python/ops/image_ops_test.py
I originally forked tensorflow from the master branch, made these changes on my local machine, rebased and commit.
Then I created and activated a virtualenv.
When running bazel test //tensorflow/python..., as recommended in the contribution guide I am recieving:
ERROR: /Users/isaacsultan/Code/tensorflow/third_party/python_runtime/BUILD:5:1: no such package '#local_config_python//': Traceback (most recent call last):
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 308
_create_local_python_repository(repository_ctx)
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 270, in _create_local_python_repository
_check_python_lib(repository_ctx, python_lib)
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 213, in _check_python_lib
_fail(("Invalid python library path: %...))
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 28, in _fail
fail(("%sPython Configuration Error:%...)))
Python Configuration Error: Invalid python library path: /usr/local/lib/python2.7/dist-packages
and referenced by '//third_party/python_runtime:headers'
ERROR: Analysis of target '//tensorflow/python:control_flow_util' failed; build aborted: Analysis failed
INFO: Elapsed time: 4.603s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (8 packages loaded)
FAILED: Build did NOT complete successfully (8 packages loaded)
currently loading: tensorflow/core ... (2 packages)
What could the source of my issue be please?
Since I am only changing the python functionality, there is no need to rebuild.
EDIT: After re-running ./configure:
(tensorflow) Isaacs-MacBook:tensorflow isaacsultan$ bazel clean --expunge
INFO: Starting clean.
(tensorflow) Isaacs-MacBook:tensorflow isaacsultan$ bazel test //tensorflow/python/...
Starting local Bazel server and connecting to it...
........................
ERROR: /private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/external/local_config_cc/BUILD:50:5: in apple_cc_toolchain rule #local_config_cc//:cc-compiler-darwin_x86_64: Xcode version must be specified to use an Apple CROSSTOOL. If your Xcode version has changed recently, try: "bazel clean --expunge" to re-run Xcode configuration
ERROR: Analysis of target '//tensorflow/python/eager:core' failed; build aborted: Analysis of target '#local_config_cc//:cc-compiler-darwin_x86_64' failed; build aborted
INFO: Elapsed time: 15.184s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (93 packages loaded)
FAILED: Build did NOT complete successfully (93 packages loaded)
currently loading: tensorflow/core ... (2 packages)
EDIT 2:
After running bazel clean --expunge then ./configure:
Isaacs-MacBook:Tensorflow isaacsultan$ bazel test //tensorflow/python/...
Starting local Bazel server and connecting to it...
...................
ERROR: /private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/external/local_config_cc/BUILD:50:5: in apple_cc_toolchain rule #local_config_cc//:cc-compiler-darwin_x86_64: Xcode version must be specified to use an Apple CROSSTOOL. If your Xcode version has changed recently, try: "bazel clean --expunge" to re-run Xcode configuration
ERROR: Analysis of target '//tensorflow/python:pywrap_tensorflow_import_lib_file' failed; build aborted: Analysis of target '#local_config_cc//:cc-compiler-darwin_x86_64' failed; build aborted
INFO: Elapsed time: 14.969s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (88 packages loaded)
FAILED: Build did NOT complete successfully (88 packages loaded)
currently loading: tensorflow/core ... (5 packages)
EDIT 3:
After following these steps:
Xcode version must be specified to use an Apple CROSSTOOL
4 warnings generated.
ERROR: /Users/isaacsultan/Code/tensorflow/tensorflow/BUILD:576:1: Executing genrule //tensorflow:tensorflow_python_api_gen failed (Exit 1)
/anaconda/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/tools/api/generator/create_python_api.py", line 27, in <module>
from tensorflow.python.tools.api.generator import doc_srcs
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 81, in <module>
from tensorflow.python import keras
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/__init__.py", line 25, in <module>
from tensorflow.python.keras import applications
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/applications/__init__.py", line 21, in <module>
import keras_applications
ImportError: No module named keras_applications
INFO: Elapsed time: 57510.356s, Critical Path: 492.10s
INFO: 6867 processes: 6867 local.
FAILED: Build did NOT complete successfully

Is it possible you forgot to run ./configure before building?

H5py installation via setup.py, undefined symbol: iso_c_binding_

I'm installing the h5py according to the tutorial at http://docs.h5py.org/en/latest/build.html
The installation is sucessfull. However, the test failed,
python setup.py test
I got this:
running test
running build_py
running build_ext
Summary of the h5py configuration
Path to HDF5: '/opt/cray/hdf5-parallel/1.8.13/cray/83/'
HDF5 Version: '1.8.13'
MPI Enabled: True
Rebuild Required: False
Executing cythonize()
Traceback (most recent call last):
File "setup.py", line 140, in <module>
cmdclass = CMDCLASS,
File "/python/2.7.9/lib/python2.7/distutils/core.py", line 151, in setup
dist.run_commands()
File "/python/2.7.9/lib/python2.7/distutils/dist.py", line 953, in run_commands
self.run_command(cmd)
File "/python/2.7.9/lib/python2.7/distutils/dist.py", line 972, in run_command
cmd_obj.run()
File "setup.py", line 68, in run
import h5py
File "/h5py-2.5.0/build/lib.linux-x86_64-2.7/h5py/__init__.py", line 13, in <module>
from . import _errors
**ImportError:** /opt/cray/lib64/libmpichf90_cray.so.3: undefined symbol: iso_c_binding_
looks like cython cant not find the shared library, how can I attach that? Thanks.

(Edited for parallel build)
I got this to work on a Cray XC30 (ARCHER: http://www.archer.ac.uk) using the following:
module swap PrgEnv-cray PrgEnv-gnu
module load cray-hdf5-parallel
export CRAYPE_LINK_TYPE=dynamic
export CC=cc
ARCHER has specific modules for the Python enviroment on the compute nodes that link to performant versions of numpy etc. (see: http://www.archer.ac.uk/documentation/user-guide/python.php) so I also loaded these (this may not apply on your Cray system, in ARCHER's case mpi4py is already inlcuded in the python-compute install):
module load python-compute
module load pc-numpy
Finally, I added the custom install location I will use for h5py to PYTHONPATH
export PYTHONPATH=/path/to/h5py/install/lib/python2.7/site-packages:$PYTHONPATH
Now I can build:
python setup.py configure --mpi
python setup.py install --prefix=/path/to/h5py/install
...lots of output...
Now, running the tests on the frontend node fail but with the error message I would expect to see on a Cray XC if you try to launch MPI code on a login/service node (failed to initialize communication channel, the login/service nodes are not connected to the high performance network so cannot run MPI code). This suggests to me that the test would probably work if it was running on the compute nodes.
> python setup.py test
running test
running build_py
running build_ext
Autodetected HDF5 1.8.13
********************************************************************************
Summary of the h5py configuration
Path to HDF5: '/opt/cray/hdf5-parallel/1.8.13/GNU/49'
HDF5 Version: '1.8.13'
MPI Enabled: True
Rebuild Required: False
********************************************************************************
Executing cythonize()
[Thu Oct 22 19:53:01 2015] [unknown] Fatal error in PMPI_Init_thread: Other MPI error, error stack:
MPIR_Init_thread(547):
MPID_Init(203).......: channel initialization failed
MPID_Init(579).......: PMI2 init failed: 1
Aborted
To test properly you would have to submit a job that launched a parallel Python script on the compute nodes using aprun. I do not think the built in test framework will work easily as it probably expects the MPI launcher to be called mpiexec (as on a standard cluster) so you may need to write your own tests. The other option would be to coerce setup.py to use aprun instead somehow.