I've stuck while trying to connect Tensorflow package with Bazel build system.
I have such configurations
BUILD
cc_binary(
name = "basic_neural",
srcs = ["src/main.cc"],
deps = [
"#org_tensorflow//tensorflow:libtensorflow_cc"
],
)
WORKSPACE
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(
name = "org_tensorflow",
sha256 = "59912218ca58daf114f72a3aa45e2c8c47781a7bfa4f8811b685f4ac67d1d244",
url = "https://github.com/tensorflow/serving/archive/r2.7.tar.gz",
)
Error log
ERROR: /path/to/project/basic_neural/BUILD:1:10: no such package '#org_tensorflow//tensorflow': BUILD file not found in directory 'tensorflow' of external repository #org_tensorflow. Add a BUILD file to a directory to mark it as a package. and referenced by '//:basic_neural'
ERROR: Analysis of target '//:basic_neural' failed; build aborted: Analysis failed
INFO: Elapsed time: 0.095s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (11 packages loaded, 18 targets configured)
FAILED: Build did NOT complete successfully (11 packages loaded, 18 targets configured)
Related
I am using my mac, running Catalina 10.15.5, with Python 2.7.17, running Tensorflow 1.15.3 from source, installed using pip, not in Anaconda, and my bazel version is 0.26.1.
I am trying to convert my custom model on ssd_mobilenet_v3_small_coco to TFlite, following this tutorial: On step 3 but cannot convert my tflite_graph.pb to detect.tflite using this line:
bazel run --config=opt tensorflow/lite/toco:toco -- --input_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/tflite_graph.pb --output_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/detect.tflite --input_shapes=1,300,300,3 --input_arrays=normalized_input_image_tensor --output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_Detection_PostProcess:3 --inference_type=FLOAT --allow_custom_ops
I keep running into this error (I only included the last bit because it's quite lengthy):
WARNING: /Users/jp3spinelli/tensorflow/tensorflow/core/BUILD:2455:12: in srcs attribute of cc_library rule //tensorflow/core:lib_internal_impl: please do not import '//tensorflow/core/lib/strings:proto_text_util.cc' directly. You should either move the file to this package or depend on an appropriate rule there
WARNING: /Users/jp3spinelli/tensorflow/tensorflow/core/BUILD:2455:12: in srcs attribute of cc_library rule //tensorflow/core:lib_internal_impl: please do not import '//tensorflow/core/lib/strings:scanner.cc' directly. You should either move the file to this package or depend on an appropriate rule there
WARNING: /Users/jp3spinelli/tensorflow/tensorflow/core/BUILD:2455:12: in srcs attribute of cc_library rule //tensorflow/core:lib_internal_impl: please do not import '//tensorflow/core/lib/strings:strcat.cc' directly. You should either move the file to this package or depend on an appropriate rule there
INFO: Analyzed target //tensorflow/lite/toco:toco (0 packages loaded, 0 targets configured).
INFO: Found 1 target...
Target //tensorflow/lite/toco:toco up-to-date:
bazel-bin/tensorflow/lite/toco/toco
INFO: Elapsed time: 0.472s, Critical Path: 0.00s
INFO: 0 processes.
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/tensorflow/lite/toco/toco '--input_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/tflite_graph.pb' '--output_file=/Users/jp3spinelli/Desktop/models/research/object_detection/TFLite_model/detect.tflite' '--input_shapes=1,300,300,3' '--input_arrays=normalized_input_image_tensor' '--output_arrays=TFLite_Detection_PostProcess,TFLite_Detection_PostProcess:1,TFLite_Detection_PostProcess:2,TFLite_DetectioINFO: Build completed successfully, 1 total action
2020-07-21 18:03:47.675656: I tensorflow/lite/toco/import_tensorflow.cc:659] Converting unsupported operation: TFLite_Detection_PostProcess
2020-07-21 18:03:47.691874: F tensorflow/lite/toco/tooling_util.cc:1669] Check failed: input_array_dims[i] == input_array_proto.shape().dims(i) (320 vs. 300)
Abort trap: 6
It is creating a file, but it has zero bytes so I know something is wrong.
I am working out of my tensorflow directory, which is in my home directory. One weird thing I noticed is that the path to my "models" folder on my Desktop says it starts in "iCloud Drive" not "Users."
Please let me know how to fix this, I am new with coding so I need some step-by-step instructions. Thanks!
I think that you should set output_arrays between '' as a String, try to run something like this:
bazel run --config=opt tensorflow/lite/toco:toco -- \
--input_file=$OUTPUT_DIR/tflite_graph.pb \
--output_file=$OUTPUT_DIR/detect.tflite \
--input_shapes=1,300,300,3 \
--input_arrays=normalized_input_image_tensor \
--output_arrays='TFLite_Detection_PostProcess','TFLite_Detection_PostProcess:1','TFLite_Detection_PostProcess:2','TFLite_Detection_PostProcess:3' \
--inference_type=FLOAT \
--allow_custom_ops
I am installing Tensorflow. I encountered the following problem:
I was installing Tensorflow from Source and I am following the Tensorflow website (https://www.tensorflow.org/install/source)
I had earlier installed Bazel 3.0.0 (the latest version) but got a similar error. Upon looking at other answers I figured out this error has something to do with the version of Bazel. Hence I uninstalled Bazel 3.0.0 and installed Bazel 0.27.1. But upon typing the following command(bazel build //tensorflow/tools/pip_package:build_pip_package) in the terminal, I am getting the following error:
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/home/aviraj/tensorflow/tools/bazel.rc
Starting local Bazel server and connecting to it...
ERROR: /home/aviraj/tensorflow/WORKSPACE:3:1: name 'http_archive' is not defined
ERROR: Error evaluating WORKSPACE file
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '#io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '#io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package
INFO: Elapsed time: 2.023s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
Any help would be much appreciated. Thanks
Starting with version 0.20.0, http_archive native rule has been deprecated, it has to be loaded from: #bazel_tools//tools/build_defs/repo:http.bzl. I.e. you need to add the following (unless you have a customized version) in WORKSPACE or bzl files using it:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
I have added functionality to tensorflow/tensorflow/python/ops/image_ops_impl.py and corresponding unit tests in tensorflow/tensorflow/python/ops/image_ops_test.py
I originally forked tensorflow from the master branch, made these changes on my local machine, rebased and commit.
Then I created and activated a virtualenv.
When running bazel test //tensorflow/python..., as recommended in the contribution guide I am recieving:
ERROR: /Users/isaacsultan/Code/tensorflow/third_party/python_runtime/BUILD:5:1: no such package '#local_config_python//': Traceback (most recent call last):
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 308
_create_local_python_repository(repository_ctx)
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 270, in _create_local_python_repository
_check_python_lib(repository_ctx, python_lib)
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 213, in _check_python_lib
_fail(("Invalid python library path: %...))
File "/Users/isaacsultan/Code/tensorflow/third_party/py/python_configure.bzl", line 28, in _fail
fail(("%sPython Configuration Error:%...)))
Python Configuration Error: Invalid python library path: /usr/local/lib/python2.7/dist-packages
and referenced by '//third_party/python_runtime:headers'
ERROR: Analysis of target '//tensorflow/python:control_flow_util' failed; build aborted: Analysis failed
INFO: Elapsed time: 4.603s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (8 packages loaded)
FAILED: Build did NOT complete successfully (8 packages loaded)
currently loading: tensorflow/core ... (2 packages)
What could the source of my issue be please?
Since I am only changing the python functionality, there is no need to rebuild.
EDIT: After re-running ./configure:
(tensorflow) Isaacs-MacBook:tensorflow isaacsultan$ bazel clean --expunge
INFO: Starting clean.
(tensorflow) Isaacs-MacBook:tensorflow isaacsultan$ bazel test //tensorflow/python/...
Starting local Bazel server and connecting to it...
........................
ERROR: /private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/external/local_config_cc/BUILD:50:5: in apple_cc_toolchain rule #local_config_cc//:cc-compiler-darwin_x86_64: Xcode version must be specified to use an Apple CROSSTOOL. If your Xcode version has changed recently, try: "bazel clean --expunge" to re-run Xcode configuration
ERROR: Analysis of target '//tensorflow/python/eager:core' failed; build aborted: Analysis of target '#local_config_cc//:cc-compiler-darwin_x86_64' failed; build aborted
INFO: Elapsed time: 15.184s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (93 packages loaded)
FAILED: Build did NOT complete successfully (93 packages loaded)
currently loading: tensorflow/core ... (2 packages)
EDIT 2:
After running bazel clean --expunge then ./configure:
Isaacs-MacBook:Tensorflow isaacsultan$ bazel test //tensorflow/python/...
Starting local Bazel server and connecting to it...
...................
ERROR: /private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/external/local_config_cc/BUILD:50:5: in apple_cc_toolchain rule #local_config_cc//:cc-compiler-darwin_x86_64: Xcode version must be specified to use an Apple CROSSTOOL. If your Xcode version has changed recently, try: "bazel clean --expunge" to re-run Xcode configuration
ERROR: Analysis of target '//tensorflow/python:pywrap_tensorflow_import_lib_file' failed; build aborted: Analysis of target '#local_config_cc//:cc-compiler-darwin_x86_64' failed; build aborted
INFO: Elapsed time: 14.969s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (88 packages loaded)
FAILED: Build did NOT complete successfully (88 packages loaded)
currently loading: tensorflow/core ... (5 packages)
EDIT 3:
After following these steps:
Xcode version must be specified to use an Apple CROSSTOOL
4 warnings generated.
ERROR: /Users/isaacsultan/Code/tensorflow/tensorflow/BUILD:576:1: Executing genrule //tensorflow:tensorflow_python_api_gen failed (Exit 1)
/anaconda/lib/python2.7/site-packages/h5py/__init__.py:36: FutureWarning: Conversion of the second argument of issubdtype from `float` to `np.floating` is deprecated. In future, it will be treated as `np.float64 == np.dtype(float).type`.
from ._conv import register_converters as _register_converters
Traceback (most recent call last):
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/tools/api/generator/create_python_api.py", line 27, in <module>
from tensorflow.python.tools.api.generator import doc_srcs
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/__init__.py", line 81, in <module>
from tensorflow.python import keras
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/__init__.py", line 25, in <module>
from tensorflow.python.keras import applications
File "/private/var/tmp/_bazel_isaacsultan/0e2667ab20883652d759a6a805575b2d/execroot/org_tensorflow/bazel-out/host/bin/tensorflow/create_tensorflow.python_api.runfiles/org_tensorflow/tensorflow/python/keras/applications/__init__.py", line 21, in <module>
import keras_applications
ImportError: No module named keras_applications
INFO: Elapsed time: 57510.356s, Critical Path: 492.10s
INFO: 6867 processes: 6867 local.
FAILED: Build did NOT complete successfully
Is it possible you forgot to run ./configure before building?
Currently attempting to run Google's Pixel Level Domain Adaptation Model. We were able to run MNIST to MNIST-M training and testing. Added the USPS tfrecords in the datasets folder and we recreated the MNIST_M.py file into a USPS.py file. Imported the filed like they import it for MNIST. Somehow when we run training, the architecture isn't able to find the USPS file.
Google's Original Model: Google's Model
Command being run:
bazel run domain_daptation/pixel_domain_adaptation:pixeldatrain -- --dataset_dir $DSN_DATA_DIR --source_dataset mnist --target_dataset usps
Location of usps.py file:
/home/.../models/domain_adaptation/datasets:usps
Original Stack Trace:
INFO: Analysed target //domain_adaptation/pixel_domain_adaptation:pixelda_train (2 packages loaded).
INFO: Found 1 target...
Target //domain_adaptation/pixel_domain_adaptation:pixelda_train up-to-date:
bazel-bin/domain_adaptation/pixel_domain_adaptation/pixelda_train
INFO: Elapsed time: 0.575s, Critical Path: 0.12s
INFO: Build completed successfully, 1 total action
INFO: Running command line: bazel-bin/domain_adaptation/pixel_domain_adaptation/pixelda_train --dataset_dir /home/gabe/models/domain_adaptation/datasets --source_dataset mnist --target_dataset usps
Traceback (most recent call last):
File "/home/.../.cache/bazel/_bazel_root/b15fa6ac5bc49187f8190f3d0425c8db/execroot/__main__/bazel-out/k8-fastbuild/bin/domain_adaptation/pixel_domain_adaptation/pixelda_train.runfiles/__main__/domain_adaptation/pixel_domain_adaptation/pixelda_train.py", line 24, in <module>
from domain_adaptation.datasets import dataset_factory
File "/home/.../.cache/bazel/_bazel_root/b15fa6ac5bc49187f8190f3d0425c8db/execroot/__main__/bazel-out/k8-fastbuild/bin/domain_adaptation/pixel_domain_adaptation/pixelda_train.runfiles/__main__/domain_adaptation/datasets/dataset_factory.py", line 26, in <module>
from domain_adaptation.datasets import usps
ImportError: cannot import name usps
ERROR: Non-zero return code '1' from command: Process exited with status 1
After following one of the users advice of adding location to the domain_adaptation/datasets/build. I ran the same command again.
py_library(
name = "dataset_factory",
srcs = ["dataset_factory.py"],
deps = [
":usps",
"//slim:mnist",
],
)
New Stack Trace:
ERROR: /home/.../models/domain_adaptation/datasets/BUILD:23:12: in deps attribute of py_library rule //domain_adaptation/datasets:dataset_factory: rule '//domain_adaptation/datasets:usps' does not exist
ERROR: Analysis of target '//domain_adaptation/pixel_domain_adaptation:pixelda_train' failed; build aborted: Analysis of target '//domain_adaptation/datasets:dataset_factory' failed; build aborted
INFO: Elapsed time: 0.219s
FAILED: Build did NOT complete successfully (0 packages loaded)
ERROR: Build failed. Not running target
Currently trying to run Google's research, I come into the error below when following the instructions provided. Instruction on Github - Mint Linux system with Anaconda/Tensorflow download. This is the second step in "Getting the Datasets"
~/test/models/research $ bazel run domain_adaptation/datasets:download_and_convert_mnist_m -- --dataset_dir $DSN_DATA_DIR
ERROR: /home/.../models/research/domain_adaptation/datasets/BUILD:29:1: no such package 'slim': BUILD file not found on package path and referenced by '//research/domain_adaptation/datasets:download_and_convert_mnist_m'
ERROR: Analysis of target '//research/domain_adaptation/datasets:download_and_convert_mnist_m' failed; build aborted: no such package 'slim': BUILD file not found on package path
INFO: Elapsed time: 0.168s
The research models, including slim and domain_adaptation, have been moved to the //research package in this commit from the top level.
However, this change has not been reflected in the BUILD files for https://github.com/tensorflow/models/tree/master/research/domain_adaptation. You will need to prepend research into the BUILD targets such as this, this, and this (i.e. from //slim:mnist to //research/slim:mnist), therefore referencing the correct BUILD file for slim.