Develop Reference

Python3 with Bazel 0.5.4 - hard-coded to python executable?

I'm maintaining a project that is using Bazel 0.5.4, and it needs to run on an environment where python does not exist. I'd like to point the application to using python3.
This issue is similar to Bazel 0.26.1 use Python3 on py_test, but the version of rules_python mentioned in that ticket is so new that it's not compatible with bazel 0.5.4
GOAL: Issue python commands with /usr/bin/python3, not python
As-is, the application fails when it's unable to find python
bazel build completes without an issue, but bazel test continues to look for python, and fail.
bazel test ... --test_output=all --python_top=//sandbox_app:python-3.8.10
...
/usr/bin/env: 'python': No such file or directory
================================================================================
INFO: Elapsed time: 0.658s, Critical Path: 0.10s
INFO: Build completed, 1 test FAILED, 2 total actions
//sandbox_app:sandbox_test FAILED in 0.1s
This didn't work
The following configuration worked for a similar user, described here:
https://groups.google.com/g/bazel-discuss/c/nVQ48R94S_8
But even after a bazel clean, bazel still continues to invoke python when running the tests.
WORKSPACE
git_repository(
name = "io_bazel_rules_go",
remote = "https://github.com/bazelbuild/rules_go.git",
tag = "0.5.4",
)
BUILD.bazel
package(default_visibility = ["//visibility:public"])
py_test(
name = "sandbox_test",
srcs = ["sandbox_test.py"],
default_python_version = "PY3",
srcs_version = "PY3",
)
py_runtime(
name = "python-3.8.10",
files = [],
interpreter_path = "/usr/bin/python3",
)
Version details:
$ bazel info release
release 0.5.4
It would minimize disruption to the application if this can be done without upgrading bazel.
sandbox_test.py
#!/usr/bin/python3
import unittest
class SandboxTest(unittest.TestCase):
def testRunSandbox(self):
# Valid for Python 3, syntax error for python2
print(print("Hello, Python!"))
if __name__ == '__main__':
unittest.main()
Possibly related issues
https://github.com/bazelbuild/bazel/issues/4815
https://github.com/bazelbuild/bazel/issues/200
Rules spec:
https://bazel.build/reference/be/python
Similar issues:
How do I select the runtime in bazel for Python and pip?
Bazel, python3 and python interpreter options
Getting Bazel to run using Python3 (syntax error)

Error: "http_archive" is not defined when starting the local Bazel server in the TensorFlow Installation

I am installing Tensorflow. I encountered the following problem:
I was installing Tensorflow from Source and I am following the Tensorflow website (https://www.tensorflow.org/install/source)
I had earlier installed Bazel 3.0.0 (the latest version) but got a similar error. Upon looking at other answers I figured out this error has something to do with the version of Bazel. Hence I uninstalled Bazel 3.0.0 and installed Bazel 0.27.1. But upon typing the following command(bazel build //tensorflow/tools/pip_package:build_pip_package) in the terminal, I am getting the following error:
WARNING: The following rc files are no longer being read, please transfer their contents or import their path into one of the standard rc files:
/home/aviraj/tensorflow/tools/bazel.rc
Starting local Bazel server and connecting to it...
ERROR: /home/aviraj/tensorflow/WORKSPACE:3:1: name 'http_archive' is not defined
ERROR: Error evaluating WORKSPACE file
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '#io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package
ERROR: error loading package '': Encountered error while reading extension file 'closure/defs.bzl': no such package '#io_bazel_rules_closure//closure': error loading package 'external': Could not load //external package
INFO: Elapsed time: 2.023s
INFO: 0 processes.
FAILED: Build did NOT complete successfully (0 packages loaded)
Any help would be much appreciated. Thanks

Starting with version 0.20.0, http_archive native rule has been deprecated, it has to be loaded from: #bazel_tools//tools/build_defs/repo:http.bzl. I.e. you need to add the following (unless you have a customized version) in WORKSPACE or bzl files using it:
load("#bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")

No such package 'slim': BUILD file not found - Tensorflow

Currently trying to run Google's research, I come into the error below when following the instructions provided. Instruction on Github - Mint Linux system with Anaconda/Tensorflow download. This is the second step in "Getting the Datasets"
~/test/models/research $ bazel run domain_adaptation/datasets:download_and_convert_mnist_m -- --dataset_dir $DSN_DATA_DIR
ERROR: /home/.../models/research/domain_adaptation/datasets/BUILD:29:1: no such package 'slim': BUILD file not found on package path and referenced by '//research/domain_adaptation/datasets:download_and_convert_mnist_m'
ERROR: Analysis of target '//research/domain_adaptation/datasets:download_and_convert_mnist_m' failed; build aborted: no such package 'slim': BUILD file not found on package path
INFO: Elapsed time: 0.168s

The research models, including slim and domain_adaptation, have been moved to the //research package in this commit from the top level.
However, this change has not been reflected in the BUILD files for https://github.com/tensorflow/models/tree/master/research/domain_adaptation. You will need to prepend research into the BUILD targets such as this, this, and this (i.e. from //slim:mnist to //research/slim:mnist), therefore referencing the correct BUILD file for slim.

Error installing/compiling grpc on Ubuntu

I have protobuf compiler version 3.0 and need to install grpc and grpc python plugin. Following the tutorial, I added deb http://http.debian.net/debian jessie-backports main to my sources.list file and did sudo apt-get update and sudo apt-get install libgrpc-dev which returned
Package libgrpc-dev is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package 'libgrpc-dev' has no installation candidate
So, I decided to compile it from source as mentioned in INSTALL notes and did:
$ git clone https://github.com/grpc/grpc.git
$ cd grpc
$ git submodule update --init
$ make
$ [sudo] make install
However, on the make step, I get
[MAKE] Generating cache.mk
make: Circular /home/vagrant/grpc2/grpc/libs/opt/libboringssl.a <- /home/vagrant/grpc2/grpc/libs/opt/libboringssl.a dependency dropped.
[C] Compiling third_party/boringssl/crypto/bio/connect.c
third_party/boringssl/crypto/bio/connect.c: In function 'split_host_and_port':
third_party/boringssl/crypto/bio/connect.c:127:17: error: declaration of 'close' shadows a global declaration [-Werror=shadow]
cc1: all warnings being treated as errors
make: *** [/home/vagrant/grpc2/grpc/objs/opt/third_party/boringssl/crypto/bio/connect.o] Error 1
On switching to the release-0_11 branch, running make results in
[HOSTCXX] Compiling src/compiler/csharp_generator.cc
src/compiler/csharp_generator.cc:47:43: error: 'google::protobuf::compiler::csharp::GetUmbrellaClassName' has not been declared
src/compiler/csharp_generator.cc: In function 'void grpc_csharp_generator::{anonymous}::GenerateServiceDescriptorProperty(grpc::protobuf::io::Printer*, const ServiceDescriptor*)':
src/compiler/csharp_generator.cc:237:62: error: 'GetUmbrellaClassName' was not declared in this scope
make: *** [/home/vagrant/grpc2/grpc/objs/opt/src/compiler/csharp_generator.o] Error 1
I can't figure out how to install this. Any help would be appreciated.

For me the issue got fixed after i made change in the file as:
diff --git a/src/compiler/csharp_generator.cc
b/src/compiler/csharp_generator.cc
index 7b497df..5a8746d 100644
--- a/src/compiler/csharp_generator.cc
+++ b/src/compiler/csharp_generator.cc
## -44,7 +44,7 ##
using google::protobuf::compiler::csharp::GetFileNamespace;
using google::protobuf::compiler::csharp::GetClassName;
-using google::protobuf::compiler::csharp::GetUmbrellaClassName;
+using google::protobuf::compiler::csharp::GetReflectionClassName;
using grpc::protobuf::FileDescriptor;
using grpc::protobuf::Descriptor;
using grpc::protobuf::ServiceDescriptor;
## -234,7 +234,7 ## void GenerateServiceDescriptorProperty(Printer* out, const ServiceDescriptor *se
out->Print("public static global::Google.Protobuf.Reflection.ServiceDescriptor Descriptor\n");
out->Print("{\n");
out->Print(" get { return $umbrella$.Descriptor.Services[$index$]; }\n",
- "umbrella", GetUmbrellaClassName(service->file()), "index",
+ "umbrella", GetReflectionClassName(service->file()), "index",
index.str());
out->Print("}\n");
out->Print("\n");
More specifically, please open the file src/compiler/csharp_generator.cc and replace all references of GetUmbrellaClassName by GetReflectionClassName

Working backwards:
for release-0_11: it looks like you're trying to compile against the most recent protobuf. Since we're both in development right now there's occasional breakage - but grpc does track the version of protobuf it's tested against in third_party/protobuf. Try checking out and installing that version. I filed https://github.com/grpc/grpc/issues/4697 to update to the latest protobuf 3.0 version.
for master from github: which compiler and OS are you using? I recently checked in the boringssl integration work, so it's fresh, and not nearly as battle-tested. I'd like to get it battle-tested. That said, if you do a 'make EMBED_OPENSSL=false' then things should work out for you.
for the debian package problem: I'm not sure what's going on. I'm happy to try and spin up a VM with your OS and repro if you can let me know which OS it is.

Using cmake with enthought for python wrapping

I recently started trying to switch to canopy from ipython notebooks, which I switched to from MATLAB. I am working towards getting a matlab-like experience for development and research.
I am doing most of these setups cross-platform, but I am starting on OS X.
I have been attempting to install packages for use in canopy, some of which require compilation such as SimpleITK and VTK6.0. I had these both successfully installed in my regular python2.7 installation, but I'm having some trouble installing them for canopy. Right now, Im working on recompiling SimpleITK/Superbuild with CMAKE to recreate the python wrappers. It asks for your python executable, include dir, and python lib location. I found the python executeable from sys.executable from within canopy, and the include dir with 'mdfind -name Python.h, which turned out to be the default location, I used the default python lib.
These were
executeable: /Users/jmerkow/Library/Enthought/Canopy_64bit/User/bin/python
Include: /System/Library/Frameworks/Python.framework/Headers
Lib: /usr/lib/libpython2.7.dylib
But I get the following error upon make:
AssertionError: Filename /Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/os.py does not start with any of these prefixes: ['/Users/jmerkow/Library/Enthought/Canopy_64bit/User', '/Library/Python/2.7/site-packages', '/Users/jmerkow/Library/Enthought/Canopy_64bit/User/Extras/lib/python', '/Users/jmerkow/Library/Python/2.7/site-packages', '/Users/jmerkow/.local/lib/python/2.7/site-packages', '/Users/jmerkow/Library/Python/2.7/lib/python/site-packages']
ERROR
-- Installing numpy from /Users/jmerkow/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/numpy to /numpy
ERROR
/bin/sh: ../Testing/Installation/PythonVirtualenv/bin/python: No such file or directory
make[5]: *** [lib/_SimpleITK.so] Error 127
make[4]: *** [Wrapping/CMakeFiles/SimpleITK_PYTHON.dir/all] Error 2
make[3]: *** [all] Error 2
make[2]: *** [SimpleITK-prefix/src/SimpleITK-stamp/SimpleITK-build] Error 2
make[1]: *** [CMakeFiles/SimpleITK.dir/all] Error 2
make: *** [all] Error 2
Also, anyother tips to help me get this done are very welcome
EDIT BELOW
So I attempted to compile using:
/System/Library/Frameworks/Python.framework/Versions/Current/bin/python
/System/Library/Frameworks/Python.framework/Versions/Current/include/python2.7/
/System/Library/Frameworks/Python.framework/Versions/Current/lib/libpython2.7.dylib
This compiled and created and egg file named
SimpleITK-0.7.0.dev88_ge297c-py2.7-macosx-10.8-intel.egg
but I get the following error when I use enpkg
Traceback (most recent call last):
File "/Users/jmerkow/Library/Enthought/Canopy_64bit/User/bin/enpkg", line 10, in
sys.exit(main())
File "/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/site-packages/enstaller/main.py", line 702, in main
reqs.append(Req(name + ' ' + version))
File "/Applications/Canopy.app/appdata/canopy-1.1.0.1371.macosx-x86_64/Canopy.app/Contents/lib/python2.7/site-packages/enstaller/resolve.py", line 32, in init
raise Exception("Not a valid requirement: %r" % req_string)
Exception: Not a valid requirement: 'SimpleITK 0.7.0.dev88_ge297c-py2.7-macosx-10.8-intel.egg'
with easy_install I get this error:
Processing SimpleITK-0.7.0.dev88_ge297c-py2.7-macosx-10.8-intel.egg
removing '/Users/jmerkow/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/SimpleITK-0.7.0.dev88_ge297c-py2.7-macosx-10.8-intel.egg' (and everything under it)
creating /Users/jmerkow/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/SimpleITK-0.7.0.dev88_ge297c-py2.7-macosx-10.8-intel.egg
Extracting SimpleITK-0.7.0.dev88_ge297c-py2.7-macosx-10.8-intel.egg to /Users/jmerkow/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages
SimpleITK 0.7.0.dev88-ge297c is already the active version in easy-install.pth
Installed /Users/jmerkow/Library/Enthought/Canopy_64bit/User/lib/python2.7/site-packages/SimpleITK-0.7.0.dev88_ge297c-py2.7-macosx-10.8-intel.egg
Processing dependencies for SimpleITK==0.7.0.dev88-ge297c
Searching for SimpleITK==0.7.0.dev88-ge297c
Reading cant post
Reading cant post
Reading cant post
No local packages or download links found for SimpleITK==0.7.0.dev88-ge297c
error: Could not find suitable distribution for Requirement.parse('SimpleITK==0.7.0.dev88-ge297c')
It looked like I installed it, so I tried to import:
import SimpleITK
Fatal Python error: PyThreadState_Get: no current thread
Abort trap: 6

You have configure SimpleITK to use your system python executable, library and include files. However, you are trying to run with the Canopy distribution. They are not binary compatible and the mismatch caused that type of program termination.
You canopy python executable was correctly found:
/Users/jmerkow/Library/Enthought/Canopy_64bit/User/bin/python
But you need to make the PYTHON_INCLUDE_DIR and PYTHON_LIBRARY consistent. Try the following commands to help you determine the correct path:
find /Users/jmerkow/Library/Enthought/Canopy_64bit/ -name Python.h
find /Users/jmerkow/Library/Enthought/Canopy_64bit/ -name lib python*.dylib