Develop Reference

How do you install bazel using bazelisk

Or better yet, how do you install bazel at all. I have been using cd to get into a folder on my desktop, then I used git clone https://github.com/bazelbuild/bazelisk and then? What am I supposed to do now, honestly I didn't find any real instruction for this, I have many folder in there now, am I supposed to run a special file like bazelisk.py now? Or like build it? Was I supposed to clone the github repository in a special location? As mentioned I just choose a folder on my desktop, but I added the path to my Path enviromental variable, I added C:\Users\user\Desktop\folder\bazel and C:\Users\user\Desktop\folder\bazel\bazelisk because I wasn't sure how far I should go. So if anybody could show me how to install bazel at all I would be thankful, I just need it for a python project, I am using python 3.6.2 and I am on windows 10. I don't know what to do now because the installation steps I've read only go as far as saying to git clone the repository, but when I enter cmd and type bazel or bazel help cmd can not find the command.

From Official Bazel Documentation
https://github.com/bazelbuild/bazelisk/blob/master/README.md ->
Bazelisk is a wrapper for Bazel written in Go. It automatically picks a good version of Bazel given your current working directory, downloads it from the official server (if required) and then transparently passes through all command-line arguments to the real Bazel binary. You can call it just like you would call Bazel.
Steps to Installing Bazelisk in Linux/Mac/Windows
Below steps show installation of bazelisk v1.8.1, but you can install any desired version from https://github.com/bazelbuild/bazelisk/releases using same steps
Steps for Bazelisk Linux Install
> wget https://github.com/bazelbuild/bazelisk/releases/download/v1.8.1/bazelisk-linux-amd64
> chmod +x bazelisk-linux-amd64
> sudo mv bazelisk-linux-amd64 /usr/local/bin/bazel
# make sure you get the binary available in $PATH
> which bazel
bazel is /usr/local/bin/bazel
Steps for Bazelisk Mac Install
wget https://github.com/bazelbuild/bazelisk/releases/download/v1.8.1/bazelisk-darwin-amd64
chmod +x bazelisk-darwin-amd64
sudo mv bazelisk-darwin-amd64 /usr/local/bin/bazel
Steps for Bazelisk Windows Install
Download and install Bazelisk windows binary from https://github.com/bazelbuild/bazelisk/releases
How to use Bazelisk and set desired Bazel version
If the environment variable USE_BAZEL_VERSION is set, it will use the
version specified in the value.
Otherwise, if a .bazeliskrc file exists in the workspace root and
contains the USE_BAZEL_VERSION variable, this version will be used.
Otherwise, if a .bazelversion file exists in the current directory or
recursively any parent directory, it will read the file and use the
version specified in it.
Otherwise it will use the official latest Bazel release.
Common issues you might face
When bazel version is updated in WORKSPACE file and if you are unable to see bazel version update after bazel version call
Then try removing old bazel binary /usr/local/bin/bazel and again do bazelisk installation mentioned above.

Since you are using Windows I will describe the Windows way of doing it (it is also similar on other platforms)
Download a prebuild version of bazelisk.exe. You can find prebuilt versions here. For instance, v1.7.4. Then rename bazelisk-windows-amd64.exe to bazelisk.exe and add it to your PATH environment variable. When you open now a PowerShell and type bazelisk info you should see some output.
bazelisk is only a wrapper around bazel which means you have still to setup everything to make Bazel working on your system. bazelisk just redirects all commands to Bazel. bazelisk just cares about which version of Bazel is used (e.g. you can define in a .bazelversion file wich version of Bazel should be used). I suggest you to follow the instructions listed here that describe what needs to be done to get Bazel working on Windows (for instance installing MSYS2 makes sense).

I can not use bazel to build tensorflow because I installed bazel with bazelisk?

I installed a bazelisk exe file and included that file in my environmental Path variable. I can now run bazelisk commands but no bazel commands and I think I was told that that was normal. Is it? If it is, if I cd into my tensorflow folder and run python ./configure.py because I think that that is a step I need to do to build tensorflow from source I get the message Cannot find bazel. Please install bazel. What am I supposed to do? I am using python 3.6.2 and windows 10 and bazelisk is on v1.7.4

Try to rename bazelisk to bazel because it is just a wrapper for bazel

OSError when importing XGBoost

I have installed XGBoost using sudo pip install. When I attempt to lmport I receive an OSError library not loaded, image not found. I have installed Xcode and my compiler is clack. When I enter clack at the command line I receive another error: no input files possibly suggesting $PATH may need to be modified. How can $PATH be modified and will this allow XGBoost to be loaded properly?

The error saying no input files means you haven't given it any input files to compile. This is an expected error which even gcc, clang, etc. give. So, I do not think this is the correct debug step for your problem.
With pypi
When installing xgboost using pip on OSX, I receive an error saying clang-omp++ was not found. And eventually gives an error __builtin__.XGBoostLibraryNotFound: Cannot find XGBoost Libarary in the candicate path, did you install compilers and run build.sh in root path?. So, it has some bug and can't be installed it seems.
And this seems to be a known issue: https://github.com/dmlc/xgboost/issues/1446#issuecomment-239655078
Working method
This is how I got xgboost to be installed:
git clone --recursive https://github.com/dmlc/xgboost.git
cd xgboost
cp make/minimum.mk config.mk
make -j
This works using the builtin clang compiler as well as gcc from brew. For more info read https://xgboost.readthedocs.io/en/latest/build.html#building-on-osx
Now to install the python package:
cd python-package
python setup.py install
Read more at https://xgboost.readthedocs.io/en/latest/build.html#python-package-installation . Note that the python setup.py install may need sudo if you aren't using a virtualenv.

Fail when building xgboost using 64-bit mingw g++

I would like to compile xgboost for Anaconda Python 3.x in 64-bit Windows 10, so I followed the steps here - https://xgboost.readthedocs.org/en/latest/build.html#building-on-windows and here - Install xgboost under python with 64-bit msys failing. Using the 64-bit mingw g++, I got the following error message when "make -j4" using mingw64.mk:
g++: error: dmlc-core/libdmlc.a: No such file or directory
g++: error: rabit/lib/librabit_empty.a: No such file or directory
Makefile:120: recipe for target 'lib/libxgboost.dll' failed
mingw32-make: *** [lib/libxgboost.dll] Error 1
Can you please let me know what's going on and how I can resolve this?
Thanks a lot.

You have to first go to the dmlc-core and rabit sub directories and execute make there. Then come to the xgboost library and execute build.sh.

Maybe you were not in the right folder when running install. Try the following.
Pull up Anaconda prompt, run:
cd xgboost\python-package
python setup.py install
If this still fails, try the process in this link from scratch and it should work. Quote from the link:
First follow the official guide with the following procedure (in Git
Bash on Windows):
git clone --recursive https://github.com/dmlc/xgboost
git submodule init
git submodule update
then install TDM-GCC here and do the following in Git Bash:
alias make='mingw32-make'
cp make/mingw64.mk config.mk; make -j4
Last, do the following using anaconda prompt or Git Bash:
cd xgboost\python-package
python setup.py install

You are missing those libraries in your directory. You have to clone xgboost recursively to include them as well. Just follow the clone process as mentioned in the installation guide and you should be fine!

Python grpc protobuf stubs generation issue: --grpc_out: protoc-gen-grpc: Plugin failed with status code 1

As the question says, I compiled grpc from source and also did sudo pip install grpcio, however, the which grpc_python_plugin doesn't return anything. This is a problem because the grpc python example for route_guide requires me to run protoc -I . --python_out=. --grpc_out=. --plugin=protoc-gen-grpc='which grpc_python_plugin' ./route_guide.proto
in order to generate the python stubs. Since, which grpc_python_plugin doesn't return anything, I get the following error:
: program not found or is not executable
--grpc_out: protoc-gen-grpc: Plugin failed with status code 1.
If I shorten the command I'm trying to run to:protoc -I . --python_out=. ./route_guide.proto, it generates the route_guide_pb2.py file but without the Servicer and Stub classes, and server and stub methods. Ofc, these methods are necessary if one wants to use grpc for any purpose. Any help would be appreciated.

python -m grpc_tools.protoc --proto_path=. --python_out=. --grpc_python_out=. my_proto.proto
Edited:
Apparently, there is open Issue on gRPC github site regarding this problem. Protoc seems to have a compatibility issue with grpc_python_plugin? I solved this problem by installing grpc_tools, then used grpc_tools.protoc instead of protoc.
$ pip install grpcio-tools
$ pip install googleapis-common-protos
Useful python tutorial: https://grpc.io/docs/tutorials/basic/python.html
See Protoc Issues [791 and 4961]:
https://github.com/google/protobuf/issues/791
https://github.com/grpc/grpc/issues/4961

To resolve this error make sure you have grpc_python_plugin installed on your system.
On python platforms pip install grpcio doesn't install platform specific plugins, so you have to install them separately by taking following steps
a) cd grpc (grpc repository)
b) git submodule update --init
c) make grpc_python_plugin
This will build the grpc python plugin for you. Now, find out the location of grpc_python_plugin on your system, lets call it binary_path
If binary_path is under $PATH environment variable (echo $PATH), you are good to go. However, if it is not under $PATH variable, you have two options
Update run_codegen.sh to --plugin=protoc-gen-grpc=binary_path
or, copy the binary to one of the locations tracked by $PATH environment variable

It sounds like you don't have /usr/local/bin in your PATH; make install uses a prefix of /usr/local by default. Alternatively, after compilation grpc_python_plugin should be in bins/opt/.

the python's gRPC plugin for protoc is not installed by default
$ protoc -I=grpc/protos helloworld.proto --plugin=grpc_python_plugin --python_out=grpc/helloworld --grpc_python_out=grpc/helloworld
protoc-gen-grpc_pythong is not recognized as an internal or external command,
operable program or batch file.
--grpc_python_out: protoc-gen-grpc_python: Plugin failed with status code 1.
install protoc-gen-grpclib_python and protoc-gen-python_grpc
$ pip install grpclib protobuf
generate your python gRPC stubs (replace --grpc_python_out with --grpclib_python_out)
$ protoc -I=grpc/protos helloworld.proto --plugin=grpc_python_plugin --python_out=grpc/helloworld --grpclib_python_out=grpc/helloworld
https://github.com/grpc/grpc/issues/15675

I got a similar problem (which grpc_cpp_plugin was empty). In my case the problem was that I didn't have installed grpc and I solved this for my OS X with brew install grpc.