I am trying to build TensorFlow from source. After configuring the installation, when I try to build to the pip package with the following command,
$ bazel build --config=opt //tensorflow/tools/pip_package:build_pip_package
I get the following error message:
ERROR: /workspace/tensorflow/core/BUILD:1312:1: Executing genrule //tensorflow/core:version_info_gen failed: bash failed: error executing command
(cd /root/.cache/bazel/_bazel_root/eab0d61a99b6696edb3d2aff87b585e8/execroot/workspace && \
exec env - \
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin \
/bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; tensorflow/tools/git/gen_git_source.py --generate tensorflow/tools/git/gen
/spec.json tensorflow/tools/git/gen/head tensorflow/tools/git/gen/branch_ref "bazel-out/host/genfiles/tensorflow/core/util/version_info.cc"'): com.goo
gle.devtools.build.lib.shell.BadExitStatusException: Process exited with status 1.
Traceback (most recent call last):
File "tensorflow/tools/git/gen_git_source.py", line 260, in <module>
generate(args.generate)
File "tensorflow/tools/git/gen_git_source.py", line 212, in generate
git_version = get_git_version(data["path"])
File "tensorflow/tools/git/gen_git_source.py", line 152, in get_git_version
str("--work-tree=" + git_base_path), "describe", "--long", "--tags"
File "/usr/lib/python2.7/subprocess.py", line 566, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Target //tensorflow/tools/pip_package:build_pip_package failed to build
INFO: Elapsed time: 8.567s, Critical Path: 7.90s
What's going wrong?
(Ubuntu 14.04, CPU only)
Your build appears to be encountering an error in
tensorflow/tools/git/gen_git_source.py
at line 152. At this stage in the build the script is trying to get the git version number of your tensor flow repo. Have you used git to check out your tensor flow repo? Are the .git files present in the /tensorflow/ root dir? Maybe you need to update your version of git?
Looks similar to this question: Build Error Tensorflow
I encountered this error even though I had git in my PATH variable. I got a hint from https://stackoverflow.com/a/5659249/212076 that the launched subprocess was not getting the PATH variable.
The solution was to hard-code the git command in <ROOT>\tensorflow\tensorflow\tools\git\gen_git_source.py by replacing
val = bytes(subprocess.check_output([
"git", str("--git-dir=%s/.git" % git_base_path),
str("--work-tree=" + git_base_path), "describe", "--long", "--tags"
]).strip())
with
val = bytes(subprocess.check_output([
"C:\Program Files (x86)\Git\cmd\git.cmd", str("--git-dir=%s/.git" % git_base_path),
str("--work-tree=" + git_base_path), "describe", "--long", "--tags"
]).strip())
Once this got fixed I got another error: fatal: Not a git repository: './.git'.
I figured the tensorflow root folder was the one that should have been referenced and so I edited <ROOT>\tensorflow\tensorflow\tools\git\gen_git_source.py to replace
git_version = get_git_version(".")
with
git_version = get_git_version("../../../../")
After that the build was successful.
NOTE: Unlike OP, my build platform was Windows 7 64 bit
Related
I'm trying to get PyLucene going on my computer and have followed all of the steps but am getting stuck at the "make" step. I'm working on 64-bit Windows 10 Machine and my version of Python is 3.9.12.
What I've done:
1.) Install Apache Ant and set environments. In this step, I also installed Java JDK 18 as per the recommendation in Apache Ant. I set all of the environment variables as instructed for JAVA_HOME, ANT_HOME, and added Ant to the path. I checked and verified that these all worked:
2.) Download PyLucene 9.1.0, extract the tar file, cd into the jcc folder and run python setup.py build. I changed the JAVAHOME in the setup.py file to JAVA_HOME to match the variable path name I made in the above step. I also included JCC_JDK because I got an error the first time I ran the setup.py where it asked for JCC_JDK.
3.) I edited the Makefile and made sure to have make installed for Windows. The top part of the make file is:
VERSION=9.1.0
LUCENE_VER=9.1.0
PYLUCENE:=$(shell pwd)
LUCENE_SRC=lucene-java-$(LUCENE_VER)
LUCENE=$(LUCENE_SRC)/lucene
PREFIX_PYTHON=C:\Users\mgsci\anaconda3 # this is where my python.exe is located
PYTHON=$(PREFIX_PYTHON)\python.exe
JCC=$(PYTHON) -m jcc --shared
NUM_FILES=16
When I ran make I got the error:
(base) C:\Users\mgsci\Downloads\PyLucene\pylucene-9.1.0>make
process_begin: CreateProcess(NULL, pwd, ...) failed.
process_begin: CreateProcess(NULL, which icupkg, ...) failed.
process_begin: CreateProcess(NULL, uname, ...) failed.
process_begin: CreateProcess(NULL, uname, ...) failed.
(cd lucene-java-9.1.0; ./gradlew collectRuntimeJars)
process_begin: CreateProcess(NULL, uname, ...) failed.
process_begin: CreateProcess(NULL, (cd lucene-java-9.1.0; ./gradlew collectRuntimeJars), ...) failed.
make (e=2): The system cannot find the file specified.
make: *** [lucene] Error 2
And when I just ran python -m jcc I got the error:
Traceback (most recent call last):
File "C:\Users\mgsci\anaconda3\lib\runpy.py", line 188, in _run_module_as_main
mod_name, mod_spec, code = _get_module_details(mod_name, _Error)
File "C:\Users\mgsci\anaconda3\lib\runpy.py", line 147, in _get_module_details
return _get_module_details(pkg_main_name, error)
File "C:\Users\mgsci\anaconda3\lib\runpy.py", line 111, in _get_module_details
__import__(pkg_name)
File "C:\Users\mgsci\anaconda3\lib\site-packages\jcc-3.12-py3.9-win-amd64.egg\jcc\__init__.py", line 26, in <module>
add_jvm_dll_directory_to_path(find_jvm_dll)
File "C:\Users\mgsci\anaconda3\lib\site-packages\jcc-3.12-py3.9-win-amd64.egg\jcc\windows.py", line 104, in add_jvm_dll_directory_to_path
raise ValueError("jvm.dll could not be found")
ValueError: jvm.dll could not be found
I have the jvm.dll file. I can see it in the path jdk-18.0.2\bin\server. I also copied the jvm.dll file and placed it both in the jdk-18.0.2 folder and in the bin folder for good measure and still got this error.
What part of process did I screw up?
I've a package on Bitbucket which contains code files in Python, R, and bash.
I'm using a laptop running Linux CentOS 7.
I'd like to create a conda package for it. I've started with creating a conda recipe, but I probably made some mistakes. I'm using conda 4.3.18.
I tried to build my conda recipe with the following command, but it generated several errors that I cannot interpret:
$ conda build behst_conda_recipe/
BUILD START: behst--0
pulling from https://bitbucket.org/PROJECT_ADDRESS
searching for changes
no changes found
checkout: 'tip'
updating to branch default
108 files updated, 0 files merged, 0 files removed, 0 files unresolved
0 files updated, 0 files merged, 0 files removed, 0 files unresolved
Package: behst--0
source tree in: /home/davide/miniconda3/conda-bld/behst_1495134385344/work
+ source /home/davide/miniconda3/bin/activate /home/davide/miniconda3/conda-bld/behst_1495134385344/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac
+ set -o nounset -o pipefail -o errexit
+ set -o xtrace
+ echo 'Running build.sh'
Running build.sh
INFO conda_build.build:bundle_conda(861): Packaging behst--0
number of files: 0
Fixing permissions
Fixing permissions
Traceback (most recent call last):
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/utils.py", line 133, in _copy_with_shell_fallback
stderr=subprocess.PIPE, stdout=subprocess.PIPE)
File "/home/davide/miniconda3/lib/python3.5/subprocess.py", line 581, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command 'cp -a /home/davide/miniconda3/conda-bld/behst_1495134385344/work/LICENSE /home/davide/miniconda3/conda-bld/behst_1495134385344/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/info/LICENSE.txt' returned non-zero exit status 1
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/davide/miniconda3/bin/conda-build", line 6, in <module>
sys.exit(conda_build.cli.main_build.main())
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 334, in main
execute(sys.argv[1:])
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/cli/main_build.py", line 325, in execute
noverify=args.no_verify)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/api.py", line 97, in build
need_source_download=need_source_download, config=config)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1518, in build_tree
config=config)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 1154, in build
built_package = bundlers[output_dict.get('type', 'conda')](output_dict, m, config, env)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 893, in bundle_conda
create_info_files(metadata, files, config=config, prefix=config.build_prefix)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 494, in create_info_files
copy_license(m, config)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/build.py", line 272, in copy_license
locking=config.locking)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/utils.py", line 177, in copy_into
_copy_with_shell_fallback(src, dst_fn)
File "/home/davide/miniconda3/lib/python3.5/site-packages/conda_build/utils.py", line 136, in _copy_with_shell_fallback
raise OSError("Failed to copy {} to {}. Error was: {}".format(src, dst, e))
OSError: Failed to copy /home/davide/miniconda3/conda-bld/behst_1495134385344/work/LICENSE to /home/davide/miniconda3/conda-bld/behst_1495134385344/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/info/LICENSE.txt. Error was: Command 'cp -a /home/davide/miniconda3/conda-bld/behst_1495134385344/work/LICENSE /home/davide/miniconda3/conda-bld/behst_1495134385344/_b_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_plac/info/LICENSE.txt' returned non-zero exit status 1
Exception ignored in: <bound method BaseFileLock.__del__ of <filelock.UnixFileLock object at 0x7f30df7349e8>>
Traceback (most recent call last):
File "/home/davide/miniconda3/lib/python3.5/site-packages/filelock.py", line 305, in __del__
File "/home/davide/miniconda3/lib/python3.5/site-packages/filelock.py", line 292, in release
File "/home/davide/miniconda3/lib/python3.5/site-packages/filelock.py", line 371, in _release
AttributeError: 'NoneType' object has no attribute 'flock'
Does anyone know what these mistakes mean?
EDIT: Here's the meta.yaml file:
package:
name: behst
source:
hg_url: https://bitbucket.org/PROJECT_ADDRESS
about:
home: https://bitbucket.org/PROJECT_ADDRESS
license: BSD
license_file: LICENSE
While the build.sh at the moment is just an echo command:
#!/bin/bash
#
#$ -cwd
#$ -S /bin/bash
#
set -o nounset -o pipefail -o errexit
set -o xtrace
echo "Running build.sh"
Do you have your home dir encrypted? Here are related issues on github.
The solution is to use --croot argument pointing somewhere outside your home, for e.g. /tmp/_conda_build_.
You seem to have your Home directory encrypted. The character length when dealing with encrypted folder is reduced from the normal 255 character length.
Hence the only solution, even as per the contributors for conda, is to use --croot with a non encrypted location like /tmp/whateverfolder
conda config --prepend pkgs_dirs /tmp/temp_conda_recipe
and then
conda build behst_conda_recipe/ --croot /tmp/temp_conda_recipe
I'm trying to configure 2 Ubuntu servers to use Python and Tensorflow for my project. I finished the 1st server, however, there are some very unusual and annoying errors with the 2nd one. This is the log when I run pip check on the 1st server:
~$: pip check
No broken requirements found.
And the error log on the 2nd one:
~$: pip check
No broken requirements found.
Traceback (most recent call last):
File "/home/mju-hpc-02/.local/bin/pip", line 11, in <module>
sys.exit(main())
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/__init__.py", line 233, in main
return command.main(cmd_args)
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/basecommand.py", line 251, in main
timeout=min(5, options.timeout)) as session:
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/basecommand.py", line 72, in _build_session
insecure_hosts=options.trusted_hosts,
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/download.py", line 329, in __init__
self.headers["User-Agent"] = user_agent()
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/download.py", line 93, in user_agent
from pip._vendor import distro
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/_vendor/distro.py", line 1050, in <module>
_distro = LinuxDistribution()
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/_vendor/distro.py", line 594, in __init__
if include_lsb else {}
File "/home/mju-hpc-02/.local/lib/python3.5/site-
packages/pip/_vendor/distro.py", line 931, in _get_lsb_release_info
raise subprocess.CalledProcessError(code, cmd, stdout, stderr)
subprocess.CalledProcessError: Command 'lsb_release -a' returned non-zero
exit status 1
This error also shows everytime I'm trying to install a new package with pip. Anyone faced similar problems with pip or having a way to diagnose the error?
EDIT 1:
As languitar suggestion, I ran lsb_release -a on both machines and this is the log on the 1st one:
~$: lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description: Ubuntu 16.04.2 LTS
Release: 16.04
Codename: xenial
2nd one:
~$: lsb_release -a
Traceback (most recent call last):
File "/usr/bin/lsb_release", line 25, in <module>
import lsb_release
ImportError: No module named 'lsb_release'
You did not mention your Python version(s) available on your system.
Not sure how this happen, but often 'lsb_version' is available to run on python v2 and others on v3.
Most likely you have a multiple python version on your host.
subprocess.CalledProcessError: Command 'lsb_release -a' returned non-zero
exit status 1
Is, indeed, failing because the 'lsb_release' library called by '/usr/bin/lsb_release' is not available for your particular Python version.
Your first attempt to fix this is simple:
sudo apt install lsb-release
Note is '-' and not '_'
Try again, if keep failing try the following: Look for your Python version library on /usr/lib, and confirm that 'lsb_release.py' is not there. Then, link the shared version to your library:
ln -sfv /usr/share/pyshared/lsb_release.py /usr/lib/python3.7/site-packages/lsb_release.py
Last resource, search /usr for any lsb_release.py and link it to your Python library.
find /usr -name lsb_release.py
You can just use default lsb_release or perhaps a bit modified version:
1 #!/bin/bash
171 [ -z "$LSB_VERSION" ] && LSB_VERSION="1.4"
172 #$MSG_NA
It seems your lsb_release binary is broken. Try to reinstall lsb-release (ubuntu or debian package).
I'm currently trying to create a ranker by running the train.py script provided by the Retrieve & Rank Documentation.
I issue the following command when trying to create the ranker:
train.py -u usernameHere:PasswordHere -i Mainframe_Batch_Automation_CA7.csv -c clusterIdHere -x Mainframe_Batch_Automation_CA7 -n Mainframe_Batch_Automation_CA7
And I receive the following error:
Input file is Mainframe_Batch_Automation_CA7.csv
Solr cluster is sc0c2398d4_d087_4516_99dd_10c25cc06b0a
Solr collection is Mainframe_Batch_Automation_CA7
Ranker name is Mainframe_Batch_Automation_CA7
Rows per query 10
Generating training data...
Traceback (most recent call last):
File "C:\Users\IBM_ADMIN\Documents\Retrieve & Rank\train.py", line 83, in <module>
process = subprocess.Popen(shlex.split(curl_cmd), stdout=subprocess.PIPE)
File "C:\Python27\lib\subprocess.py", line 390, in __init__
errread, errwrite)
File "C:\Python27\lib\subprocess.py", line 640, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the specified file
At first I thought it was because Python wasn't in my PATH environment variable but after adding it to the PATH variable it still gives me the same error ...
Script runs curl command which connects to solr and ranker urls. This error is due to firewall issues. Try to execute the script outside firewall.
I'm trying to install Open CV 2 on a shared hosting inside a virtualenv.
I already got numpy and all those stuff downloaded using Pip. I'm just having a bit of trouble with OpenCV2
I run this command in the ssh session
(penv)[dire]$ cmake -D MAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=$VIRTUAL_ENV/local/ -D PYTHON_EXECTUABLE=$PYTHONPATH/python2.7 -D PYTHON_PACKAGES_PATH=$VIRTUAL_ENV/lib/python2.7/site-packages -D INSTALL_PYTHON_EXAMPLES=ON ..
and the error I get is
Traceback (most recent call last):
File "/home/bashtroubles/website.com/public/NNPics/penv/bin/cmake", line 11, in <module>
sys.exit(cmake())
File "/home/bashtroubles/website.com/public/NNPics/penv/local/lib/python2.7/site-packages/cmake/__init__.py", line 33, in cmake
raise SystemExit(_program('cmake', sys.argv[1:]))
File "/home/bashtroubles/website.com/public/NNPics/penv/local/lib/python2.7/site-packages/cmake/__init__.py", line 29, in _program
return subprocess.call([os.path.join(CMAKE_BIN_DIR, name)] + args)
File "/usr/lib/python2.7/subprocess.py", line 493, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 679, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
OSError: [Errno 13] Permission denied
(penv)[dire]$
This is also my .bashrc if it matters
# ~/.bashrc: executed by bash(1) for non-login shells.
# Load pythonbrew
alias pb='pythonbrew'
export PYTHONPATH=~/.pythonbrew/pythons/Python-2.7.3/lib
[[ -s /home/bashtroubles/.pythonbrew/etc/bashrc ]] && source /home/bashtroubles/.python$
# Load custom python installation
export PATH=~/opt/python-2.7.3/bin:${PATH}
export PYTHONPATH=~/opt/python-2.7.3/lib
The specific version is opencv-2.4.13 and the python version is 2.7.3
I believe the issue is because it's using the python2.7 from
File "/usr/lib/python2.7/subprocess.py", line 1249, in _execute_child
raise child_exception
Any ideas on how to get this going without a permission denied issue?
I ran into this problem. It looks like the binary it's trying to call isn't marked as an executable. I ran this to change the permissions
sudo chmod +x -R /usr/local/lib/python2.7/dist-packages/cmake-3.13.3-py2.7-linux-x86_64.egg/cmake/data/bin