I have python 2.7 installed on my linux box, and I'm trying to schedule a python script via crontab. The script works fine from the command line, however when running via cron I get:
Traceback (most recent call last):
File "/usr/local/lib/python2.7/site.py", line 553, in <module>
main()
File "/usr/local/lib/python2.7/site.py", line 535, in main
known_paths = addusersitepackages(known_paths)
File "/usr/local/lib/python2.7/site.py", line 268, in addusersitepackages
user_site = getusersitepackages()
File "/usr/local/lib/python2.7/site.py", line 243, in getusersitepackages
user_base = getuserbase() # this will also set USER_BASE
File "/usr/local/lib/python2.7/site.py", line 233, in getuserbase
USER_BASE = get_config_var('userbase')
File "/usr/local/lib/python2.7/sysconfig.py", line 535, in get_config_var
return get_config_vars().get(name)
File "/usr/local/lib/python2.7/sysconfig.py", line 434, in get_config_vars
_init_posix(_CONFIG_VARS)
File "/usr/local/lib/python2.7/sysconfig.py", line 298, in _init_posix
raise IOError(msg)
IOError: invalid Python installation: unable to open /usr/include/python2.7/pyconfig.h (No such file or directory)
I see that /usr/include/python2.7 does't exist, but /usr/local/include/python2.7/ does. Did I make a mistake while installing python?
You probably just have 2 versions installed, one of which is broken. If your cron is just directly calling python instead of a specific path, your PATH probably contains /usr/bin before /usr/local/bin (which is typical) - so in your cron, specify which python to use, or remove the existing one in /usr/bin and symlink /path/to/good/python to /usr/bin/python.
Edit: scratch that, just re-read and saw that it works fine from the command line. python-dev is probably the way to go. Sorry!
You need python2.7-dev, which installs the includes and headers.
For Ubuntu, you run sudo apt-get install python2.7-dev to install it. What Linux distro are you running?
I am assuming that in your crontab file, you are giving the full path to the python executable, and not just relying on she-bang with executable permissions. If not, please give point to the full-path python2.7 in the crontab file and also use the same full-path on the command line to ensure that you don't get this problem.If you get this on command line too, then it is probably missing some development headers. (Are you trying to compile something like using setup.py build and trying to do it via crontab?) I am trying to understand where would one need those headers. So, apart from the above suggestion, extra information from your end might ofcourse help further.
Related
If found some hits on stackoverflow where people have issues using ete3 tools when PyQT4 is not installed, and this (used to) also apply for me. However, I've now tried to reinstall a piece of software and it keeps complaining about 'Module' has no attritube 'Treestyle'.
To test whether PyQT was installed, I ran this python script:
from PyQt4.Qt import PYQT_VERSION_STR
print("PyQt version:", PYQT_VERSION_STR)
Which prints:
('PyQt version:', '4.12.1')
I get this (common) error:
Traceback (most recent call last):
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 1820, in <module>
sys.exit(main())
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 1793, in main
args.start(args)
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 408, in start_evo_sim
result = init_and_simulate(simu, intermediate_load_file, _options)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/my_tools/utility.py", line 117, in wrapper
raise ex_type(message)
AttributeError: 'module' object has no attribute 'TreeStyle' (in subprocess)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/my_tools/utility.py", line 91, in process_func
ret = func(*args, **kwargs)
File "/mnt/f/mypy/bin/virtualmicrobes.py", line 295, in init_and_simulate
sim = sim_mod.create_simulation(**options)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 1940, in create_simulation
sim = ODE_simulation(params_dict)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 1342, in __init__
super(ODE_simulation, self).__init__(params)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 86, in __init__
self.init_graphs()
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/simulation/Simulation.py", line 795, in init_graphs
show=show, clean=clean, create=create)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1188, in __init__
self.init_phylo_tree_graph(clean=clean)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1219, in init_phylo_tree_graph
show=show, attribute_dict=self.attribute_mapper, create=clean, **kwargs)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 985, in __init__
self.init_tree_style_dict() # NOTE: unordered ok
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1110, in init_tree_style_dict
branch_vertical_margin=branch_vertical_margin)
File "/mnt/f/mypy/local/lib/python2.7/site-packages/VirtualMicrobes/plotting/Graphs.py", line 1083, in make_tree_style
ts = ete3.TreeStyle()
Note: this time arround I am trying to install it on Ubuntu for Windows, but up till now almost everything that ran on Linux ran on this one too. It might be relevent though.
Any ideas?
PS To reproduce the problem, follow the following steps:
$ sudo apt-get install python-qt4
$ virtualenv ~/mypy --system-site-packages
$ source ~/mypy/bin/activate
$ (mypy) > pip install VirtualMicrobes
$ (mypy) > virtualmicrobes evo --name TestMicrobes >> error
PPS User eyllanesc confirmed this is an 'Ubuntu for Windows' problem. I'm still trying to fix it though, if anyone can help me figure out how to link the pyqt4 packages correctly to my virtualenv that would be great. I've tried linken the native one with ln -s but I'm not getting any further
I had similar issues. The problem might be that you don't have all the depencies installed and ete3 is not telling you all of them.
You can see which dependencies are missing with this code:
import ete3
ete3.__file__
Run it and it will tell you which dependencies are missing. Note that you might need to run it multiple times before you install them all!
I have a Python 2 package that I'm trying to upgrade to Python 3. It was written by someone who used to work on the same team I'm on now but who is no longer with the company, and unfortunately nobody left on the team is able to help out.
After running 2to3 on the files of the package, I ran python setup.py sdist to create a package, placed the package in a local repository, then attempted to use pip install to install the package. It ended up erroring out with the following:
Exception:
Traceback (most recent call last):
File "/home/user/project/lib/python3.5/site-packages/pip/basecommand.py", line 223, in main
status = self.run(options, args)
File "/home/user/project/lib/python3.5/site-packages/pip/commands/install.py", line 297, in run
root=options.root_path,
File "/home/user/project/lib/python3.5/site-packages/pip/req/req_set.py", line 622, in install
**kwargs
File "/home/user/project/lib/python3.5/site-packages/pip/req/req_install.py", line 808, in install
self.move_wheel_files(self.source_dir, root=root)
File "/home/user/project/lib/python3.5/site-packages/pip/req/req_install.py", line 1003, in move_wheel_files
isolated=self.isolated,
File "/home/user/project/lib/python3.5/site-packages/pip/wheel.py", line 340, in move_wheel_files
assert info_dir, "%s .dist-info directory not found" % req
AssertionError: my-package-name .dist-info directory not found
The old Python 2 version of the package didn't have anything called .dist-info in the .tgz archive, and it installed just fine. Does anyone know what's going on here and how to fix it?
In my case this was resolved by deleting AppData\Local\pip\Cache folder (windows). Should be fairly similar for other operating systems.
I'm attempting to install pip for arcpy (arcgis 10.2 on windows 7). Running get-pip.py results in the following error message:
X:\python>python get-pip.py
Traceback (most recent call last):
File "get-pip.py", line 20061, in <module>
main()
File "get-pip.py", line 194, in main
bootstrap(tmpdir=tmpdir)
File "get-pip.py", line 82, in bootstrap
import pip
File "c:\temp\tmpou5fje\pip.zip\pip\__init__.py", line 26, in <module>
File "c:\temp\tmpou5fje\pip.zip\pip\utils\__init__.py", line 27, in <module>
File "c:\temp\tmpou5fje\pip.zip\pip\_vendor\pkg_resources\__init__.py", line 73, in <module>
File "c:\temp\tmpou5fje\pip.zip\pip\_vendor\packaging\specifiers.py", line 275, in <module>
File "c:\temp\tmpou5fje\pip.zip\pip\_vendor\packaging\specifiers.py", line 373, in Specifier
File "C:\Python27\ArcGIS10.2\Lib\re.py", line 190, in compile
return _compile(pattern, flags)
File "C:\Python27\ArcGIS10.2\Lib\re.py", line 242, in _compile
raise error, v # invalid expression
sre_constants.error: nothing to repeat
Using an administrator command prompt doesn't help. My real goal is to get win32com working under arcpy. I usual just copy the appropriate directories out of c:\python27\lib\site-packages to c:\python27\arcgis10.2\lib\site-packages to install a package under arcpy (why doesn't arcpy come with pip?) but that's not working for win32com, presumably do to a missing dll or other windows specific file.
I would recommend the following:
Get the setuptools module
Get the pip module`
And then run the following in command line (assuming windows)
path-to-python path-to-setuptools install
path-to-python path-to-pip install
I work on a closed network (away from the interwebs of old) and cannot use get-pip.py so I find it best to simply download the actual modules and hard install.
Keep us posted!
Copy get_pip.py to "C:\Python27\ArcGIS10.2", then perform command "python get-pip.py" in the directory.
Note that keep network connected in the process, so that auto-download and setup setuptools,wheels,etc.
Hope that can help you.
Try opening a CMD prompt and typing:
C:\Python27\ArcGIS10.2\python.exe -m pip install -U pip
Context: I started using OSX about a year ago, and I had a kind of screwy python installation. That is, I was using system python, and installed packages with sudo when that seemed to make things work. Now, I'm starting from a fresh OSX install, and trying to do it the Right Way. I've installed python and python3 from brew, and trying to use python3 whenever possible.
Problem: I have a .pythonrc file, which just imports a handful of commonly used packages - mostly standard lib, a few popular nonstandard packages, and a few of my own. In the past, this file has only been run when I start an interactive shell. Now, when using brew python, it is run whenever I run any python program.
There must be some gap in my understanding of the rc file - I thought the purpose was specifically for interactive use. Still, when I use system python, the rc file isn't used - so something is different about my system python (2.7.10 at /usr/bin/python) vs brew python (2.7.12 at /usr/local/bin/python; 3.5.2 at /usr/local/bin/python3). The behavior is the same if I remove everything except a print statement from the rc file.
edit: I realized that the rc file is running because I'm importing ipdb. This makes sense I suppose, but I still don't understand why that would happen in some of the python environments/versions but not others.
edit: https://github.com/gotcha/ipdb/blob/master/ipdb/main.py#L44 this line fails, not sure if this means anything.
Full stacktrace from within .pythonrc:
File "hello.py", line 1, in <module>
from ipdb import set_trace
File "/usr/local/lib/python2.7/site-packages/ipdb/__init__.py", line 7, in <module>
from ipdb.__main__ import set_trace, post_mortem, pm, run # noqa
File "/usr/local/lib/python2.7/site-packages/ipdb/__main__.py", line 51, in <module>
ipapp.initialize([])
File "<decorator-gen-109>", line 2, in initialize
File "/usr/local/lib/python2.7/site-packages/traitlets/config/application.py", line 74, in catch_config_error
return method(app, *args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/IPython/terminal/ipapp.py", line 315, in initialize
self.init_code()
File "/usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py", line 263, in init_code
self._run_startup_files()
File "/usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py", line 342, in _run_startup_files
self._exec_file(python_startup)
File "/usr/local/lib/python2.7/site-packages/IPython/core/shellapp.py", line 328, in _exec_file
raise_exceptions=True)
File "/usr/local/lib/python2.7/site-packages/IPython/core/interactiveshell.py", line 2469, in safe_execfile
self.compile if kw['shell_futures'] else None)
File "/usr/local/lib/python2.7/site-packages/IPython/utils/py3compat.py", line 288, in execfile
builtin_mod.execfile(filename, *where)
File "~/.pythonrc", line 57, in <module>
import traceback; traceback.print_stack()
You can ask the traceback module:
$ cat .pythonrc
import traceback; traceback.print_stack()
$ cat test.py
import ipdb
Then by running
$ PYTHONSTARTUP="$HOME/.pythonrc" python test.py
you should get a traceback that tells you exactly from where the startup script is being run. Most likely, this is due to a call
start_ipython()
somewhere in the ipdb import.
I have Python 2.6.6. to run a certain script on my Red Hat Linux VM. Need to upgrade to 3.4.1. Tried
easy_install --upgrade python
But I always get
bash-4.1# easy_install --upgrade python
Searching for python
Reading http://pypi.python.org/simple/python/
Reading http://www.python.org
Reading http://www.python.org/2.3
Reading http://www.python.org/2.4
Reading http://www.python.org/2.4.1
Reading http://www.python.org/2.5
Reading http://www.python.org/download/
Best match: Python 3.4.1
Downloading https://www.python.org/ftp/python/3.4.1/Python-3.4.1.tgz
Processing Python-3.4.1.tgz
Running Python-3.4.1/setup.py -q bdist_egg --dist-dir /tmp/easy_install-Qd_HVP/Python-3.4.1/egg-dist-tmp-iqaG4H
Traceback (most recent call last):
File "/usr/bin/easy_install", line 9, in <module>
load_entry_point('distribute==0.6.10', 'console_scripts', 'easy_install')()
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 1715, in main
with_ei_usage(lambda:
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 1696, in with_ei_usage
return f()
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 1719, in <lambda>
distclass=DistributionWithoutHelpCommands, **kw
File "/usr/lib64/python2.6/distutils/core.py", line 152, in setup
dist.run_commands()
File "/usr/lib64/python2.6/distutils/dist.py", line 975, in run_commands
self.run_command(cmd)
File "/usr/lib64/python2.6/distutils/dist.py", line 995, in run_command
cmd_obj.run()
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 236, in run
self.easy_install(spec, not self.no_deps)
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 472, in easy_install
return self.install_item(spec, dist.location, tmpdir, deps)
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 502, in install_item
dists = self.install_eggs(spec, download, tmpdir)
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 681, in install_eggs
return self.build_and_install(setup_script, setup_base)
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 958, in build_and_install
self.run_setup(setup_script, setup_base, args)
File "/usr/lib/python2.6/site-packages/setuptools/command/easy_install.py", line 947, in run_setup
run_setup(setup_script, args)
File "/usr/lib/python2.6/site-packages/setuptools/sandbox.py", line 29, in run_setup
lambda: execfile(
File "/usr/lib/python2.6/site-packages/setuptools/sandbox.py", line 70, in run
return func()
File "/usr/lib/python2.6/site-packages/setuptools/sandbox.py", line 31, in <lambda>
{'__file__':setup_script, '__name__':'__main__'}
File "setup.py", line 1865
exec(f.read(), globals(), fficonfig)
SyntaxError: unqualified exec is not allowed in function 'configure_ctypes' it contains a nested function with free variables
bash-4.1# SyntaxError: unqualified exec is not allowed in function 'configure_ctypes' it contains a nested function with free variables
And of course I can't delete Python 2.6.6 from the machine as a lot of system scripts depend on it. Any ideas what to do?
I generally advise against manually upgrading packages on an RPM-managed distribution such as RHEL. If an upgrade is needed, it should only be done by upgrading via rpm. There is a reason why the rpm tools exists, and why it is used. There's more than one reason, actually. It serves an important purpose, and manually installing or upgrading packages completely subverts it.
If it's absolutely necessary, the following procedure should be as follows:
Grab the source RPM for Red Hat's python package.
Grab the source for the new version of python.
Figure out what patches in Red Hat's python RPM, if any, are still applicable to the new version of python.
Use the spec file from the python RPM to build rpms with a new version of python.
Hope that step #4 works. If so, yay! Install it. If not, figure out why not, patch the spec file, as needed, go back to step #4.
You now installed a new version of python. Check that none of the other RHEL rpms that use python break with the new version. If not, yay! You're done. If something is broken, figure out what to do about it.
Many RHEL system management tools and scripts use Python. It's not entirely out of the question that something ends up being broken, as a result of the new python package being installed.
The whole reason for using RHEL is to have a commercially-supported, stable Linux distribution whose all components have been tested for interoperability. Upgrading random parts of it misses the whole point of having RHEL; typically you upgrade to an entire new release of RHEL, instead of the individual packages; but, to each their own...
Red Hat supplies multiple versions of Python (and other packages) that can be installed in parallel as part of software collections. They leave the system /usr/bin/ intact and are supported by Red Hat.
You need to:
* enable the RHSCL and optional software repos
* then you can yum install rh-python36 or python27 (2.7.13)
* Use scl enable rh-python 36 bash to add python to your path.
See How to install Python 3 on RHEL. It covers many tips for working with multiple versions of Python, Python virtual environments, and software collections.
You will need to find a source for Python-3.4 packages, or build it yourself.
A quick search finds http://wiki.guibin.info/?p=133 for building Python-3.4 yourself.
With some more searching, you may be able to find a source for RPM packages, but of course it's up to you how much you trust those packages -- I wouldn't use a random RPM in production!