I'm pip installing PySide into a virtualenv:
sudo virtualenv --always-copy $VENV
source $VENV/bin/activate
sudo $VENV/bin/pip install pyside
sudo $VENV/bin/python $VENV/bin/pyside_postinstall.py -install
deactivate
The virtualenv works great and I can run my Python/PySide script without problems:
$VENV/bin/python $SCRIPT
Now, I make my virtualenv relocatable:
virtualenv --relocatable $VENV
I move it:
mv $VENV $VENV_RELOCATED
...and again run my script:
source $VENV_RELOCATED/bin/activate
$VENV_RELOCATED/bin/python $SCRIPT
This time around, I'm getting an error:
Traceback (most recent call last):
File "script.py", line 1, in <module>
from PySide import QtCore, QtGui, QtUiTools
ImportError: dlopen(/absolute/path/to/relocated_venv/lib/python2.7/site-packages/PySide/QtCore.so, 2): Library not loaded: #rpath/libpyside-python2.7.1.2.dylib
Referenced from: /absolute/path/to/relocated_venv/lib/python2.7/site-packages/PySide/QtCore.so
Reason: image not found
These are the contents of the script:
from PySide import QtCore, QtGui, QtUiTools
Please note: the absolute path printed twice in the error message is the $VENV_RELOCATED
This is all on OS X 10.11.0 with Python 2.7.10 and PySide 1.2.2.
Question: How should I properly install PySide into a relocatable virtualenv?
Turns out that the next PySide release (probably tagged "1.2.3" when released) has this fixed: https://github.com/PySide/PySide/issues/129
However, I also made PySide 1.2.2 work inside a relocatable virtualenv on OS X 10.11 (El Capitan). For this, I modified the localize_libpaths() function in the $VIRTUALENV/bin/pyside_postinstall.py script.
Patch details: If a virtualenv is detected when the script is executed, it will not use #rpath. Instead it will use #executable_path and search relatively from the python binary in the virtualenv. Since the PySide module is always installed in the same place within a virtualenv, I thought this should work in all scenarios.
def localize_libpaths(libpath, local_libs, enc_path=None):
""" Set rpaths and install names to load local dynamic libs at run time
Use ``install_name_tool`` to set relative install names in `libpath` (as
named in `local_libs` to be relative to `enc_path`. The default for
`enc_path` is the directory containing `libpath`.
Parameters
----------
libpath : str
path to library for which to set install names and rpaths
local_libs : sequence of str
library (install) names that should be considered relative paths
enc_path : str, optional
path that does or will contain the `libpath` library, and to which the
`local_libs` are relative. Defaults to current directory containing
`libpath`.
"""
if enc_path is None:
enc_path = abspath(dirname(libpath))
install_names = osx_get_install_names(libpath)
need_rpath = False
for install_name in install_names:
if install_name[0] in '/#':
continue
if hasattr(sys, 'real_prefix'):
# virtualenv detected - use #executable_path
back_tick('install_name_tool -change %s #executable_path/../lib/python2.7/site-packages/PySide/%s %s' %
(install_name, install_name, libpath))
else:
# not a virtualenv - use #rpath
back_tick('install_name_tool -change %s #rpath/%s %s' %
(install_name, install_name, libpath))
need_rpath = True
if need_rpath and enc_path not in osx_get_rpaths(libpath):
back_tick('install_name_tool -add_rpath %s %s' %
(enc_path, libpath))
On Windows, there are no issues with relocating a virtualenv with PySide in it. However, For Linux I have not managed to find a solution. I guess waiting for PySide 1.2.3 is the way to go here.
Related
I have written a python script which depends on paramiko to work. The system that will run my script has the following limitations:
No internet connectivity (so it can't download dependencies on the fly).
Volumes are mounted with 'noexec' (so I cannot run my code as a binary file generated with something like 'pyInstaller')
End-user cannot be expected to install any dependencies.
Vanilla python is installed (without paramiko)
Python version is 2.7.5
Pip is not installed either and cannot be installed on the box
I however, have access to pip on my development box (if that helps in any way).
So, what is the way to deploy my script so that I am able to provide it to the end-user with the required dependencies, i.e paramiko (and sub-dependencies of paramiko), so that the user is able to run the script out-of-the-box?
I have already tried the pyinstaller 'one folder' approach but then faced the 'noexec' issue. I have also tried to directly copy paramiko (and sub-dependencies of paramiko) to a place from where my script is able to find it with no success.
pip is usually installed with python installation. You could use it to install the dependencies on the machine as follows:
import os, sys
def selfInstallParamiko():
# assuming paramiko tar-ball/wheel is under the current working directory
currentDir = os.path.abspath(os.path.dirname(__file__))
# paramikoFileName: name of the already downloaded paramiko tar-ball,
# which you'll have to ship with your scripts
paramikoPath = os.path.join(currentDir, paramikoFileName)
print("\nInstalling {} ...".format(paramikoPath))
# To check for which pip to use (pip for python2, pip3 for python3)
pipName = "pip"
if sys.version_info[0] >= 3:
pipName = "pip3"
p = subprocess.Popen("{} install --user {}".format(pipName, paramikoPath).split())
out, err= p.communicate("")
if err or p.returncode != 0:
print("Unable to self-install {}\n".format(paramikoFileName))
sys.exit(1)
# Needed, because sometimes windows command shell does not pick up changes so good
print("\n\nPython paramiko module installed successfully.\n"
"Please start the script again from a new command shell\n\n")
sys.exit()
You can invoke it when your script starts and an ImportError occurs:
# Try to self install on import failure:
try:
import paramiko
except ImportError:
selfInstallParamiko()
when I run mitmproxy command in command line, I get the following error.
% mitmproxy
Traceback (most recent call last):
File "/usr/local/bin/mitmproxy", line 7, in <module>
from libmproxy.main import mitmproxy
File "/usr/local/lib/python3.5/site-packages/libmproxy/main.py", line 5, in <module>
import thread
ImportError: No module named 'thread'
I googled this error and found this stackoverflow Q&A page.
pydev importerror: no module named thread, debugging no longer works after pydev upgrade
according to the page above, the error occurs because module "thread" is renamed to "_thread" in python3.
So, I know what's causing this error, but then what?
I don't know what to do now in order to get rid of this error.
I'm new to python. I've just installed Python and pip into my mac OSX as shown below because I want to use mitmproxy.
% which pip
/usr/local/bin/pip
% pip --version
pip 8.1.1 from /usr/local/lib/python3.5/site-packages (python 3.5)
% which python
/usr/bin/python
% which python3
/usr/local/bin/python3
% python --version
Python 2.7.10
% python3 --version
Python 3.5.1
could anyone please tell me what to do now?
Additional Info
As #linusg answered, I created "thread.py" file in "site-packages" directory and pasted the code below in "thread.py"
from _thread import *
__all__ = ("error", "LockType", "start_new_thread", "interrupt_main", "exit", "allocate_lock", "get_ident", "stack_size", "acquire", "release", "locked")
After I did this, "ImportError: No module named 'thread'" disappeared, but now I have another ImportError, which is "import Cookie ImportError: No module named 'Cookie'".
It seems that in Python 3, Cookie module is renamed to http.cookies (stackoverflow.com/questions/3522029/django-mod-python-error).
Now what am I supposed to do?
What I have in "site-packages" directory
% ls /usr/local/lib/python3.5/site-packages (git)-[master]
ConfigArgParse-0.10.0.dist-info/ mitmproxy-0.15.dist-info/
OpenSSL/ netlib/
PIL/ netlib-0.15.1.dist-info/
Pillow-3.0.0.dist-info/ passlib/
PyYAML-3.11.dist-info/ passlib-1.6.5.dist-info/
__pycache__/ pathtools/
_cffi_backend.cpython-35m-darwin.so* pathtools-0.1.2.dist-info/
_markerlib/ pip/
_watchdog_fsevents.cpython-35m-darwin.so* pip-8.1.1.dist-info/
argh/ pkg_resources/
argh-0.26.1.dist-info/ pyOpenSSL-0.15.1.dist-info/
backports/ pyasn1/
backports.ssl_match_hostname-3.5.0.1.dist-info/ pyasn1-0.1.9.dist-info/
blinker/ pycparser/
blinker-1.4.dist-info/ pycparser-2.14.dist-info/
certifi/ pyparsing-2.0.7.dist-info/
certifi-2016.2.28.dist-info/ pyparsing.py
cffi/ pyperclip/
cffi-1.6.0.dist-info/ pyperclip-1.5.27.dist-info/
click/ setuptools/
click-6.2.dist-info/ setuptools-19.4-py3.5.egg-info/
configargparse.py sitecustomize.py
construct/ six-1.10.0.dist-info/
construct-2.5.2.dist-info/ six.py
cryptography/ test/
cryptography-1.1.2.dist-info/ thread.py
easy_install.py tornado/
hpack/ tornado-4.3.dist-info/
hpack-2.0.1.dist-info/ urwid/
html2text/ urwid-1.3.1.dist-info/
html2text-2015.11.4.dist-info/ watchdog/
idna/ watchdog-0.8.3.dist-info/
idna-2.1.dist-info/ wheel/
libmproxy/ wheel-0.26.0-py3.5.egg-info/
lxml/ yaml/
lxml-3.4.4.dist-info/
In Python 3 instead of:
import thread
Do:
import _thread
You are trying to run Python 2 code on Python 3, which will not work.
As of April 2016, mitmproxy only supports Python 2.7. We're actively working to fix that in the next months, but for now you need to use Python 2 or the binaries provided at http://mitmproxy.org.
As of August 2016, the development version of mitmproxy now supports Python 3.5+. The next release (0.18) will be the first one including support for Python 3.5+.
As of January 2017, mitmproxy only supports Python 3.5+.
Go to you site-packages folder, create a file called thread.py and paste this code in it:
from _thread import *
__all__ = ("error", "LockType", "start_new_thread", "interrupt_main", "exit", "allocate_lock", "get_ident", "stack_size", "acquire", "release", "locked")
This creates an 'alias' for the module _thread called thread. While the _thread module is very small, you can use dir() for bigger modules:
# Examle for the Cookies module which was renamed to http.cookies:
# Cookies.py in site-packages
import http.cookies
__all__ = tuple(dir(http.cookies))
Hope this helps!
Easiest solution is to create a virtualenv with python2 and run mitmproxy on this virtualenv
virtualenv -p `which python2` .env
source .env/bin/activate
pip install mitmproxy
.env/bin/mitmproxy
The name of the file saved could be threading, this would give an error as threading is a predefined class in Python. Try changing the name of your file. It would help....
I am trying to install pdfMiner to work with CollectiveAccess. My host (pair.com) has given me the following information to help in this quest:
When compiling, it will likely be necessary to instruct the
installation to use your account space above, and not try to install
into the operating system directories. Typically, using "--
home=/usr/home/username/pdfminer" at the end of the install command
should allow for that.
I followed this instruction when trying to install.
The result was:
running install
running build
running build_py
running build_scripts
running install_lib
running install_scripts
changing mode of /usr/home/username/pdfminer/bin/latin2ascii.py to 755
changing mode of /usr/home/username/pdfminer/bin/pdf2txt.py to 755
changing mode of /usr/home/username/pdfminer/bin/dumppdf.py to 755
running install_egg_info
Removing /usr/home/username/pdfminer/lib/python/pdfminer-20140328.egg-info
Writing /usr/home/username/pdfminer/lib/python/pdfminer-20140328.egg-info
I don't see anything wrong with that (I'm very new to python), but when I try to run the sample command $ pdf2txt.py samples/simple1.pdf I get this error:
Traceback (most recent call last): File "pdf2txt.py", line 3, in <module>
from pdfminer.pdfdocument import PDFDocument ImportError: No module named pdfminer.pdfdocument
I'm running python 2.7.3. I can't install from root (shared hosting). The most recent version of pdfminer, which is 2014/03/28.
I've seen some posts on similar issues ("no module named. . . " but nothing exactly the same. The proposed solutions either don't help (such as installing with sudo - not an option; specifying the path for python (which doesn't seem to be the issue), etc.).
Or is this a question for my host? (i.e., something amiss or different about their setup)
I had an error like this:
No module named 'pdfminer.pdfinterp'; 'pdfminer' is not a package
My problem was that I had named my script pdfminer.py which for the reasons that I don't know, Python took it for the original pdfminer package files and tried to compiled it.
I renamed my script to something else, deleted all the *.pyc file and __pycache__ directory and my problem was solved.
use this command worked for me and removed the error
pip install pdfminer.six
Since the package pdfminer is installed to a non-standard/non-default location, Python won't be be able to find it. In order to use it, you will need to add it to your 'pythonpath'. Three ways:
At run time, put this in your script pdf2txt.py:
import sys
# if there are no conflicting packages in the default Python Libs =>
sys.path.append("/usr/home/username/pdfminer")
or
import sys
# to always use your package lib before the system's =>
sys.path.insert(1, "/usr/home/username/pdfminer")
Note: The install path specified with --home is used as the Lib for all packages which you might want to install, not just this one. You should delete that folder and re-install with --
home=/usr/home/username/myPyLibs (or any generic name) so that when you install other packages with that install path, you would only need the one path to add to your local Lib to be able to import them:
import sys
sys.path.insert(1, "/usr/home/username/myPyLibs")
Add it to PYTHONPATH before executing your script:
export PYTHONPATH="${PYTHONPATH}:/usr/home/username/myPyLibs"
And then put that in your ~/.bashrc file (/usr/home/username/.bashrc) or .profile as applicable. This may not work for programs which are not executed from the console.
Create a VirtualEnv and install the packages you need to that.
I have a virtual environment and I had to activate it before I did a pip3 install to have the venv see it.
source ~/venv/bin/activate
I'm working on OS X Mavericks and want to use the NodeBox modules in Python scripts.
The post about how to install the modules for console is from 2009 and doesn't work anymore as this refers to version 1.9.x (current is 3.0.40). Also the SVN source isn't there anymore. The sources are available at GitHub.
By cloning the project and running:
ant run
all I get is a build of the desktop version.
How do I properly install and run the up to date NodeBox modules in Python scripts?
As said in the docs here in section 2. Installing the NodeBox module:
If you want to use NodeBox from the command line, you will have to install it. We currently recommend using Subversion to grab a copy:
svn co http://dev.nodebox.net/svn/nodebox/trunk/ nodebox
...
cd src
python setup.py install
we should be installing the usual way from the source, but as you say the procedure is rather outdated. The source apparently moved from SVN to GitHub at https://github.com/nodebox/nodebox-pyobjc as mentioned on the download page and the source package structure changed too.
Let's grab the source and try to install it:
$ git clone https://github.com/nodebox/nodebox-pyobjc.git
$ cd nodebox-pyobjc
$ python nodebox/setup.py install
Traceback (most recent call last):
File "nodebox/setup.py", line 17, in <module>
import nodebox
ImportError: No module named nodebox
So setup.py needs to import the nodebox package, let's add the project root dir to Python path, so that the nodebox package can be found and try again:
$ export PYTHONPATH=$PYTHONPATH:.
$ python nodebox/setup.py install
...
clang: error: no such file or directory: 'nodebox/ext/cGeo.c'
clang: error: no input files
error: command 'clang' failed with exit status 1
Now it turns out some lib paths in setup.py are wrong, no one probably used this for some time while the libs moved around, but we can fix it:
# ext_modules = [
# Extension('cGeo', ['nodebox/ext/cGeo.c']),
# Extension('cPathmatics', ['nodebox/ext/cPathmatics.c']),
# Extension('cPolymagic', ['nodebox/ext/gpc.c', 'nodebox/ext/cPolymagic.m'], extra_link_args=['-framework', 'AppKit', '-framework', 'Foundation'])
# ]
ext_modules = [
Extension('cGeo', ['libs/cGeo/cGeo.c']),
Extension('cPathmatics', ['libs/pathmatics/pathmatics.c']),
Extension('cPolymagic', ['libs/polymagic/gpc.c', 'libs/polymagic/polymagic.m'], extra_link_args=['-framework', 'AppKit', '-framework', 'Foundation'])
]
Try install again:
$ python nodebox/setup.py install
...
running install_egg_info
Writing <python>/lib/python2.7/site-packages/NodeBox-1.9.7rc2-py2.7.egg-info
$ pip list
...
NodeBox (1.9.7rc2)
...
Now the package installed successfully and we should be able to use it:
$ python
>>> import nodebox
>>> dir(nodebox)
['__builtins__', '__doc__', '__file__', '__name__', '__package__', '__path__', '__version__', 'get_version']
>>> nodebox.__version__
'1.9.7rc2'
Also, you may still need to manually install some of the dependencies for everything to work correctly, as noted in setup.py itself:
# We require some dependencies:
# - PyObjC
# - psyco
# - py2app
# - cPathMatics (included in the "libs" folder)
# - polymagic (included in the "libs" folder)
# - Numeric (included in the "libs" folder)
# - Numpy (installable using "easy_install numpy")
I already created a pull request with fixed setup.py lib paths, see here.
Tested on OS X Mavericks (System Version: OS X 10.9.3 (13D65), Kernel Version: Darwin 13.2.0) using Homebrew Python 2.7.6.
This question already has answers here:
How can I Install a Python module within code?
(12 answers)
Closed 2 years ago.
Can I download and install Python modules from PyPi strictly inside a script, without using a shell at all?
I use a non-standard Python environment, Autodesk Maya's Python interpreter. This does not come with "easy_install," and there is no "shell," only a python script interpreter invoked by the main Maya executable. Copying and pasting ez_setup.py's contents into the script editor window and running it correctly installs an easy_install somewhere into Maya's directory, but the script incorrectly records the Python interpreter as "...maya.exe" instead of "...mayapy.exe" Furthermore, using easy_install requires a shell.
The objective is to deliver a Python script that, e.g., installs NumPy into the Maya Python system. This could be accomplished by dropping eggs into the site-packages directory, but that requires manual user intervention. Anything an end user has to do outside the Maya environment is essentially untouchable, especially messing with the file system. But messing with the filesystem through a script? That's fine.
Is there something more elegant than ez_setup.py + editing the resulting easy_install...py's + subprocess calls? I feel like this is a basic feature. I see documentation online for programmatic module installation through pip... but pip needs to be installed first!
What is the most elegant way to install a module strictly within the confines of a script?
Installing easy_install for Maya on windows.
Download ez_setup.py.
open windows cmd elevated (start, type cmd, rmb click on it ->run as administrator)
change the cmd directory to x:\maya install dir\bin
example: cd c:\Program Files\MayaXX\bin
execute following command mayapy x:\WhereYouSaved\ez_setup.py
Now easy install should be set up properly. You may want to still do following steps:
cd x:\maya install dir\python\scripts
rename all files in this folder to start with ma
example: for %i in (*) do ren %i ma%i
add this folder to your path
hit win+e
rmb my computer and choose properties
Advanced system settings -> Environment variables
search variable path edit it and append ;x:\maya install dir\python\scripts
Now you can call maeasy_install pythonModule from cmd for installing stuff. Also you can call following inside Maya to install modules:
from setuptools.command import easy_install
easy_install.main( ["pythonModule"] )
NOTE: If Maya is installed in program files then you can not really install stuff without elevating. Unless you change disk permissions to the Maya python directory.
#!/usr/bin/env python
from __future__ import print_function
REQUIREMENTS = [ 'distribute', 'version', 'Cython', 'sortedcollection' ]
try:
from setuptools import find_packages
from distutils.core import setup
from Cython.Distutils import build_ext as cython_build
import sortedcollection
except:
import os, pip
pip_args = [ '-vvv' ]
proxy = os.environ['http_proxy']
if proxy:
pip_args.append('--proxy')
pip_args.append(proxy)
pip_args.append('install')
for req in REQUIREMENTS:
pip_args.append( req )
print('Installing requirements: ' + str(REQUIREMENTS))
pip.main(initial_args = pip_args)
# do it again
from setuptools import find_packages
from distutils.core import setup
from Cython.Distutils import build_ext as cython_build
import sortedcollection
To make it work, open the ez_setup.py file and simply add an s after http at this line:
DEFAULT_URL = "http://pypi.python.org/packages/%s/s/setuptools/" % sys.version[:3]
so that it becomes
DEFAULT_URL = "https://pypi.python.org/packages/%s/s/setuptools/" % sys.version[:3]