This question already has answers here:
How can I Install a Python module within code?
(12 answers)
Closed 2 years ago.
Can I download and install Python modules from PyPi strictly inside a script, without using a shell at all?
I use a non-standard Python environment, Autodesk Maya's Python interpreter. This does not come with "easy_install," and there is no "shell," only a python script interpreter invoked by the main Maya executable. Copying and pasting ez_setup.py's contents into the script editor window and running it correctly installs an easy_install somewhere into Maya's directory, but the script incorrectly records the Python interpreter as "...maya.exe" instead of "...mayapy.exe" Furthermore, using easy_install requires a shell.
The objective is to deliver a Python script that, e.g., installs NumPy into the Maya Python system. This could be accomplished by dropping eggs into the site-packages directory, but that requires manual user intervention. Anything an end user has to do outside the Maya environment is essentially untouchable, especially messing with the file system. But messing with the filesystem through a script? That's fine.
Is there something more elegant than ez_setup.py + editing the resulting easy_install...py's + subprocess calls? I feel like this is a basic feature. I see documentation online for programmatic module installation through pip... but pip needs to be installed first!
What is the most elegant way to install a module strictly within the confines of a script?
Installing easy_install for Maya on windows.
Download ez_setup.py.
open windows cmd elevated (start, type cmd, rmb click on it ->run as administrator)
change the cmd directory to x:\maya install dir\bin
example: cd c:\Program Files\MayaXX\bin
execute following command mayapy x:\WhereYouSaved\ez_setup.py
Now easy install should be set up properly. You may want to still do following steps:
cd x:\maya install dir\python\scripts
rename all files in this folder to start with ma
example: for %i in (*) do ren %i ma%i
add this folder to your path
hit win+e
rmb my computer and choose properties
Advanced system settings -> Environment variables
search variable path edit it and append ;x:\maya install dir\python\scripts
Now you can call maeasy_install pythonModule from cmd for installing stuff. Also you can call following inside Maya to install modules:
from setuptools.command import easy_install
easy_install.main( ["pythonModule"] )
NOTE: If Maya is installed in program files then you can not really install stuff without elevating. Unless you change disk permissions to the Maya python directory.
#!/usr/bin/env python
from __future__ import print_function
REQUIREMENTS = [ 'distribute', 'version', 'Cython', 'sortedcollection' ]
try:
from setuptools import find_packages
from distutils.core import setup
from Cython.Distutils import build_ext as cython_build
import sortedcollection
except:
import os, pip
pip_args = [ '-vvv' ]
proxy = os.environ['http_proxy']
if proxy:
pip_args.append('--proxy')
pip_args.append(proxy)
pip_args.append('install')
for req in REQUIREMENTS:
pip_args.append( req )
print('Installing requirements: ' + str(REQUIREMENTS))
pip.main(initial_args = pip_args)
# do it again
from setuptools import find_packages
from distutils.core import setup
from Cython.Distutils import build_ext as cython_build
import sortedcollection
To make it work, open the ez_setup.py file and simply add an s after http at this line:
DEFAULT_URL = "http://pypi.python.org/packages/%s/s/setuptools/" % sys.version[:3]
so that it becomes
DEFAULT_URL = "https://pypi.python.org/packages/%s/s/setuptools/" % sys.version[:3]
Related
Given a package random.whl containing hello.py:
print("Hello World!")
Is there a way to create a setup.py, setup.cfg or pyproject.toml, that when executed, will install the package in such a way that hello.py will be executed every time Python is started?
pip install random.whl
python unrelated.py # First prints "Hello World", then continues on.
I know it's possible to hook on readline.py that Python automatically loads, but is there a different and less "hacky" way to achieve it?
Some impossible ways that I thought of:
Running a post-install script on a .whl distribution (post-install is only avaiable on sdist).
Modifying PYTHONSTARTUP env variable or copying files.
Changing the import machinery.
While being a security risk, a method achieving it is good for implementing debuggers or auditing tools without requiring a change in either pre-compiled or post-compiled Python code, or used for penetration testing in side-channel attacks.
So far, using sitecustomize.py and publishing an sdist with a custom install command was the most reliable, and worked in virtual environments unlike usercustomize.py or .pth files.
Relevant setup.py code:
import sys
import os
import setuptools
import sysconfig
from setuptools.command.install import install
class PreInstall(install):
def run(self):
site_packages_dir = sysconfig.get_path("purelib")
sitecustomize_path = os.path.join(site_packages_dir, "sitecustomize.py")
if os.path.exists(sitecustomize_path):
raise FileExistsError("Site customize file already exists. "
"Please remove it before installing.")
install.run(self)
setuptools.setup(
name='...',
version='0.0.1',
py_modules=["sitecustomize"],
cmdclass={'install': PreInstall,},
)
It still doesn't work with .whl distributions as it might overwrite an existing sitecustomize without being able to check.
This question already has answers here:
Getting Python error "from: can't read /var/mail/Bio"
(6 answers)
Closed 8 months ago.
I'm trying to have a Python script available on a user's path when they install my package from PyPI using pip:
pip install MyPackage
MyPackage is on PyPI and installs successfully--apparently--in a conda virtual environment. The setup.py file (excerpted) looks like this:
try:
from setuptools import setup
except ImportError:
from distutils.core import setup
config = {
'name': 'MyPackage',
'version': '0.2.1.dev',
'install_requires': [...],
'packages': [
'MyPackage', 'MyPackage.utils'
],
'py_modules': [
'MyCLI',
],
'scripts': [
'MyPackage/MyCLI.py',
],
...
On GNU/Linux, when I type MyCLI and hit Tab, it successfully auto-completes to MyCLI.py. When I ask which MyCLI.py it shows me the fully qualified path to Python script in the virtual environment folder:
$ which MyCLI.py
/home/arthur/Applications/miniconda3/envs/MyPackage/bin/MyCLI.py
MyCLI.py uses fire to wrap a Python class, expose its methods at the command line, present docstrings as help documentation, and parse arguments. It looks like:
'''
My Command Line Interface
'''
class CLIRuntime(object):
def run(self):
do_something()
if __name__ == '__main__':
import fire
fire.Fire(CLIRuntime)
If I run this script with the Python interpreter, it executes correctly.
python $(which MyCLI.py)
The problem is, when I try to run it without specifying the Python interpreter, it seems to think it is a bash script or binary file and wrecks my terminal session:
$ MyCLI.py
/home/arthur/Applications/miniconda3/envs/MyPackage/bin/MyCLI.py: line 8:
My Command Line Interface
: No such file or directory
from: can't read /var/mail/__future__
How can I change setup.py so that this script is available on a user's path but also known as/ runs as a Python script?
I want to note that if I install this package from source using pip in editable mode (pip install -e .), MyCLI.py is on my path and runs correctly as a Python script. It just doesn't appear to work when installing from PyPI.
To tell your shell what program should be used to execute a script file, you need to add a "hash-bang" declaration as the first line in the script. For Python executing inside of a virtualenv, either
#!python
or
#!/usr/bin/env python
will do the trick. If you're using Python 3, use python3 instead.
I have written a python script which depends on paramiko to work. The system that will run my script has the following limitations:
No internet connectivity (so it can't download dependencies on the fly).
Volumes are mounted with 'noexec' (so I cannot run my code as a binary file generated with something like 'pyInstaller')
End-user cannot be expected to install any dependencies.
Vanilla python is installed (without paramiko)
Python version is 2.7.5
Pip is not installed either and cannot be installed on the box
I however, have access to pip on my development box (if that helps in any way).
So, what is the way to deploy my script so that I am able to provide it to the end-user with the required dependencies, i.e paramiko (and sub-dependencies of paramiko), so that the user is able to run the script out-of-the-box?
I have already tried the pyinstaller 'one folder' approach but then faced the 'noexec' issue. I have also tried to directly copy paramiko (and sub-dependencies of paramiko) to a place from where my script is able to find it with no success.
pip is usually installed with python installation. You could use it to install the dependencies on the machine as follows:
import os, sys
def selfInstallParamiko():
# assuming paramiko tar-ball/wheel is under the current working directory
currentDir = os.path.abspath(os.path.dirname(__file__))
# paramikoFileName: name of the already downloaded paramiko tar-ball,
# which you'll have to ship with your scripts
paramikoPath = os.path.join(currentDir, paramikoFileName)
print("\nInstalling {} ...".format(paramikoPath))
# To check for which pip to use (pip for python2, pip3 for python3)
pipName = "pip"
if sys.version_info[0] >= 3:
pipName = "pip3"
p = subprocess.Popen("{} install --user {}".format(pipName, paramikoPath).split())
out, err= p.communicate("")
if err or p.returncode != 0:
print("Unable to self-install {}\n".format(paramikoFileName))
sys.exit(1)
# Needed, because sometimes windows command shell does not pick up changes so good
print("\n\nPython paramiko module installed successfully.\n"
"Please start the script again from a new command shell\n\n")
sys.exit()
You can invoke it when your script starts and an ImportError occurs:
# Try to self install on import failure:
try:
import paramiko
except ImportError:
selfInstallParamiko()
This is my project structure. I use virtualenv in my project but when I run it ,it has an ImportError.I use Mac.
But I can run it successfully use Pycharm
So how to run it successfully by Terminal.Because I want to run it in a Ubuntu server with cron
Thanks you for your answers.Here I show my solution.I modify my handler.py I think it may be related to The Module Search Path.
So I add the project path to the PYTHONPATH.
import os
project_home = os.path.realpath(__file__)
project_home = os.path.split(project_home)[0]
import sys
sys.path.append(os.path.split(project_home)[0])
import shutil
from modules import db, json_parse, config_out
from init_log import init as initlog
initlog()
if __name__ == '__main__':
try:
columns = json_parse.json_parse()
if not columns:
sys.exit()
is_table_has_exist = db.check_tables_exist(columns=columns)
if is_table_has_exist:
db.check_columns(columns=columns)
is_ok, config_path = config_out.output(columns)
if is_ok:
file_name = os.path.split(config_path)[1]
shutil.copy(config_path, os.path.join("/app/statics_log/config", file_name))
except Exception, e:
print e
And I run with crontab by this.
cd to/my/py_file/path && /project_path/.env/bin/python /path/to/py_file
example:
13 8 1 * * cd bulu-statics/create_config/ && /home/buka/bulu-statics/.env/bin/python /home/buka/bulu-statics/create_config/handler.py >> /app/statics_log/config/create_config.log
PyCharm automatically adds project directories marked as containing sources to the PYTHONPATH environment variable, whihc is why it works from within pycharm. On the terminal use
PYTHONPATH=${PWD}/..:${PYTHONPATH} python handler.py
You can use explicit relative imports:
from .modules import db, json_parse, config_out
The proper way to do this is to turn your project into a proper Python package by adding a setup.py file and then installing it with pip install -e .
probably because PyCharm added your project folder to the PythonPath, so you can run you app inside PyCharm.
However, when you try to run it from command line, python interpreter cannot find these libs in Python python, so what you need to do is to add your python virtualenv the python python.
there are different ways to adding python path, but I would suggest you to follow:
prepare a setup.py you'll need to specify packages and install_requires.
install your app locally in development mode via pip install -e /path/to/your-package -> it'll create a egg-link in your python virtualenv, you can run your app in your local terminal from now on;
for packing and releasing, you may want to build an artifact by following https://docs.python.org/2.7/distutils/builtdist.html
you may pip install or easy_install the artifact on your other machines. you also can release your package to PyPi if you want.
I am trying to install pdfMiner to work with CollectiveAccess. My host (pair.com) has given me the following information to help in this quest:
When compiling, it will likely be necessary to instruct the
installation to use your account space above, and not try to install
into the operating system directories. Typically, using "--
home=/usr/home/username/pdfminer" at the end of the install command
should allow for that.
I followed this instruction when trying to install.
The result was:
running install
running build
running build_py
running build_scripts
running install_lib
running install_scripts
changing mode of /usr/home/username/pdfminer/bin/latin2ascii.py to 755
changing mode of /usr/home/username/pdfminer/bin/pdf2txt.py to 755
changing mode of /usr/home/username/pdfminer/bin/dumppdf.py to 755
running install_egg_info
Removing /usr/home/username/pdfminer/lib/python/pdfminer-20140328.egg-info
Writing /usr/home/username/pdfminer/lib/python/pdfminer-20140328.egg-info
I don't see anything wrong with that (I'm very new to python), but when I try to run the sample command $ pdf2txt.py samples/simple1.pdf I get this error:
Traceback (most recent call last): File "pdf2txt.py", line 3, in <module>
from pdfminer.pdfdocument import PDFDocument ImportError: No module named pdfminer.pdfdocument
I'm running python 2.7.3. I can't install from root (shared hosting). The most recent version of pdfminer, which is 2014/03/28.
I've seen some posts on similar issues ("no module named. . . " but nothing exactly the same. The proposed solutions either don't help (such as installing with sudo - not an option; specifying the path for python (which doesn't seem to be the issue), etc.).
Or is this a question for my host? (i.e., something amiss or different about their setup)
I had an error like this:
No module named 'pdfminer.pdfinterp'; 'pdfminer' is not a package
My problem was that I had named my script pdfminer.py which for the reasons that I don't know, Python took it for the original pdfminer package files and tried to compiled it.
I renamed my script to something else, deleted all the *.pyc file and __pycache__ directory and my problem was solved.
use this command worked for me and removed the error
pip install pdfminer.six
Since the package pdfminer is installed to a non-standard/non-default location, Python won't be be able to find it. In order to use it, you will need to add it to your 'pythonpath'. Three ways:
At run time, put this in your script pdf2txt.py:
import sys
# if there are no conflicting packages in the default Python Libs =>
sys.path.append("/usr/home/username/pdfminer")
or
import sys
# to always use your package lib before the system's =>
sys.path.insert(1, "/usr/home/username/pdfminer")
Note: The install path specified with --home is used as the Lib for all packages which you might want to install, not just this one. You should delete that folder and re-install with --
home=/usr/home/username/myPyLibs (or any generic name) so that when you install other packages with that install path, you would only need the one path to add to your local Lib to be able to import them:
import sys
sys.path.insert(1, "/usr/home/username/myPyLibs")
Add it to PYTHONPATH before executing your script:
export PYTHONPATH="${PYTHONPATH}:/usr/home/username/myPyLibs"
And then put that in your ~/.bashrc file (/usr/home/username/.bashrc) or .profile as applicable. This may not work for programs which are not executed from the console.
Create a VirtualEnv and install the packages you need to that.
I have a virtual environment and I had to activate it before I did a pip3 install to have the venv see it.
source ~/venv/bin/activate