Hello internet strangers,
I need to do XSD verification with lxml, but do not have sudo abilities on the machine I'm using - so pip is not an option. I'm on Fedora 27 and found the source code for lxml, which requires:
sudo apt-get install libxml2-dev libxslt-dev python-dev
But I cant use sudo on the machine I'm deploying to. Once I have those dependencies all I need to do is get the lxml source code through wget or github, and then run setup.py install, but I cant do that if I cant install the above dependencies.
Help?
Useful links:
https://gist.github.com/blite/868292
https://github.com/lxml/lxml
I'd look into creating a virtual environment and installing dependencies there. Here are the docs: https://docs.python.org/3.6/library/venv.html
Basically you create a virtual environment like this:
$ python -m venv my_venv_name
That creates a folder call my_venv_name with a few folders under it (do this somewhere in your home directory). The issue the command
$ source my_venv_name/bin/activate
That activates your environment. At this point you can pip install into your virtual environment to your hearts content. You just need to remember to activate the environment for each session that you want to use it in.
Related
I do not have root previlege on a linux server so I want to creat a virtual python according to creating a "virtual" python.
After I run virtual-python.py, I do have python in ~/bin/python:
Then, according to setuptools PyPI page, I download ez_setup.py and run ~/bin/python ez_setup.py. Error occurs:
What should I do?
Looking at the linked website, it looks outdated. You use pip, not easy_install.
For installing development packages, I always take the following rules in account:
The system package manager is responsible for system-wide packages, so never use sudo pip. This doesn't just match the question, but this is always a good idea.
The package manager packages are probably outdated. You'll want an up-to-date version for development tools.
I recommend the following way to install local development tools.
$ # Install pip and setuptools on a user level
$ curl https://bootstrap.pypa.io/get-pip.py | python - --user
$ # Add the executables to your path. Add this to your `.bashrc` or `.profile` as well
$ export PATH=$PATH/$HOME/.local/bin
At this point pip should be accessible from the command line and usable without sudo. Use this to install virtualenv, the most widely used tools to set up virtual environments.
$ pip install virtualenv --user
Now simply use virtualenv to set up an environment to run your application in:
$ virtualenv myapp
Now activate the virtual environment and do whatever you would like to do with it. Note that after activating the virtual environment, pip refers to pip installed inside of the virtualenv, not the one installed on a user level.
$ source myapp/bin/activate
(myapp)$ pip install -r requirements.txt # This is just an example
You'll want to create a new virtual environment for each application you run on the server, so the dependencies can't conflict.
I'm using a virtualenv to run Python 2.7 in my local machine and everything works as expected. When I transfer "site-packages" to my production sever, I get the follow error:
PIL/_imaging.so: invalid ELF header
This happens on the Pillow 2.5.3 pypi package found here
I am running OS X, while my production server is running Debian. I suspect the OS differences might be causing the issue but I'm not sure. I have no idea how to fix this. Can anyone help?
Note: I cannot install packages directly to my production server, so I have to upload them directly to use them.
In your current virtual environment, execute the following command
pip freeze > requirements.txt
Copy this requirements.txt file to your server.
Create your new virtualenvironment (delete the one you were using before).
Activate the virtual environment and then type pip install -r requirements.txt
Now, the libraries will be installed correctly and built accurately as well.
If you see errors for PIL, execute the following commands:
sudo apt-get install build-essential python-dev
sudo apt-get build-dep python-imaging
virtual environments are for isolating Python on your current machine; they are not for creating portable environments. The benefit is to work with different versions of Python packages without modifying the system Python installation.
Using virtual environments does not require super-user permissions; so you can install packages even if you are not "root".
It does, however, require Internet access as packages are downloaded from the web. If your server does not have access to the Internet, back on your mac, do the following, from your virtual environment:
pip install basket
This will install basket which is a small utility that allows you to download packages but not install them. Great for keeping a local archive of packages that you can move to other machines.
Once its installed, follow these steps as listed in the documentation:
basket init
pip freeze > requirements.txt
awk -F'==' '{print $1}' requirements.txt | basket download
This will download all the packages from your requirements.txt file into ~/.basket
Next, copy this directory to your server and then run the following command from your virtualenvironment
pip install --no-index -f /path/to/basket -r requirements.txt
It it's possible, of course.
For example - I can download python-dbus like this:
$ sudo apt-get download python-dbus
But what I should to do next, with this .deb package in my current virtualenv?
If you really need to do it this way, you can just copy the files that get installed globally directly into your virtualenv. For example I couldn't get pycurl working since the required libraries weren't installing, but apt-get install python-pycurl did. So I did the following:
sudo apt-get install python-pycurl
cp /usr/lib/python2.7/dist-packages/pycurl* ~/.virtualenvs/myenv/lib/python2.7/site-packages/
The install said it was adding it to /usr/lib/python2.7. So I looked in that directory for a site-packages or dist-packages with pycurl, after looking at the files I copied them into my virtualenv. You'd have to also copy any executables from bin into your virtualenv's bin directory.
Also, running a pip install -r requirements.txt successfully found pycurl in there and just skipped over it as if I had installed it via pip.
To include system site packages in your existing virtual environment open the config file:
<PATH_TO_YOUR_VENV_FOLDER>/pyvenv.cfg
and change false to true for include-system-site-packages
include-system-site-packages = true
Save and reload your virtual environment.
(tested with virtualenv 20.2.2 on Raspbian GNU/Linux 10 (buster) to pull in python3-pyqt5 installed with apt into my virtual environment)
If it is for a new environment #Joshua Kan's answer using the --system-site-packages flag with the venv command is probably what you want.
Why would you want to do this? The whole point is to avoid doing stuff like that...
virtualenv whatever
cd whatever
bin/pip install dbus-python
You may also choose to specify --no-site-packages to virtualenv to keep it extra isolated.
An alternative solution is to install globally, then followed by allowing the virtualenv to be able to see it.
As an example, let's say we want to install matplotlib for Python 3:
sudo apt update # Update first
sudo apt install python3-matplotlib # Install globally
sudo pip3 install -U virtualenv # Install virtualenv for Python 3 using pip3
virtualenv --system-site-packages -p python3 ./venv #the system-site-packages option allows venv to see all global packages including matplotlib
source ./venv/bin/activate #activate the venv to use matplotlib within the virtualenv
deactivate # don't exit until you're done using the virtualenv
First install the dbus development libraries (you may need some other dev libraries, but this is all I needed)
sudo apt-get install libdbus-1-dev libdbus-glib-1-dev
Next, with your virtualenv activated, run the following. It'll fail but that's ok.
pip install dbus-python
Finally, go into your virtualenv's build directory and install it the non-pythonic way.
cd $VIRTUAL_ENV/build/dbus-python
chmod +x configure
./configure --prefix=$VIRTUAL_ENV
make
make install
I have easy_install and pip.
I had many errors on my Linux Mint 12, I just re-installed it and I want to install everything from scratch again.
This is one of the errors that I had. I received an interesting answer there:
Stop using su and sudo to run virtualenv.
You need to run virtualenv as your normal user.
You have created the virtualenv with sudo which is why you are getting these errors.
So how to install virtualenv without using sudo? Can i use pipor easy_install without using sudo? Or is there another way?
This solution is suitable in cases where no virtualenv is available system wide and you can not become root to install virtualenv. When I set up a debian for python development or deployment I always apt-get install python-virtualenv. It is more convenient to have it around than to do the bootstrap pointed out below. But without root power it may be the the way to go:
There is a bootstrap mechanism that should get you going.
Read: http://peak.telecommunity.com/DevCenter/EasyInstall#creating-a-virtual-python
In essence you would do this in your home directory in a unix environment:
Given your python is version 2.6
$ mkdir ~/bin
$ mkdir -p ~/lib/python2.6
$ mkdir -p ~/local/lib/python2.6/dist-packages
$ wget http://peak.telecommunity.com/dist/virtual-python.py
$ python virtual-python.py --no-site-packages
$ wget http://peak.telecommunity.com/dist/ez_setup.py
$ ~/bin/python ez_setup.py
$ ~/local/bin/easy_install virtualenv
$ ~/local/bin/virtualenv --no-site-packages thereyouare
There may be room for optimization. I don't like the local path. Just bin and lib would be nice. But it does its job.
You can also use the command below, it worked for me without sudo access.
You may also need to modify your PYTHONPATH environment variable using export, see this SO answer for more details.
pip install --user virtualenv
The general idea is to install virtualenv itself globaly, i.e. sudo easy_install virtualenv or sudo pip install virtualenv, but then create the actual virtual environment ("run virtualenv") locally.
http://opensourcehacker.com/2012/09/16/recommended-way-for-sudo-free-installation-of-python-software-with-virtualenv/ suggests the following:
curl -L -o virtualenv.py https://raw.githubusercontent.com/pypa/virtualenv/master/virtualenv.py
python virtualenv.py vvv-venv
. vvv-venv/bin/activate
pip install vvv
It seems to work well. It lets me install https://github.com/miohtama/vvv with pip.
If you get:
Cannot find sdist setuptools-*.tar.gz
Cannot find sdist pip-*.tar.gz
Try --extra-search-dir after downloading the tarballs at https://github.com/pypa/virtualenv/tree/develop/virtualenv_support
This worked for me:
pip install --target=$HOME/virtualenv/ virtualenv
cd somewhere/
python $HOME/virtualenv/virtualenv.py env
. env/bin/activate
Now I can pip install whatever I want (except for everything that needs to compile stuff with gcc and has missing dependencies such as the python development libraries and Python.h).
Basically the idea is to install virtualenv (or any other python package) into ${HOME}/.local. This is the most appropriate location since it is included into python path by default (and not only Python).
That you do by pip3 install virtualenv --prefix=${HOME}/.local (you may need to expand ${HOME}).
Make sure that you have export PATH=${HOME}/.local/bin:${PATH} in your ~/.profile (you may need to source ~/.profile it if just added)
I've created a "portable" version of virtualenv.
wget https://bitbucket.org/techtonik/locally/raw/tip/06.get-virtualenv.py
python 06.get-virtualenv.py
It downloads virtualenv.py script with dependencies into .locally subdir and executes it from there. Once that's done, the script with .locally/ subdir can be copied anywhere.
I solved my problem installing virtualenv for each user.
python3 -m pip install --user virtualenv
You might want to consider using Anaconda. It's a full-fledged Python distribution, that lives in a folder in e.g. your home directory. No sudo is necessary at any point and you get most of the popular packages.
$ wget https://.../Anaconda2-2.5.0-Linux-x86_64.sh # check the website for the exact URL, it can change
$ bash Anaconda2-2.5.0-Linux-x86_64.sh
$ conda install virtualenv
The easiest way I have seen so far is to install Anaconda.
It may be an overkill for you. For me the centOS running on the remote server had only python2.6 installed. Anaconda by default installs everything locally + it is python2.7
curl -O https://repo.continuum.io/archive/Anaconda2-4.2.0-Linux-x86_64.sh
Then
bash Anaconda2-4.2.0-Linux-x86_64.sh
Boom. You have all the packages like numpy and pip installed.
Then if you want virtualenv, just type
pip install virtualenv
sudo virtualenv -p python myenv1
sudo su
source myenv1/bin/activate
pip install mypackage
this is will install inside virtual environment
The lack of sudo is a common situation in many shared remote server.
It turns out, there is a simpler, lightweight, more secure solution. Just download an official "portable" virtualenv from here: https://bootstrap.pypa.io/virtualenv.pyz
And that is it! You can now run python virtualenv.pyz --help to your heart's content.
Official document: https://virtualenv.pypa.io/en/latest/installation.html#via-zipapp
I'm taking some university classes and have been given an 'instructional account', which is a school account I can ssh into to do work. I want to run my computationally intensive Numpy, matplotlib, scipy code on that machine, but I cannot install these modules because I am not a system administrator.
How can I do the installation?
In most situations the best solution is to rely on the so-called "user site" location (see the PEP for details) by running:
pip install --user package_name
Below is a more "manual" way from my original answer, you do not need to read it if the above solution works for you.
With easy_install you can do:
easy_install --prefix=$HOME/local package_name
which will install into
$HOME/local/lib/pythonX.Y/site-packages
(the 'local' folder is a typical name many people use, but of course you may specify any folder you have permissions to write into).
You will need to manually create
$HOME/local/lib/pythonX.Y/site-packages
and add it to your PYTHONPATH environment variable (otherwise easy_install will complain -- btw run the command above once to find the correct value for X.Y).
If you are not using easy_install, look for a prefix option, most install scripts let you specify one.
With pip you can use:
pip install --install-option="--prefix=$HOME/local" package_name
No permissions to access nor install easy_install?
Then, you can create a python virtualenv (https://pypi.python.org/pypi/virtualenv) and install the package from this virtual environment.
Executing 4 commands in the shell will be enough (insert current release like 16.1.0 for X.X.X):
$ curl --location --output virtualenv-X.X.X.tar.gz https://github.com/pypa/virtualenv/tarball/X.X.X
$ tar xvfz virtualenv-X.X.X.tar.gz
$ python pypa-virtualenv-YYYYYY/src/virtualenv.py my_new_env
$ . my_new_env/bin/activate
(my_new_env)$ pip install package_name
Source and more info: https://virtualenv.pypa.io/en/latest/installation/
You can run easy_install to install python packages in your home directory even without root access. There's a standard way to do this using site.USER_BASE which defaults to something like $HOME/.local or $HOME/Library/Python/2.7/bin and is included by default on the PYTHONPATH
To do this, create a .pydistutils.cfg in your home directory:
cat > $HOME/.pydistutils.cfg <<EOF
[install]
user=1
EOF
Now you can run easy_install without root privileges:
easy_install boto
Alternatively, this also lets you run pip without root access:
pip install boto
This works for me.
Source from Wesley Tanaka's blog : http://wtanaka.com/node/8095
If you have to use a distutils setup.py script, there are some commandline options for forcing an installation destination. See http://docs.python.org/install/index.html#alternate-installation. If this problem repeats, you can setup a distutils configuration file, see http://docs.python.org/install/index.html#inst-config-files.
Setting the PYTHONPATH variable is described in tihos post.
Important question. The server I use (Ubuntu 12.04) had easy_install3 but not pip3. This is how I installed Pip and then other packages to my home folder
Asked admin to install Ubuntu package python3-setuptools
Installed pip
Like this:
easy_install3 --prefix=$HOME/.local pip
mkdir -p $HOME/.local/lib/python3.2/site-packages
easy_install3 --prefix=$HOME/.local pip
Add Pip (and other Python apps to path)
Like this:
PATH="$HOME/.local/bin:$PATH"
echo PATH="$HOME/.local/bin:$PATH" > $HOME/.profile
Install Python package
like this
pip3 install --user httpie
# test httpie package
http httpbin.org
The best and easiest way is this command:
pip install --user package_name
http://www.lleess.com/2013/05/how-to-install-python-modules-without.html#.WQrgubyGOnc
I use JuJu which basically allows to have a really tiny linux distribution (containing just the package manager) inside your $HOME/.juju directory.
It allows to have your custom system inside the home directory accessible via proot and, therefore, you can install any packages without root privileges. It will run properly to all the major linux distributions, the only limitation is that JuJu can run on linux kernel with minimum reccomended version 2.6.32.
For instance, after installed JuJu to install pip just type the following:
$>juju -f
(juju)$> pacman -S python-pip
(juju)> pip
Install virtualenv locally (source of instructions):
Important: Insert the current release (like 16.1.0) for X.X.X.
Check the name of the extracted file and insert it for YYYYY.
$ curl -L -o virtualenv.tar.gz https://github.com/pypa/virtualenv/tarball/X.X.X
$ tar xfz virtualenv.tar.gz
$ python pypa-virtualenv-YYYYY/src/virtualenv.py env
Before you can use or install any package you need to source your virtual Python environment env:
$ source env/bin/activate
To install new python packages (like numpy), use:
(env)$ pip install <package>
Install Python package without Admin rights
import sys
!{sys.executable} -m pip install package_name
Example
import sys
!{sys.executable} -m pip install kivy
Reference: https://docs.python.org/3.4/library/sys.html#sys.executable