I'm trying to install python2.7 on linux from source on a system where I don't have admin rights. There is a version installed in /usr/bin but I want my install to be completely independent of this. I'm also testing builds with a newer gcc compiler so I want to avoid prebuilt packages like anaconda.
I downloaded python source and ran the following
./configure --prefix=$INSTALL_DIR --exec-prefix=$INSTALL_DIR --enable-shared --enable-optimizations --enable-unicode=ucs4
make
make install
where $INSTALL_DIR is where I want my install to go.
The configure and build ran without error, however, when I run the built python executable
$INSTALL_DIR/bin/python
and check the search paths it is still using /usr as the prefix:
Python 2.7.13 (default, May 31 2017, 17:58:17)
[GCC 7.1.1 20170503 (Red Hat 7.1.1-1)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.prefix
'/usr'
This means modules I install into $INSTALL_DIR aren't being found. Also running pip install XXX doesn't do anything if the module is already installed in the /usr space.
I'd like to avoid using things like $PYTHONPATH and $LD_LIBRARY_PATH so is there a way to set this up where sys.prefix and sys.path point to $INSTALL_DIR and know nothing about the system level install?
Thanks for your help.
Stop passing flags other than prefix to configre, and your python will give correct sys.path that stats with $INSTALL_DIR.
$ ./configure --prefix=$INSTALL_DIR
Related
I am trying to make a Python script on my Mac (MacOS 10.14.6 Mojave) and I am getting trouble with installing a module (watchdog). I have the built-in Python 2 and I installed Python 3 with Homebrew.
If a type in the terminal python, I get:
Python 2.7.16 (default, Oct 16 2019, 00:34:56)
[GCC 4.2.1 Compatible Apple LLVM 10.0.1 (clang-1001.0.37.14)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
So Python 2 correctly uses GCC (the one provided by Apple? I am not sure) instead of Clang. Now, if I do python -m pip install watchdog it just works. But I want to install the module on Python 3, that for some reason uses Clang instead of GCC. In fact, if I type `python3', I get:
Python 3.8.0 (v3.8.0:fa919fdf25, Oct 14 2019, 10:23:27)
[Clang 6.0 (clang-600.0.57)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
I assume that as a consequence python3 -m pip install watchdog gives a compile error because Python 3 is not using GCC. The error message is very long and can be found here: https://pastebin.com/DEAKANQ9
In my $PATH I have /usr/local/bin (where gcc is installed) before /usr/bin, i.e.
echo $PATH
/Library/Frameworks/Python.framework/Versions/3.8/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Library/TeX/texbin:/opt/X11/bin
I would say everything is set up correctly, but apparently that is not the case. How can I make Python 3 use GCC instead of Clang?
I have fixed the issue.
First, I have removed Python 3.8 which was installed using the pkg installer (by accident actually). Then, I have created an alias for Python 3 in the .bash_profile. Another mistake was that I created the file ~/.bashrc (as in Ubuntu) instead of editing the file ~/.bash_profile to modify the $PATH.
# ~/.bash_profile
#
# Python alias
alias python=/usr/local/bin/python3
# Setting PATH for Python 3.7.5 (Homebrew)
PATH="/usr/local/bin:${PATH}"
export PATH
# Set module path
PYTHONPATH=$PYTHONPATH:/usr/local/lib/python3.7/site-packages
# Set PATH priority to Homebrew installation folder
export PATH=/usr/local/bin:/usr/local/sbin:$PATH
With this settings, I was able to install the module and make it work in my script!
I install python 3.6.5_1 using
brew install https://raw.githubusercontent.com/Homebrew/homebrew-core/f2a764ef944b1080be64bd88dca9a1d80130c558/Formula/python.r
and get
Warning: python 3.6.5_1 is already installed
The currently linked version is 3.7.3
You can use brew switch python 3.6.5_1 to link this version.
I use brew switch python 3.6.5_1 and get
Cleaning /usr/local/Cellar/python/3.6.5_1
Cleaning /usr/local/Cellar/python/3.7.3
4 links created for /usr/local/Cellar/python/3.6.5_1
Typing which python3, there is no output, which python gives
/usr/local/bin/python
and typing python3 gives
-bash: /usr/local/bin/python3: No such file or directory
typing python gives
Python 2.7.15 (default, Nov 27 2018, 21:24:58)
[GCC 4.2.1 Compatible Apple LLVM 10.0.0 (clang-1000.11.45.5)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
How do I make sure that the installed python3.6.5_1 is installed at correct location and is the one referred to by python3?
Extra background :
I have been working with OpenCV for around 5 months now, following this installation guide for installing all requirements including python3.6. I was using this version of brew's python instead of mac's one. Recently I had to install FFmpeg, some errors came up, unwittingly I did something (I don't remember, probably unlinked python) and then I wasn't able to install FFmpeg anyway but also this destroyed my workspace. Right now I keep getting the following error everytime I open up terminal:
-bash: /usr/local/bin/python3: No such file or directory
virtualenvwrapper.sh: There was a problem running the initialization hooks.
If Python could not import the module virtualenvwrapper.hook_loader,
check that virtualenvwrapper has been installed for
VIRTUALENVWRAPPER_PYTHON=/usr/local/bin/python3 and that PATH is
set properly.
I tried brew install python3 and that solves the above warning for virtualenvwrapper. But it installs python3.7.3, whereas I need python3.6 for tensor flow etc.
python3 is my local Anaconda version of python, while python3.4 is the system one. I can import gi module with python3.4 (probably because i installed it with sudo apt-get install python3-gi) but python3 doesn't see it:
$ python3 -c 'import gi'
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named 'gi'
$ python3.4 -c 'import gi'
$ python3 --version
Python 3.5.1 :: Anaconda 4.0.0 (64-bit)
$ python3.4 --version
Python 3.4.3
$ which python3
/home/kossak/anaconda3/bin/python3
$ which python3.4
/usr/bin/python3.4
$
How should i install gi for Anaconda python? Or maybe i can somehow import sysem-wide modules?
My os:
System: Kernel: 3.19.0-32-generic x86_64 (64 bit gcc: 4.8.2) Desktop: Cinnamon 2.8.8 (Gtk 2.24.23) dm: mdm
Distro: Linux Mint 17.3 Rosa
If you're using conda virtualenv for python-3, you can use
$ conda install -c conda-forge pygobject
in your virtualenv
You can read more about this on:
https://anaconda.org/conda-forge/pygobject
This is how you do it: (example for Linux Mint and python3)
First install gi module using your distro package manager. For Linux Mint it would be:
sudo apt-get install python3-gi
Then run your distro python to check where the module is located:
$ /usr/bin/python3
Python 3.5.2 (default, Sep 10 2016, 08:21:44)
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>> print(gi)
<module 'gi' from '/usr/lib/python3/dist-packages/gi/__init__.py'>
So in my case the module gi was installed to /usr/lib/python3/dist-packages/gi. Assuming you have your anaconda installed in /home/kossak/anaconda3, create a link to gi module in the proper folder:
ln -s /usr/lib/python3/dist-packages/gi /home/kossak/anaconda3/lib/python3.5/site-packages/
If you have conda virtual environment and want gi module to be available there, the path should be a bit different. Assuming the virtual env is called TEST:
ln -s /usr/lib/python3/dist-packages/gi /home/kossak/anaconda3/envs/TEST/lib/python3.5/site-packages/
and it works:
$ python3
Python 3.5.2 |Anaconda custom (64-bit)| (default, Jul 2 2016, 17:53:06)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-1)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import gi
>>>
If you want to perform a proper install (without soft linking) inside a Linux anaconda environment. Keep in mind that the errors may vary if you have not installed gcc previously (I assumed it was installed by default when I posted, however not everyone does so). Install it if you don't know very well what you're doing to avoid missunderstandings
Create or open your conda environment.
Attemp to install pygobject (don't worry, it will most likely throw an error):
pip install pygobject
In linux, it will promp the usual installation progress followed by an error:
(...) Please, try executing the following in your system:
sudo apt install libgirepository1.0-dev
Depending on your operation sistem or installed dependencies, the command name or package name may vary. Just follow the instructions and allow the system to install your packets. This step doesn't change anything, is just to give you the precise info of the package you need on your system. If you run this on Windows, it will ask you to install a specific version of Visual Studio. If you are in Windows, download the required Visual Studio from MS website, install it, reboot your computer and go to step 5 (in my case I never needed step 4 in windows, however, I'm not a MS expert.
Close your conda environment
conda deactivate
Next you need to install pygobject from conda-forge repository in your conda environment. You can add the repo to your favourite conda package manager or simply run the following command as root (it is important to be root). I did it outside the project, but you may do it inside:
conda install -c conda-forge pygobject
In my case conda was not in the path. I had is installed in:
/opt/anaconda3/bin/
You can run the following command from your normal user to find out where conda is:
which conda
Open the conda environment
source activate <your env name>
or the corresponding anaconda activate syntax (I never use it so I cant remember precisely)
Repeat the first step and now the installation wont fail:
pip install pygobject
OR if you specifically want to install gi you can run:
pip install pgi
the correct package is "pgi" NOT "gi"
As gi is a dependency of pygobject, everything will get properly installed. You can check it by running
python
>>> import gi
You may find the following usefull for Windows, although you may need to work it out a bit:
GStreammer python bindings on Windows
for me
conda install -c pkgw/label/superseded gtk3
worked
I have a setup.py that looks a bit (okay, exactly) like this:
#!/usr/bin/env python
from setuptools import setup
import subprocess
import distutils.command.build_py
class BuildWithMake(distutils.command.build_py.build_py):
"""
Build using make.
Then do the default build logic.
"""
def run(self):
# Call make.
subprocess.check_call(["make"])
# Keep installing the Python stuff
distutils.command.build_py.build_py.run(self)
setup(name="jobTree",
version="1.0",
description="Pipeline management software for clusters.",
author="Benedict Paten",
author_email="benedict#soe.ucsc.edu",
url="http://hgwdev.cse.ucsc.edu/~benedict/code/jobTree.html",
packages=["jobTree", "jobTree.src", "jobTree.test", "jobTree.batchSystems",
"jobTree.scriptTree"],
package_dir= {"": ".."},
install_requires=["sonLib"],
# Hook the build command to also build with make
cmdclass={"build_py": BuildWithMake},
# Install all the executable scripts somewhere on the PATH
scripts=["bin/jobTreeKill", "bin/jobTreeStatus",
"bin/scriptTreeTest_Sort.py", "bin/jobTreeRun",
"bin/jobTreeTest_Dependencies.py", "bin/scriptTreeTest_Wrapper.py",
"bin/jobTreeStats", "bin/multijob", "bin/scriptTreeTest_Wrapper2.py"])
It installs the package perfectly fine when run with ./setup.py install. However, it does this whether or not the "sonLib" package is installed, ignoring the dependency.
Is this expected behavior? Should a setup.py install blithely proceed if the dependencies are not installed, leaving it up to pip or whatever to install them beforehand? If not, and setup.py install ought to fail when dependencies are absent, what am I doing wrong?
EDIT: Some version information:
Python 2.7.2 (default, Jan 19 2012, 21:40:50)
[GCC 4.1.2 20080704 (Red Hat 4.1.2-48)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import setuptools
>>> setuptools.__version__
'0.6c12'
>>>
The default install command for Distutils setup doesn't know anything about dependencies. If you are running that, you're right that dependencies will not be checked.
Just going by what you've show in the setup.py, though, you are using Setuptools for the setup function. The Setuptools install command is declared to run easy_install, which in turn does check and download dependencies.
It is possible you are explicitly invoking the Distutils install, by specifying install --single-version-externally-managed.
I have a problem similar to this post: Install python module to non default version of python on Mac, so I am aware of those solutions, but they do not work for me.
I am installing M2Crypto on CentOS, which means I much use fedora_setup.sh build followed by fedora_setup.sh install in order to install on my architecture.
Unfortunately, the default Python version is 2.6, but I use 2.7. How do I execute the build and install commands so that they build and install to Python2.7 site-packages? Is there a simple command I don't know? I've been searching around here: http://docs.python.org/2/install/ in the Python Docs, but I don't see anything about .sh scripts?
You should run your scripts in a virtualenv created for your app's environment. This creates an isolated environment that uses the Python interpreter you created the virtualenv with, but with its own set of libraries.
# create the virtualenv folder: M2Crypto-venv
python2.7 virtualenv.py --distribute M2Crypto-venv
# activate the virtualenv, changing environment variables to use its Python interpreter
. M2Crypto-venv/bin/activate
# see how the current python has changed
which python # should be M2Crypto-venv/bin/python
python --version # should be 2.7
# after activating, run your install scripts
If you're using mod_wsgi or something similar to serve content, you'll want to modify your WSGI file to activate the virtualenv before doing anything else (adapted from mod_wsgi instructions):
import os.path
virtualenv_path = '/path/to/M2Crypto-venv'
activate_this = os.path.join(virtualenv_path, 'bin/activate_this.py')
execfile(activate_this, dict(__file__ = activate_this))
# rest of the WSGI file...
This was an incredibly difficult answer to come by, but the support team at Webfaction where I am hosted were spectacular in assisting me. Directly from the support I was given:
First build swig,
wget http://prdownloads.sourceforge.net/swig/swig-2.0.8.tar.gz
tar -xf swig-2.0.8.tar.gz
cd swig-2.0.8
./configure --prefix=$HOME
make
make install
Than get m2crypto,
svn checkout http://svn.osafoundation.org/m2crypto/tags/0.21/ m2crypto-0.21
cd m2crypto-0.21/
Edit fedora_setup.sh from this
SWIG_FEATURES=-cpperraswarn python setup.py $*
to this,
SWIG_FEATURES=-cpperraswarn python2.7 setup.py $*
Then build, then install,
./fedora_setup.sh build
./fedora_setup.sh install --prefix=$HOME
[me#web342 lib]$ python2.7
Python 2.7.5 (default, May 16 2013, 20:16:09)
[GCC 4.4.7 20120313 (Red Hat 4.4.7-3)] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import M2Crypto
>>> print M2Crypto
<module 'M2Crypto' from '/home/me/lib/python2.7/site-packages/M2Crypto-0.21-py2.7-linux-x86_64.egg/M2Crypto/__init__.pyc'>
Obviously, substitute your own details throughout. Hope this helps the next guy trying to install M2Crytpo using fedora_setup to a non-default python version.