Develop Reference

Does the pip option "--force-reinstall" recompiles a module? (I'm trying to reinstall cx_Oracle)

My setup has:
- oracle-instantclient (downgraded from 12 to 11)
- Virtual Environment with Python + Django + cx_Oracle
When I downgraded the oracle-instantclient, I tried to reinstall cx_oracle using:
pip install --upgrade --force-reinstall cx_Oracle
The problem is that cx_Oracle keeps complaining that I'm not using the previous installed version 12:
$ python -c "import cx_Oracle"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: libclntsh.so.12.1: cannot open shared object file: No such file or directory
So, some questions here: "--force-reinstall" is not supposed to make a full reinstall, including a module recompile (aka cx_Oracle)? Somebody has some suggestion about how to workaround this issue? I looked at the pip manual and just tells me that it reinstalls the package. Not very useful.
Update1:
I tried to follow the solution here, adding my cx_Oracle path to the LD_LIBRARY_PATH but it's still not working.
echo $LD_LIBRARY_PATH
/usr/lib/oracle/11.2/client64/lib:/home/myuser/myuserenv/lib/python3.5/site-packages/cx_Oracle-5.2.1.dist-info
Note that my root user has a different python while my virtualenv uses its own python installation.
[root#myserver]# which python
/usr/bin/python
From my virtual environment:
(myvirtualenv) [myuser#myserver]$ which python
~/myvirtualenv/bin/python
Is there some way of installing cx_Oracle manually using my root user but putting the files at the virtual environment somehow?
Update2:
I tried to download cx_Oracle and compile manually using the following command:
sudo -u myuser ~/myvirtualenv/bin/python setup.py build install
But now I'm seeing some permission errors:
(...)
error: could not create 'build/bdist.linux-x86_64/egg': Permission denied
Am I doing the right thing? I don't want to mess this environment. Thanks.

The answer to your question (although you seem to have had another problem) is as follows:
It depends on the module's structure. If its setup script calls the compiler, then yes it does. Because, you see, pip modules by default do not include compiled parts, it's up to the module author to provide them.

I could finally install. The problem is that I had to use the python of my virtualenv. A simple solution but that I completely missed.
Steps:
pip download cx_Oracle
tar -xvf cx_Oracle-5.2.1.tar.gz
~/myenv/bin/python setup.py -v build
sudo ~/myenv/bin/python setup.py install
Note that I had to use sudo in the last step. Without this I was having this permission error:
error: could not create 'build/bdist.linux-x86_64/egg': Permission denied

cx_Oracle does not recognize location of Oracle software installation for installation on Linux

I have been able to successfully install cx_Oracle for use with Python 3.4 on my Windows 8 laptop, and I am now trying to get the same setup (cx_Oracle with Python 3.4) onto a Linux machine. When running the setup.py file from cx_Oracle-5.1.3.tar.gz, I end up with this error:
sudo python3 setup.py install
Traceback (most recent call last):
File "setup.py", line 135, in <module>
raise DistutilsSetupError("cannot locate an Oracle software " \
distutils.errors.DistutilsSetupError: cannot locate an Oracle software installation
Following some other answers I looked at (easy_install cx_Oracle (python package) on Windows, https://gist.github.com/jarshwah/3863378) I have installed these 3 instant client rpms:
rpm -ivh oracle-instantclient12.1-basic-12.1.0.2.0-1.i386.rpm
rpm -ivh oracle-instantclient12.1-devel-12.1.0.2.0-1.i386.rpm
rpm -ivh oracle-instantclient12.1-sqlplus-12.1.0.2.0-1.i386.rpm
And then I set ORACLE_HOME to the folder that they were installed to, which is supposed to help python identify the location of the oracle files so it can do the installation properly.
I still get the same "cannot locate an Oracle software installation" error each time I try to run the setup.py file.
Any idea what I need to do to be able to successfully install cx_oracle?
Update for more info:
echo $ORACLE_HOME returns /instantclient_12_1, which is where the rpm files installed to.
This is the contents of my /instantclient_12_1 directory:
adrci libnnz12.so libsqlplusic.so tnsnames.ora
BASIC_README libocci.so libsqlplus.so tnsnames.ora_andy
genezi libocci.so.12.1 ojdbc6.jar uidrvci
glogin.sql libociei.so ojdbc7.jar xstreams.jar
libclntshcore.so.12.1 libocijdbc12.so sdk
libclntsh.so libons.so sqlplus
libclntsh.so.12.1 liboramysql12.so SQLPLUS_README
This is a bit different from the directory I have for my Windows 8 install - that one has .dll and .sym files, like orasql12.dll. Should the Linux version of the instant client install have different files?
Update with partial solution:
I found a solution that installed cx_Oracle properly, but only during that shell instance:
I set these two environment variables:
export ORACLE_HOME=/instantclient_12_1
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME
And then I created a Symbolic link:
ln -s libclntsh.so.12.1 libclntsh.so
After that, going to the cx_oracle folder and doing this worked:
python3 setup.py build
python3 setup.py install
For some reason, sudo python3 setup.py install did not work for this.
Update with link to related question:
My next problem is getting the environment variables to persist outside of the shell instance so I don't have to define the environment variables each time. The environment variables I put in profile.d show up when I echo them, but python fails to import cx_oracle properly, and I have to export the environment variables again for some reason. I don't know the proper procedure for posting a different question related to one, so I opened a new question here:
Linux profile.d environment variables don't work with cx_oracle in Python
Please help me out with this, I feel completely stuck on what to try to make it work. The environment variables show up when I echo them, but they only seem to be functional if I export them again before running the python code.

Updated
As Petriborg suggested, setting LD_RUN_PATH at build time will include the path to the Oracle shared library files in the cx_Oracle shared library that is built during installation. This obviates the need for LD_LIBRARY_PATH as I suggested in my first answer.
For the RPMs that you are using, ORACLE_HOME should be set to /usr/lib/oracle/12.1/client. If you are using pip:
$ export ORACLE_HOME=/usr/lib/oracle/12.1/client
$ export LD_RUN_PATH=/usr/lib/oracle/12.1/client/lib:$LD_RUN_PATH
$ pip install cx_Oracle
$ python -c 'import cx_Oracle; print(cx_Oracle.version)'
5.1.3
Read this documentation for some info on installing and executing applications that use the client libraries.

When I tried installing cx_Oracle with LD_LIBRARY_PATH variable alone in Ubuntu 16.04 with python 2.7.12 and Oracle client 12.1.0.2 pip install fails and is looking for header files which are no more available with Oracle 12.1.0.2 client. It works fine with LD_RUN_PATH

How to use pip with multiple instances of python

I recently installed python3 only to realize that mysql-python as well as many other modules were not well supported with it yet. So I changed the path in my bashrc file to point to an installation of python 2.7. The problem is that when I installed python 3 I also installed distribute and pip along with it. I removed the pip and distribute files from the python3 bin directory and installed setuptools and pip using python 2.7 however now when I use the pip command to install django and mysql-python, I get a bash error python331/bin/pip No such file or directory. It's still looking for pip in the python3 install. How can I remedy this?
Thanks

...I get a bash error python331/bin/pip No such file or directory.
It's still looking for pip in the python3 install. How can I remedy
this?
bash, by default, hashes the locations of commands to avoid searching $PATH each time, so if, when you execute...
$ type pip
...you get something like...
pip is hashed (python331/bin/pip)
...you just need to clear the hash table for bash with...
$ hash -r
...then it'll pick up the version in Python 2.7 the next time you try to run pip.

Fixed it.
Renamed the directory of where the python3 was installed, bash automatically looks for the next available python install python 2.7

Install mysqldb on snow leopard

I am trying to get started on working with Python on Django I am by profession a PHP developer and have been told to set up django and python on my current apache and mysql setup however I am having trouble getting the Mysqldb module for python to work, I must of followed about 6 different set of instructions, I am running snow leopard and have mysql installed natively it is not part of MAMP or similar. Please can some tell me where I need to start and what steps I need to follew I would be most grateful.
Thanks

On MAC OS X 10.6, Install the package as usual. The dynamic import error occurs because of wrong DYLD path. Export the path and open up a python terminal.
$ sudo python setup.py clean
$ sudo python setup.py build
$ sudo python setup.py install
$ export DYLD_LIBRARY_PATH=/usr/local/mysql/lib:$DYLD_LIBRARY_PATH
$python
import MySQLdb
Now import MySQLdb should work fine.
You may also want to manually remove the build folder, before build and install. The clean command does not do a proper task of cleaning up the build files.

I'd recomend installing macports (latest svn trunk) and installing mysql from there.
sudo port install mysql5-server
Download the MySQL-python-1.2.2 source
make sure /opt/local/lib/mysql5/bin is in your path or edit site.cfg to include:
mysql_config = /opt/local/lib/mysql5/bin/mysql_config
Comment out line 38 of _mysql.c
// #define uint unsigned int
Then run:
sudo python setup.py install
should be all good.

[Partial Answer]
You'll have more fun pulling out your teeth. MySQL/Django/Mac is a disaster. This is the farthest I've gotten:
Get MySQLDB 1.2.3
Go into that and modify setup_posix.py:
Change:
mysql_config.path = "mysql_config"
To (depending on the version number of your MySQL):
mysql_config.path = "/usr/local/mysql-5.1.34-osx10.5-x86_64/bin/mysql_config"
python setup.py build
python setup.py install
Here's a good article

First and foremost, make sure that XCode is installed. Without XCode, many pieces of the built-in Apache2 server important to developers are missing; most notably, the GNU Compiler Collection, which I would think to be requisite for MySQL bindings.

One of the key things here is to make sure you're running both MySQL and the Python adaptor in 64 bit. The default Python 2.6.1 install on Snow Leopard is 64 bit so you must install MySQL in 64 bit and build the MySQLdb adapter in 64 bit.
Make sure you have the latest Xcode installed
Install 64-bit MySQL from DMG - http://dev.mysql.com/downloads/mirror.php?id=401941#mirrors
Download MySQL-python-1.2.3 from http://download.sourceforge.net/sourceforge/mysql-python/MySQL-python-1.2.3.tar.gz
Extract and make the following edit to site.cfg:
mysql_config = /usr/local/mysql/bin/mysql_config
Then from your terminal run
ARCHFLAGS="-arch x86_64" python setup.py build
sudo python setup.py install
You should then open a Python shell and type:
import MySQLdb
If you don't get any errors you're golden.

Xcode was installed.
Follow these instructions for setting up apache/php/mysql:
http://maestric.com/doc/mac/apache_php_mysql_snow_leopard
I installed the free 32-bit version of EPD (this is optional but I wanted numpy/scipy).
Make sure you have these lines in your ~/.profile (the second line is only if you installed EPD):
export PATH=/usr/local/mysql/bin/:/usr/local/bin:/usr/local/sbin:$PATH
export PATH="/Library/Frameworks/Python.framework/Versions/Current/bin:${PATH}"
Downloaded mysqldb and run:
python setup.py build
sudo python setup.py install
Install the official release of Django or web2py. Everything worked after that. You can try the "Writing Your First Django App, Part 1" to test it out where in settings.py use:
DATABASE_ENGINE = 'mysql'
DATABASE_NAME = 'Django Polls'
DATABASE_USER = '*****'
DATABASE_PASSWORD = '*****'
DATABASE_HOST = '127.0.0.1'
DATABASE_PORT = '3306'
I also use Navicat Lite and Sequel Pro for working with MySQL.
Pydev is a nice IDE to use with Django.

Install the newest 64bit DMG version of MySQL. Remember to backup your databases if you already have MySQL installed.
enter this line in terminal:
sudo ln -s /usr/local/mysql/lib /usr/local/mysql/lib/mysql
Edit the file setup_posix in the mysql-python installation directory.
Change the line
mysql_config.path = "mysql_config"
to
mysql_config.path = "/usr/local/mysql/bin/mysql_config"
Myslq-Python needs a 64bit Version of Python. The new Python 2.7 64bit version creates an alias in /usr/local/bin/python.
Enter the following lines in the mysql-python folder
sudo /usr/local/bin/python setup.py clean
sudo ARCHFLAGS="-arch x86_64"
sudo /usr/local/bin/python setup.py build
sudo /usr/local/bin/python setup.py install
And finally try it out:
python
import MySQLdb

You could make your life a lot easier (especially on Lion) by just installing Mariadb instead:
http://mariadb.org/
It's a drop-in replacement for MySQL and is plug&play on OSX. The only thing you don't get is the MySQL system setting.

How to build sqlite for Python 2.4?

I would like to use pysqlite interface between Python and sdlite database. I have already Python and SQLite on my computer. But I have troubles with installation of pysqlite. During the installation I get the following error message:
error: command 'gcc' failed with exit status 1
As far as I understood the problems appears because version of my Python is 2.4.3 and SQLite is integrated in Python since 2.5. However, I also found out that it IS possible to build sqlite for Python 2.4 (using some tricks, probably).
Does anybody know how to build sqlite for Python 2.4?
As another option I could try to install higher version of Python. However I do not have root privileges. Does anybody know what will be the easiest way to solve the problem (build SQLite fro Python 2.4, or install newer version of Python)? I have to mention that I would not like to overwrite the old version version of Python.
Thank you in advance.

You can download and install Python to your home directory.
$ cd
$ mkdir opt
$ mkdir downloads
$ cd downloads
$ wget http://www.python.org/ftp/python/2.6.2/Python-2.6.2.tgz
$ tar xvzf Python-2.6.2.tgz
$ cd Python-2.6.2
$ ./configure --prefix=$HOME/opt/ --enable-unicode=ucs4
$ make
$ make install
Then, (if you are using bash) in your .bash_profile do
export PATH=$HOME/opt/bin/:$PATH
export PYTHONPATH=$HOME/opt/lib:$HOME/opt/lib/site-packages:$PYTHONPATH
Then, source the file to make it available
$ cd
$ source .bash_profile
$ python -V
where python -V will return the python version. If the correct version appears, any packages that you run with Python's setup.py util (assuming the developer followed the correct conventions) will install in ~/opt/lib/python2.x/site-packages directory.

Download pysqlite here, cd into the directory you downloaded to, unpack the tarball:
$ tar xzf pysqlite-2.5.5.tar.gz
then just do (if your permissions are set right for this; may need sudo otherwise):
$ cd pysqlite-2.5.5
$ python2.4 setup.py install
one error does appear in the copious output:
File "/Library/Frameworks/Python.framework/Versions/2.4/lib/python2.4/site-packages/pysqlite2/test/py25tests.py", line 48
with self.con:
^
SyntaxError: invalid syntax
since as clearly shown that file is for py 2.5 tests only (with statement not present in 2.4!-). Nevertheless the install is successful:
$ python2.4 -c'import pysqlite2'
$
All this is on Mac OS X 10.5 but using python2.4 separately installed from the system-supplied Python 2.5.
The error you report doesn't tell us much -- maybe you're missing the headers or libraries for sqlite itself? Can you show us other output lines around that single error msg...?

If you don't have root privileges, I would recommend installing a more recent version of Python in your home directory and then adding your local version to your PATH. It seems easier to go that direction than to try to make sqlite work with an old version of Python.
You will also be doing yourself a favor by using a recent version of Python, because you'll have access to the numerous recent improvements in the language.

I had the same trouble with gcc failing with Ubuntu Karmic. I fixed this by installing the python-dev package. In my case, I'm working with python2.4, so I installed the python2.4-dev package. The python-dev package should work for python2.6.