I'm using the following commands in my Dockerfile to install Miniconda. After I install it, I want to use the binaries in ~/miniconda3/bin like python and conda. I tried exporting the PATH with this new path prepended to it, but the subsequent pip command fails (pip is located in ~/miniconda3/bin.
Curiously, if I run the container in interactive terminal mode, the path is set correctly and I'm able to call the binaries as expected. It seems as though the issue is only when building the container itself.
FROM ubuntu:18.04
RUN apt-get update
RUN apt-get install -y python3.7
RUN apt-get install -y curl
RUN curl https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh --output miniconda.sh
RUN bash miniconda.sh -b
RUN export PATH="~/miniconda3/bin:$PATH"
RUN pip install pydub # errors out when building
Here's the result of echo $PATH
~/miniconda3/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
Here's the error I get
/bin/sh: 1: pip: not found
export won't work. Try ENV
Replace
RUN export PATH="~/miniconda3/bin:$PATH"
with
ENV PATH="~/miniconda3/bin:$PATH"
Even though Miniconda is located in ~, it default installs to the root directory unless otherwise specified.
Here's the right command.
RUN export PATH="/root/miniconda3/bin:$PATH"
It looks like your export PATH ... command is putting the literal symbol ~ into the path. Try this:
ENV PATH="$HOME/miniconda3/bin:$PATH"
I have the problem that when I run my code on a linux server I get:
ModuleNotFoundError: No module named '_sqlite3'
So after researching, I found out sqlite3 was supposed to have been installed when I installed python, however it didn't.
I think the problem comes from the way I installed python. Since I do not have sudo permissions, I installed python3.7 in a local directory using: This guide.
All solutions to this sqlite3 problem that I can find requires sudo commands.
Is there another way that I can install python3.7 together with sqlite3 in my local Linux directory without using any sudo commands?
I hope I have stated my question clearly and I would appreciate all the help I can get. Thank you!
While installing a python package in a Linux system without "sudo" privileges you can use
For Python 3
pip3 install --user pysqlite3
You can install any third party packages with the same method
pip3 install --user PACKAGE_NAME
The --user flag to pip install tells Pip to install packages in some specific directories within your home directory. For more information click here.
Hope it helps !
The solution is to first build sqlite3 into a user directory and then build python using that directory's libraries and include headers. In particular, #Ski has answered a similar question regarding python 2, which can be adopted to python 3:
$ mkdir -p ~/applications/src
$ cd ~/applications/src
$ # Download and build sqlite 3 (you might want to get a newer version)
$ wget http://www.sqlite.org/sqlite-autoconf-3070900.tar.gz
$ tar xvvf sqlite-autoconf-3070900.tar.gz
$ cd sqlite-autoconf-3070900
$ ./configure --prefix=~/applications
$ make
$ make install
$ # Now download and build python 2, same works for python 3
$ cd ~/applications/src
$ wget http://www.python.org/ftp/python/2.5.2/Python-2.5.2.tgz
$ tar xvvf Python-2.5.2.tgz
$ cd Python-2.5.2
$ ./configure --prefix=~/applications
$ make
$ make install
$ ~/applications/bin/python
Alternatively, if you already have to specify a different --prefix for some reason (this has happened to me with pyenv), use LDFLAGS and CPPFLAGS when configuring python build:
$ ./configure LDFLAGS=-L/home/user/applications/lib/ CPPFLAGS=-I/home/user/applications/include/
I have successfully installed python 2.7.11 on a shared Bluehost server.
In the home directory I installed get-pip.py When I run that now,
# python get-pip.py
Requirement already up-to-date: pip in ./python/lib/python2.7/site-packages
But when I try to run pip I get,
usernmame#example.com [~]# pip
-bash: pip: command not found
Why is pip not running? How can I check what python packages are installed?
My ~/.bashrc looks like this,
# .bashrc
export PATH=$HOME/python/Python-2.7.11/:$PATH
# User specific aliases and functions
alias mv='mv -i'
alias rm='rm -i'
alias cp='cp -i'
# Source global definitions
if [ -f /etc/bashrc ]; then
. /etc/bashrc
fi
Additionally,
# echo $PATH
/usr/local/jdk/bin:/home2/username/python/Python-2.7.11/:/usr/lib64/qt-3.3/bin:/home2/username/perl5/bin:/ramdisk/php/54/bin:/usr/php/54/usr/bin:/usr/bin:/bin:/usr/local/sbin:/usr/sbin:/sbin:/usr/local/bin:/usr/X11R6/bin:/home2/username/bin
EDIT:
My attempt to add the path to $PATH in ~/.bashrc
export PATH=$HOME/python/Python-2.7.11/:$HOME/python/lib/python2.7/site-packages/:$PATH
But it doesn't work, same error.
Thanks
You can run pip by invoking the -m option when you use python to run pip. Suppose you want to use pip to install a Python package called "package-of-interest" from PyPI. Then, you can install it by running:
python -m pip install package-of-interest
I've been trying to get up and running with the built-in "venv" module of Python 3.3 on my OS X machine. I've installed Python 3.3 using Homebrew.
As per the docs, creating and switching virtual environment works as you'd expect:
$ python3 -m venv myvenv
$ source myvenv/bin/activate
And I've tested something like this:
$ echo "YEAH = 'YEAH!'" > myvenv/lib/python3.3/site-packages/thingy.py
$ python
>>> import thingy
>>> print(thingy.YEAH)
'YEAH!'
But when I try to install distribute, it simply won't go in the proper place. For some reason, it insists on trying to install into /usr/local/lib/python3.3/site-packages/, which fails with the following messages:
No setuptools distribution found
running install
Checking .pth file support in /usr/local/lib/python3.3/site-packages/
/Users/victor/myvenv/bin/python -E -c pass
TEST FAILED: /usr/local/lib/python3.3/site-packages/ does NOT support .pth files
error: bad install directory or PYTHONPATH
You are attempting to install a package to a directory that is not
on PYTHONPATH and which Python does not read ".pth" files from. The
installation directory you specified (via --install-dir, --prefix, or
the distutils default setting) was:
/usr/local/lib/python3.3/site-packages/
and your PYTHONPATH environment variable currently contains:
''
This happens regardless if I try to install using distribute_setup.py or using the source distribution directly. I've even tried using --prefix=/Users/victor/myenv but it still tries to put everything in my "global" site-packages.
I can't figure out why this happens, but it's consistent on two of my machines. Note that sys.prefix reports the correct path (the virtual environment).
Is this a problem with Homebrew? OS X? Python 3.3? venv? Me?
This has been an issue with Homebrew, yes, but it is working now since https://github.com/mxcl/homebrew/commit/0b50110107ea2998e65011ec31ce45931b446dab.
$ brew update
$ brew rm python3 #if you have installed it before
$ brew install python3
$ cd /tmp
$ which python3
/usr/local/bin/python3
$ python3 -m venv myvenv
$ source myvenv/bin/activate
$ wget http://python-distribute.org/distribute_setup.py # may need brew install wget
$ python3 distribute_setup.py
...
Finished processing dependencies for distribute==0.6.45
After install bootstrap.
Creating /private/tmp/myvenv/lib/python3.3/site-packages/setuptools-0.6c11-py3.3.egg-info
Creating /private/tmp/myvenv/lib/python3.3/site-packages/setuptools.pth
You see that distribute install successfully into the /tmp dir.
This happens because homebrew installs distutils config file:
$ brew cat python3 | grep "Tell distutils" -A5
# Tell distutils-based installers where to put scripts
(prefix/"Frameworks/Python.framework/Versions/#{VER}/lib/python#{VER}/distutils/distutils.cfg").write <<-EOF.undent
[install]
install-scripts=#{scripts_folder}
install-lib=#{site_packages}
EOF
$ mv ~/.local/Frameworks/Python.framework/Versions/3.3/lib/python3.3/distutils/distutils.cfg ~/tmp/
$ cat ~/tmp/distutils.cfg
[install]
install-scripts=/Users/gatto/.local/share/python3
install-lib=/Users/gatto/.local/lib/python3.3/site-packages
$ . venv/bin/activate
(venv) $ python distribute-0.6.36/distribute_setup.py
(venv) $ ls venv/lib/python3.3/site-packages/
distribute-0.6.36-py3.3.egg easy-install.pth setuptools-0.6c11-py3.3.egg-info setuptools.pth
See "distutils.cfg Can Break venv" issue at bugs.python.org.
Am on Debian 5, I've been trying to install cx_oracle module for python without any success. First, I installed oracle-xe-client and its dependency (followed tutorial in the following link here).
Then, I used the scripts in /usr/lib/oracle/xe/app/oracle/product/10.2.0/client/bin to populate environment variables such as PATH, ORACLE_HOME and NLS_LANG.
Once, this was completed, I tried to run:
sudo easy_install cx_oracle
But I keep getting the following error:
Searching for cx-oracle
Reading http://pypi.python.org/simple/cx_oracle/
Reading http://cx-oracle.sourceforge.net
Reading http://starship.python.net/crew/atuining
Best match: cx-Oracle 5.0.4
Downloading http://prdownloads.sourceforge.net/cx-oracle/cx_Oracle-5.0.4.tar.gz?download
Processing cx_Oracle-5.0.4.tar.gz
Running cx_Oracle-5.0.4/setup.py -q bdist_egg --dist-dir /tmp/easy_install-xsylvG/cx_Oracle-5.0.4/egg-dist-tmp-8KoqIx
error: cannot locate an Oracle software installation
Any idea what I missed here?
The alternate way, that doesn't require RPMs. You need to be root.
Dependencies
Install the following packages:
apt-get install python-dev build-essential libaio1
Download Instant Client for Linux x86-64
Download the following files from Oracle's download site:
Extract the zip files
Unzip the downloaded zip files to some directory, I'm using:
/opt/ora/
Add environment variables
Create a file in /etc/profile.d/oracle.sh that includes
export ORACLE_HOME=/opt/ora/instantclient_11_2
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME
Create a file in /etc/ld.so.conf.d/oracle.conf that includes
/opt/ora/instantclient_11_2
Execute the following command
sudo ldconfig
Note: you may need to reboot to apply settings
Create a symlink
cd $ORACLE_HOME
ln -s libclntsh.so.11.1 libclntsh.so
Install cx_Oracle python package
You may install using pip
pip install cx_Oracle
Or install manually
Download the cx_Oracle source zip that corresponds with your Python and Oracle version. Then expand the archive, and run from the extracted directory:
python setup.py build
python setup.py install
I recommend that you grab the rpm files and install them with alien. That way, you can later on run apt-get purge no-longer-needed.
In my case, the only env variable I needed is LD_LIBRARY_PATH, so I did:
echo export LD_LIBRARY_PATH=/usr/lib/oracle/11.2/client/lib >> ~/.bashrc
source ~/.bashrc
I suppose in your case that path variable will be /usr/lib/oracle/xe/app/oracle/product/10.2.0/client/lib.
The following worked for me, both on mac and Linux. This one command should download needed additional files, without need need to set environment variables.
python -m pip install cx_Oracle --pre
Note, the --pre option is for development and pre-release of the Oracle driver. As of this posting, it was grabbing cx_Oracle-6.0rc1.tar.gz, which was needed. (I'm using python 3.6)
Thx Burhan Khalid, I overlooked your "You need to be root" quote, but found the way when you are not the root here.
At point 7 you need to use:
sudo env ORACLE_HOME=$ORACLE_HOME python setup.py install
Or
sudo env ORACLE_HOME=/path/to/instantclient python setup.py install
Thanks Burhan Khalid. Your advice to make a a soft link make my installation finally work.
To recap:
You need both the basic version and the SDK version of instant client
You need to set both LD_LIBRARY_PATH and ORACLE_HOME
You need to create a soft link (ln -s libclntsh.so.12.1 libclntsh.so in my case)
None of this is documented anywhere, which is quite unbelievable and quite frustrating. I spent over 3 hours yesterday with failed builds because I didn't know to create a soft link.
I think it may be the sudo has no access to get ORACLE_HOME.You can do like this.
sudo visudo
modify the text add
Defaults env_keep += "ORACLE_HOME"
then
sudo python setup.py build install
Alternatively you can install the cx_Oracle module without the PIP using the following steps
Download the source from here https://pypi.python.org/pypi/cx_Oracle
[cx_Oracle-6.1.tar.gz ]
Extract the tar using the following commands (Linux)
gunzip cx_Oracle-6.1.tar.gz
tar -xf cx_Oracle-6.1.tar
cd cx_Oracle-6.1
Build the module
python setup.py build
Install the module
python setup.py install
This just worked for me on Ubuntu 16:
Download ('instantclient-basic-linux.x64-12.2.0.1.0.zip' and 'instantclient-sdk-linux.x64-12.2.0.1.0.zip') from Oracle web site and then do following script (you can do piece by piece and I did as a ROOT):
apt-get install -y python-dev build-essential libaio1
mkdir -p /opt/ora/
cd /opt/ora/
## Now put 2 ZIP files:
# ('instantclient-basic-linux.x64-12.2.0.1.0.zip' and 'instantclient-sdk-linux.x64-12.2.0.1.0.zip')
# into /opt/ora/ and unzip them -> both will be unzipped into 1 directory: /opt/ora/instantclient_12_2
rm -rf /etc/profile.d/oracle.sh
echo "export ORACLE_HOME=/opt/ora/instantclient_12_2" >> /etc/profile.d/oracle.sh
echo "export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$ORACLE_HOME" >> /etc/profile.d/oracle.sh
chmod 777 /etc/profile.d/oracle.sh
source /etc/profile.d/oracle.sh
env | grep -i ora # This will check current ENVIRONMENT settings for Oracle
rm -rf /etc/ld.so.conf.d/oracle.conf
echo "/opt/ora/instantclient_12_2" >> /etc/ld.so.conf.d/oracle.conf
ldconfig
cd $ORACLE_HOME
ls -lrth libclntsh* # This will show which version of 'libclntsh' you have... --> needed for following line:
ln -s libclntsh.so.12.1 libclntsh.so
pip install cx_Oracle # Maybe not needed but I did it anyway (only pip install cx_Oracle without above steps did not work for me...)
Your python scripts are now ready to use 'cx_Oracle'... Enjoy!
This worked for me
python -m pip install cx_Oracle --upgrade
For details refer to the oracle quick start guide
https://cx-oracle.readthedocs.io/en/latest/installation.html#quick-start-cx-oracle-installation
If you are trying to install in MAC , just unzip the Oracle client which you downloaded and place it into the folder where you written python scripts.
it will start working.
There is too much problem of setting up environmental variables.
It worked for me.
Hope this helps.
Thanks
Try to reinstall it with the following code:
!pip install --proxy http://username:windowspwd#10.200.72.2:8080 --upgrade --force-reinstall cx_Oracle
If you require to install a specific version of cx_Oracle, like 7.3 which was the last version with support for Python 2, you can do the following:
python -m pip install cx_Oracle==7.3