Automate conda and package installations singularity definition file - python

I am trying to automate installation of miniconda and some packages thereafter within a singularity definition file to build an image. I am always getting the following error:
/.post.script: 10: conda: not found
FATAL: While performing build: while running engine: exit status 127
Assuming that one has to restart shell after installing conda, is there any way to automate this? The definition file is written below.
Bootstrap: docker
From: ubuntu:22.04
%post
apt-get update
apt-get -y install gfortran
apt-get -y install build-essential
apt-get -y install manpages-dev
apt-get -y install libgeos-dev
apt-get -y install wget
wget -c https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh
/bin/bash Miniconda3-latest-Linux-x86_64.sh -bfp /opt
conda update conda
conda update python
conda install pip -y
pip install -y setuptools>=65.6
pip install -y cartopy==0.21.0
pip install -y contextily==1.2.0
pip install -y fiona==1.8.21
pip install -y pyproj==3.3.1
pip install -y rtree==1.0.1
pip install -y Shapely==1.8.2
pip install -y numpy==1.23.0
pip install -y pandas==1.4.3
pip install -y geopandas==0.11.0
pip install -y joblib==1.2.0
pip install -y matplotlib==3.6.2
pip install -y mercantile==1.2.1
pip install -y pytest==7.2.0
pip install -y python_dateutil==2.8.2
pip install -y pytz==2021.3
pip install -y timezonefinder==6.1.6
pip install -y pyarrow==10.0.0

Related

Add RobotFramework to Ubuntu:16.04

enter image description hereI want to download robotframework in ubuntu for dockerfile but i didn't found. What can i do?
FROM ubuntu:16.04
RUN apt-get update
RUN apt-get install software-properties-common -y
RUN add-apt-repository --yes ppa:ansible/ansible-2.10
RUN apt-get install ansible -y
RUN add-apt-repository -y ppa:jblgf0/python
RUN apt-get update
RUN apt-get install python3.6 -y
RUN apt-get install python-requests -y
RUN apt-get install python-pip -y
RUN pip install --upgrade pip
RUN pip install robotframework
RUN pip install --upgrade robotframework-sshlibrary
ENTRYPOINT [/bin/bash]
enter image description here
You are having problems with the versions you are requesting. pip for example is failing to install. Also python3.6 is an old version.
You should update your Dockerfile to use Ubuntu LTS, which is currently in version 22.0.4.

Unable to install `gattlib`

I am trying to install gattlib in Python in order to use some of its Bluetooth-tools.
The OS is ubuntu 18.04.4 LTS.
I have by now tried the following (as e.g. here):
sudo apt-get install mercurial
hg clone https://bitbucket.org/OscarAcena/pygattlib
cd pygattlib
cat DEPENDS
sudo apt-get install libboost-thread-dev libboost-python-dev libbluetooth-dev libglib2.0-dev python-dev
sudo python3 setup.py install
which gives (my Python is 3.6.9):
usr/bin/ld: cannot find -lboost-python36
collect2: error: ld returned 1 exit status
error: command 'x86_64-linux-gnu-g++' failed with exit status 1
Also, I tried:
sudo apt install python3-gattlib
which gives:
the following packages have unmet dependencies:
python3-gattlib: Depends: python3 (>= 3.7~) but 3.6.7-1~18.04 is to be installed
Depends: libboost-python1.67.0 but is is not installable
Depends: libboost-thread1.67.0 but is is not installable
despite libboost-python and libboost-thread being successfully installed in the first attempt (see above), and python3 returning Python 3.6.9.
pip3 install gattlib
shows:
Building wheel for gattlib(setup.py) ... error
Running setup.py install for gattlib ... error
Which is the exact same result that I get from (following instructions from this question):
sudo pip3 download gattlib
sudo tar xvzf ./gattlib-0.20200122.tar.gz
cd gattlib-0.20200122/
sudo sed -ie 's/boost_python-py34/boost_python36/' setup.py
pip3 install .
in which I understand to be a necessary adjustment of the installation file before running it, because, if I understood correctly, the python version is somehow wrongly hardcoded in there.
Futher things I tried and that did not help:
pip3 install --upgrade setuptools
sudo apt-get install python3.6-dev libmysqlclient-dev
sudo apt-get install libmysqlclient-dev
sudo apt-get install python-dev
sudo apt-get install libpython-dev
sudo apt-get install libevent-dev
sudo pip3 install gattlib
wget -qO- http://pike.esi.uclm.es/add-pike-repo.sh | sudo sh
sudo apt update
sudo apt install python3-gattlib
sudo apt-get install libbluetooth-dev
pip3 install --upgrade pip
sudo apt-get install mercurial
pip3 install gattlib
sudo apt-get install libboost-all-dev
cd /usr/lib/x86-64-linux-gnu
sudo ln -s libboost_python-py35.so libboost_python-py36.so
sudo apt-get install libbluetooth-dev bluez bluez-hcidump libboost-python-dev libboost-thread-dev libglib2.0-dev
hg clone https://bitbucket.org/OscarAcena/pygattlib
cd pygattlib
cat DEPENDS
sudo apt-get install libboost-thread-dev libboost-python-dev libbluetooth-dev libglib2.0-dev python-dev
sudo python3 setup.py install
sudo python setup.py install
pip3 install gTTS
sudo apt-get install python3 python-dev python3-dev \
build-essential libssl-dev libffi-dev \
libxml2-dev libxslt1-dev zlib1g-dev \
python-pip
sudo apt-get install aptitude
sudo aptitude install libboost-all-dev
I run
sudo python3 setup.py install
and I also get error
usr/bin/ld: cannot find -lboost-python36
because I don't have boost-python36.a but boost-python3-py36.a.
(I found this file using locate boost-python3 which uses database with filenames so it works faster then find but it may not be installed as default)
I had to edit setup.py and change
boost_libs = ["boost_python3"+str(sys.version_info.minor)]
to
boost_libs = ["boost_python3-py36"]
or more universal
boost_libs = ["boost_python3-py3"+str(sys.version_info.minor)]
Tested od Linux Mint 19.3 Tricia based on Ubuntu 18.04
You need to install python-dev that contains the header files for the Python C API. The following should do the trick (make sure to replace X with your Python version):
sudo apt-get install python3.X-dev
With ubuntu 20.04:
$ wget https://github.com/oscaracena/pygattlib/releases/download/v.20201113/python3-gattlib_0.20201113-1_amd64.deb
$ sudo apt install ./python3-gattlib*.deb
$ pip3 install gattlib
Reference:
https://pypi.org/project/gattlib/
NOTE: This didn't work for me!
sudo apt install pkg-config libboost-python-dev libboost-thread-dev libbluetooth-dev libglib2.0-dev python-dev

Keras installed but not importing (rpi)

I successfully installed Keras on my Raspberry Pi using the instructions here:
https://medium.com/#abhizcc/installing-latest-tensor-flow-and-keras-on-raspberry-pi-aac7dbf95f2
sudo apt-get install python3-numpy
sudo apt-get install libblas-dev
sudo apt-get install liblapack-dev
sudo apt-get install python3-dev
sudo apt-get install libatlas-base-dev
sudo apt-get install gfortran
sudo apt-get install python3-setuptools
sudo apt-get install python3-scipy
sudo apt-get update
sudo apt-get install python3-h5py
sudo pip3 install keras
But when I try to import keras into a jupyter notebook (Python 3) or to a python shell, it says
ModuleNotFoundError: No module named 'keras'
Any help would be greatly appreciated.
You can check what interpreter you are using with:
which python and check where keras is installed with pip show keras.
The problem is that the python you are using in jupyter and the shell is not the one for which you installed keras.

How to install pip for python3.7 only?

I have python3.7 and I want to install pip. However when I do the following:
sudo apt install python3-pip
It seems to download python version 3.6 and pip for that. Is there a way to not download python3.6 and just download pip for python3.7? It seems rather strange that pip is download a whole other package.
Download get-pip file
$curl https://bootstrap.pypa.io/get-pip.py -o get-pip.py
Install pip for python3.7
$python3.7 get-pip.py
Check versions for both
$python3.7 -V && pip3 -V
Output:
Python 3.7.3
pip 19.1.1 from /usr/local/lib/python3.7/dist-packages/pip (python 3.7)
if you check apt show python3-pip output you see it depends on python3:any (>= 3.4~) so it doesn't strictly say I need python3.6
I assume that you didn't install python3.7 using apt so as a result apt doesn't recognize that you have python3 and it tries to install it for you.
Deadsnakes
You can use a ppa called deadsnakes.
sudo apt update
sudo apt install software-properties-common
sudo add-apt-repository ppa:deadsnakes/ppa
sudo apt install python3.7
Then check that is has been installed successfully by running python3.7 --version. To use pip you can run python3.7 -m pip install package.
Compiling from source
You can build Python by yourself, on Debian based systems you would do:
sudo apt-get update; sudo apt-get install make build-essential libssl-dev zlib1g-dev \
libbz2-dev libreadline-dev libsqlite3-dev wget curl llvm \
libncursesw5-dev xz-utils tk-dev libxml2-dev libxmlsec1-dev libffi-dev liblzma-dev
Download your preferred python version from https://www.python.org/ftp/python/
Then extract the tar archive and run make
./configure --enable-optimizations
make
sudo make altinstall
Then run python3.7 --version
try this.
python3.7 -m pip install pip

python3.6-venv hijacks pip. what is a way to prevent this?

I am chasing down an install failure. I have a fresh ubuntu xenial install. I have several different projects, each one involving dependencies requiring a different version of python, including a gnuradio project which seems to have the reasonable expectation that pip points to python2.7. Here is the first part of what I've run:
$ sudo apt-get update
$ sudo apt-get -yq upgrade
$ sudo apt-get -yq install python-pip
$ sudo pip install --upgrade pip
$ sudo apt-get install -yq python3-pip
$ sudo pip3 install --upgrade pip
$ sudo apt-get update
$ sudo apt-get -yq upgrade
$ sudo apt-get -yq install python-dev python3-dev
$ sudo apt-get install --fix-missing python-apt
$ sudo pip install numpy scipy matplotlib pybombs virtualenv
$ sudo apt-get -yq install build-essential libffi-dev libssl-dev python3-setuptools
$ sudo pip3 install --upgrade setuptools wheel
$ sudo pip3 install numpy scipy matplotlib virtualenv
$ sudo add-apt-repository ppa:deadsnakes/ppa
$ sudo apt-get install python3.6
$ sudo apt-get -yq install python3.6-venv python3.6-dev
$ sudo apt-get update
$ sudo apt-get upgrade
No complaints so far, when I check pip and pip3, they point where I want them to point:
$ pip -V
pip 9.0.1 from /usr/local/lib/python2.7/dist-packages (python 2.7)
$ pip3 --version
pip 9.0.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)
Now I create a virtual environment for python3.6, enter it, in there pip points to python3.6 as expected, I install a few more packages, then exit the virtual environment and suddenly pip points to python3.5
$ mkdir projvenv
$ python3.6 -m venv /home/username/projvenv/
$ source /home/tom/vertexprojvenv/bin/activate
(projvenv) $ pip -V
pip 9.0.1 from /home/tom/projvenv/lib/python3.6/site-packages (python 3.6)
(projvenv) $ pip install numpy scipy matplotlib
(projvenv) $ deactivate
$ pip -V
pip 9.0.1 from /usr/local/lib/python3.5/dist-packages (python 3.5)
I have looked at this question which seems to address a purely anaconda issue, while the solution does not apply in my case. This discussion is more to the point, and points to a problem with pip. That particular bug is at least not quite what is going on for me since I am able to upgrade both pip pointing to python2.7 and pip3 pointing to python3.5, and the pip hijacking is not initiated by an upgrade. Any wisdom shed on this problem is much appreciated.
(edit)
$ which pip
/usr/local/bin/pip
$ head -1 `which pip`
#!/usr/bin/python3
Restore shebang line in /usr/local/bin/pip — make it #!/usr/bin/python2.
PS. Well, formally it's not a complete answer because the question is "How to prevent?" I don't know what program changed shebang line; but I doubt it was virtual environment.

Categories

Resources