Develop Reference

Run PIP installed library in same Dockerfile during image build

I need to install a PIP library in my docker image and run it from the very same Dockerfile.
Example:
from python
COPY requirements.txt /app/requirements.txt
RUN pip3 install -U -r ete3
RUN 'python3 -c "from ete3 import NCBITaxa;NCBITaxa();"'
CMD ["gunicorn", "-k", "uvicorn.workers.UvicornWorker", "-c", "gunicorn_conf.py", "api.main:app"]
running this fails with
/bin/sh: 1: python3 -c "import ete3": not found
The command '/bin/sh -c 'python3 -c "from ete3 import NCBITaxa;NCBITaxa();"'' returned a non-zero code: 127
however, if I remove the RUN directive, finish the build, attach to the running container
python3 -c "from ete3 import NCBITaxa;NCBITaxa();" will work.
I assume since its pip and installed into /usr/local/lib/python3.8/dist-packages/ete3 it should be in the PYTHONPATH? Or is it that because this is at image build time the system is not yet available? Any tricks?
I need this functionality to pull some NCBI database updates at build time into the image so that this ete3 library is up2date for deployment. ete3 would pull the files at runtime and that is slow and terrible.
One alternative could be to prebuild an image with the PIP library installed from Dockerfile1 and run the update in Dockerfile2 ?

You need to put the commands you want to run into a python script like something.py.
Your something.py file can look like,
import eta3
...
...
...
Then change the last line to this,
CMD python /home/username/something.py
Based on your comment, you can do the same thing with a bash script and run it using the CMD command in the dockerfile like this.
Your script will have the commands you want to run like this.
#!/bin/bash
gunicorn -k uvicorn.workers.UvicornWorker -c gunicorn_conf.py
And you can add this line to your docker file.
CMD /bash_script.sh

Bad python path in bash script from crontab

I would like to use the python3 executable when I use it from crontab as manual launch (ssh session).
bash script
#!/bin/bash
PYTHONPATH="$(which python3)"
echo $PYTHONPATH
python3 test.py
result from ssh command line, launched manually
/usr/local/bin/python3
result in log file from crontab -e
/usr/bin/python3
I would like the script launched by the crontab, uses /usr/local/bin/python3 executable instead of /usr/bin/python3
OR
if it's not possible, use the dependencies of my code available for /usr/bin/python3
How can I achieve this ? Thank you very much for your help

the python inside the docker container will not necessarily have the same path. If you want all the modules installed on your VM python3, create a requirements.txt file using pip freeze > requirements.txe and COPY this file as part of your Dockerfile and install it while building the image pip install -r requirements.txt

Activcate a python virtual environment from a linux bash script

I am writing a bash script in linux that creates and activates a Python venv and then installs from a requirements.txt. Like this
python3 -m venv ~/myvenv/env
source ~/myvenv/env/bin/activate
cp requirements.txt ~/myvenv/env/requirements.txt
pip3 install -r ~/myvenv/env/requirements.txt
This doesn't work for me. It seems to create the myvenv directory but then doesnt switch in and run the requirements.txt file.
Is there a different way to activate it with source from within a bash script?

When you run script your shell spawns new process, activates it and then dies.
That's why when you get back into your shell you see unactivated one.
you can run your script just using source command. source command will load it into your active shell.
source script.sh

How can I run a python command for the correct name (py, python, python3)

I think this is related to: How can I check if a program exists from a Bash script?
Specifically, I want to run py -m http.server on computers that have py but don't have python3 and I want to run python3 -m http.server on computers that have python3 but not py. I also tried just checking the version number: py -v && py -m http.server; python3 -v && python3 -m http.server but this still seems to have the same problem, and hitting ctrl+C twenty times doesn't kill it.
I tried py && py -m http.server; python3 && python3 -m http.server but I believe it's executing the second command within python. Also, there are other aliases for Python on other computers. I know that I could just set py as an alias for python3, but I'm looking for a universal solution.
(Side note: It really bothers me that this is inconsistent. They should all just work.)
Eventually, I want a script that runs two things in parallel: the first is just npm run dev which has a --watch on it and has to continue to run, and the other is to cd docs/ then use python to host on localhost, then open chrome to localhost:8000, additional help would be much appreciated, still a beginner to Linux.
I also want to make a second command that runs npm run build, changes the second line in docs/sw.js from const dynamicCacheName = 'site-dynamic-vNUMBER'; to replace NUMBER with NUMBER+1.

Does this do what you're after? Assuming bash or similar:
( py -V && py -m http.server ) || ( python3 -V && python3 -m http.server )
On my box this does the following:
$ ( py -V && py -m http.server ) || ( python3 -V && python3 -m http.server )
py: command not found
Python 3.5.2
Serving HTTP on 0.0.0.0 port 8000 ...

For the second part of your question you could have a small bash script like this (I included the python or py check):
#!/bin/bash
npm run dev --watch &
chromium-browser "http://localhost:8000" 2>/dev/null &
cd docs/ || exit # Exit if the cd command fails
if command -v py -V &> /dev/null; then
echo "py"
py -m http.server
elif command -v python3 -V &> /dev/null; then
echo "python3"
python3 -m http.server
else
echo "Neither command found"
fi
& At the end of a command will run it in the background, as written above it will still display the output of the commands.
If you kill the script with CTRL+C it will kill everything, chromium, the python server and the npm run.

I actually wanted to post a full answer since I got what I was after from several answers and comments and chatting with a friend:
I installed the npm package concurrently and then added a local-dev npm script in my package.json:
"scripts": {
"dev-no-watch": "postcss src/styles.css -o docs/css/styles.css",
"dev": "postcss src/styles.css -o docs/css/styles.css --watch --verbose",
"build": "cross-env NODE_ENV=production postcss src/styles.css -o docs/css/styles.css && cleancss -o docs/css/styles.css docs/css/styles.css",
"localhost": "cd docs && ( py -V && py -m http.server ) || ( python3 -V && python3 -m http.server )",
"local-dev": "concurrently --kill-others \"npm run dev\" \"npm run localhost\""
},
Which runs both concurrently, and upon hitting ctrl+C, kills both as well.
I of course added localhost from the answer for the original question of detecting python version and running it correctly/failing gracefully, etc, and that works like a charm.
Hope this answer is helpful to someone in the future : )
(Now I just need to find out hos to edit a line of a file from the command line OS independently)

How to make packages using cli commands?

I tried using bitmex-market-maker python package.
Here's how to use this package.
Install: pip install bitmex-market-maker. It is strongly recommeded to use a virtualenv.
Create a marketmaker project: run marketmaker setup
I learned how to make my own command-line commands using python here.
To make my own commands, however, I had to type several terminal commands like
$ chmod +x marketmaker.py
$ mv marketmaker.py marketmaker
$ mkdir -p ~/bin
$ cp marketmaker ~/bin
$ export PATH=$PATH":$HOME/bin"
But I didn't type this commands at all but only installed bitmex-market-maker package using pip.
Can you help me figuring out how things work behind the scene?
I post marketmaker codes.
#!/usr/bin/env python
from market_maker import market_maker
market_maker.run()