Develop Reference

python, python3, python3.7 files don't work in a venv (can't be activated)

I am trying to run some code inside a server. In that server, we use docker images to create notebooks inside directories, with commands like:
docker run -it --gpus "device=1" -p 8886:8886 -v /folder/directory:/workspace/work --name container-name --shm-size=60g --ulimit memlock=-1 --ulimit stack=67108864 --rm imageid jupyter notebook --port=8886 --ip=0.0.0.0 --allow-root --no-browser
Once created the notebook with an image, we have two different environments with two different python versions in the folder that were designed to execute the code inside /folder/directory: venv3.6 and venv3.7.
Even if I didn't create them, I am confident that the environments worked at some point (there are checkpoints obtained from the execution of the code by a colleague that worked on it before me). However, it must have been messed up with at some point, maybe after some modifications on the libraries of the docker image.
The problem is that, whenever I try to activate venv3.7 by using source ./venv3.7/bin/activate and run a script with python script_name.py, the python version that is executed is not 3.7, but rather 3.6.10. When going into /venv3.7/bin/activate and trying to access or download the python, the python3 or the python3.7 files, they cannot be accessed, moved or activated (i.e., if I enter /venv3.7/bin/python3.7 on the terminal, I obtain the file not found error).
When the environment is activated:
root#XXXX:/workspace/work/path# which python
/opt/conda/bin/python
root#XXXX:/workspace/work/path# source ./venv3.7/bin/activate
(venv3.7) root#XXXX:/workspace/work/path#
Following this stackoverflow post, I make the following comprobations
(venv3.7) root#XXXX:/workspace/work/path# python -V
Python 3.6.10 :: Anaconda, Inc.
(venv3.7) root#XXXX:/workspace/work/path# echo $PATH
/workspace/work/path/venv3.7/bin:/usr/local/nvm/versions/node/v15.0.1/bin:/opt/conda/bin:/opt/cmake-3.14.6-Linux-x86_64/bin/:/usr/local/mpi/bin:/usr/local/nvidia/bin:/usr/local/cuda/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/ucx/bin:/opt/tensorrt/bin
(venv3.7) root#XXXX:/workspace/work/path# which python
/opt/conda/bin/python
(venv3.7) root#XXXX:/workspace/work/path# alias
alias egrep='egrep --color=auto'
alias fgrep='fgrep --color=auto'
alias grep='grep --color=auto'
alias l='ls -CF'
alias la='ls -A'
alias ll='ls -alF'
alias ls='ls --color=auto'
Which shows that the path is added correctly and there is no alias for python that could be messing with the activation but, still, python command uses the version from /opt/conda/bin/python instead of /workspace/work/path/venv3.7/bin
I also have checked that the path VIRTUAL_ENV in activate script (venv3.7/bin/activate) is correct.
I noticed that the directory: /venv3.7/pyvenv.cfg contains:
home = /usr/bin
include-system-site-packages = false
version = 3.7.5
And when I go to the directory /usr/bin, which should contain the python in which the environment is based, it only has python2.7 files. Could that mean that, when the first directory in $PATH is followed, no valid version of Python is found?
My guess is that the python (python, python3, python3.7) files were symlinks that were broken because the python version changed in /usr/bin. However, I don't want to risk to update the version of python in that directory, because it would probably change the default python in /opt/conda/bin/python instead, and I don't know much about docker images. Do you think it would work? In that case, how would I do it?
As additional info, the python files inside venv3.6/bin seems to work well (it can be executed and copied), but maybe because /venv3.6/pyvenv.cfg leads to the default python instead (in /opt/conda/bin/python). Also, after asking the original creator of the code, she doesn't know how to solve this issue either.
I need the environment to work, and recreating it is problematic, since many libraries were downloaded from different places (it was delicate work).
What do you suggest?
EDIT
I have tried recreating the environment with the python version I need (3.7.5). Do you know of an easy way to install the same libraries than in the other environment, considering that I can't activate it?
I was thinking to use the folders with the libraries located in /venv3.7/lib, but It is not straight forward. Any idea on how to do it?
Also, would you recommend me to create the new environment with virtualenv (to have a separate python version) or, rather, with anaconda?
Thank you so much for reading me!

After checking the python3.7 file in the environment:
root#XXXX:/# cd workspace/work/path/venv3.7/bin
root#XXXX:/workspace/work/path/venv3.7/bin# stat python
File: python -> python3.7
Size: 9 Blocks: 0 IO Block: 4096 symbolic link
Device: XXXX Inode: XXXX Links: 1
Access: (XXXX) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2022-12-06 10:31:18.165001523 +0000
Modify: 2022-05-20 12:28:37.481538688 +0000
Change: 2022-05-20 12:28:37.481538688 +0000
Birth: -
root#XXXX:/workspace/work/path/venv3.7/bin# stat python3.7
File: python3.7 -> /usr/bin/python3.7
Size: 18 Blocks: 0 IO Block: 4096 symbolic link
Device: XXXX Inode: XXXX Links: 1
Access: (XXXX) Uid: ( 0/ root) Gid: ( 0/ root)
Access: 2022-12-06 10:31:18.165001523 +0000
Modify: 2022-05-20 12:28:37.481538688 +0000
Change: 2022-05-20 12:28:37.481538688 +0000
Birth: -
It became obvious that, as stated in the post, /usr/bin should be the directory where python3.7 should be installed. That means the problem could be solved by installing it in that folder.
As I didn't know that was the default folder for the Python installation, I tried installing python from source as exposed in several guides. However, even if now the environment started accessing python3.7 in the folder, that installation didn't work either.
So I just tried apt-get install python3.7. It took around 10 seconds and, when I tried the code again, it worked!
Next time, when your environments fails because the wrong python version is executed, and the aliases and $PATH are right (see this post for more details), just remember to check where the python files in the environment point to and verify that the python installation is correct!
I hope this is useful for you.

Issues trying to install AirFlow locally

I'm new at airflow and I'm trying to install locally, following the instructions on the link below:
https://airflow.apache.org/docs/apache-airflow/stable/start/local.html
I'm running this code (as mentioned on the link):
# Airflow needs a home. `~/airflow` is the default, but you can put it
# somewhere else if you prefer (optional)
export AIRFLOW_HOME=~/airflow
# Install Airflow using the constraints file
AIRFLOW_VERSION=2.2.5
PYTHON_VERSION="$(python --version | cut -d " " -f 2 | cut -d "." -f 1-2)"
# For example: 3.6
CONSTRAINT_URL="https://raw.githubusercontent.com/apache/airflow/constraints-${AIRFLOW_VERSION}/constraints-${PYTHON_VERSION}.txt"
# For example: https://raw.githubusercontent.com/apache/airflow/constraints-2.2.5/constraints-3.6.txt
pip install "apache-airflow==${AIRFLOW_VERSION}" --constraint "${CONSTRAINT_URL}"
# The Standalone command will initialise the database, make a user,
# and start all components for you.
airflow standalone
# Visit localhost:8080 in the browser and use the admin account details
# shown on the terminal to login.
# Enable the example_bash_operator dag in the home page
and getting this error:
File "C:\Users\F43555~1\AppData\Local\Temp/ipykernel_12908/3415008398.py", line 3
export AIRFLOW_HOME=~/airflow
^
SyntaxError: invalid syntax
Someone knows how to deal with it?
I'm running on windows 10, vs code (jupyter notebook).
Tks!

Airflow is only supported on Linux and it looks like you're trying to run this on a windows machine.
If you want to install Airflow on Windows you'll need to use something like Windows Subsystem for Linux (WSL) or Docker. There are some examples around which show you how to do this on WSL (and loads using docker) - Here is one of them with WSL.

how to run multiple fedora commands in python

So I'm trying to have Python run multiple commands to install programs and enable SSH to setup my Linux computer. I would type all this in, but I'll be doing this to more devices, so I figured why not put in a Python script, but so far it's easier said than done. I did a boatload of research on this and I can't find anything like this.
So here's what I got so far.
--import subprocess
--SSH = "systemctl enable sshd"
--payload = "nmap" # it'll be one of a few I'll be installing
--subprocess.call(["sudo", "yum", "install", "-y", payload])
--subprocess.call(["sudo", SSH])
The first part of this works perfectly. It asks for my password it'll update and install nmap. But for some reason the command "systemctl enable sshd" seems to always throw it off. I know the command works because I can just type it out and it'll work just fine by itself, but for some reason it won't work through this script. I've used subprocess.run as well. What am I missing here?
Here's the error that I get:
--sudo: systemctl start sshd: command not found

What you want is Ansible.
Ansible uses SSH to connect to list of machines and perform configuration tasks. Tasks are described in YAML, which is readable and scale. You can have playbooks and ad hoc commands. For example ad hoc to install package will be
ansible -i inventory.file -m yum -a "name=payload state=present"
In a playbook will look like Install and enable openssh-server
---
- hosts: all # Single or group of hosts from inventory file
become: yes # Become sudo
tasks: # List of tasks
- name: Install ssh-server # Description free text
yum: # Module name
name: openssh-server # Name of the package
state: present # State " state: absent will uninstall the package"
- name: Start and enable service # Description of the task free text
service: # Service
name: sshd # Name of the service
state: started # Started or Stopped
enabled: yes # Start the service on boot
- name: Edit config file sshd_config # Description of the task
lineinfile: # Name of the module
path: /etc/sshd/sshd_config # Which file to edit
regex: ^(# *)?PasswordAuthentication # Which line to edit
line: PasswordAuthentication no # Whit what to change it
Ansible have great documentation https://docs.ansible.com/ in a few days you will be up to speed.
Best regards.

install z3 in a remote linx server control without being root

I am trying to install z3 on a remote server that I am not the root of. I followed the steps to the point where I have this message:
Z3 was successfully built.
Z3Py scripts can already be executed in the 'build/python'
Use the following command to install Z3 at prefix /usr.
sudo make install
Since it says that Z3py scripts can already be executed, is the next command necessary? if so, how can I execute it without being root. Is there an alternative?
I have changed the prefix to a directory that I have write access for. Again, it installed z3 and z3py successfully but then it says:
Use the following command to install Z3 at prefix /z3/z3-master.
sudo make install
when I use make install this is what I get:
mkdir: cannot create directory ‘/z3’: Permission denied
Makefile:4462: recipe for target 'install' failed
make: *** [install] Error 1

Configure it like this: python scripts/mk_make.py --prefix=/a/place/with/write/access

Errno 13 while running docker-compose up

I'm building an application using django and I wanted to add docker to this project.
I'm trying to run
sudo docker-compose up
Which gives me this output:
ERROR: .IOError: [Errno 13] Permission denied: './docker-compose.yml'
I checked the permissions using GUI. Everything is fine.
I'm trying to run my app from an mounted drive. I also tested it on other drives. The only drive this problem does not appear is my main drive running Ubuntu 18.04.
Looking forward to some answers

I found a working solution.
Don't use the snap installation and do this instead (tested Ubuntu 20.04)
apt install docker.io docker-compose

adding the directory where I am running my docker-compose.yml using the apparmor reconfigure tool:
$ sudo dpkg-reconfigure apparmor

You need to update your AppArmor configuration :
Snap Dockers are heavily controlled with AppArmor.
To diagnose if it is really the case, check the last lines of the syslog after you triggered the error :
dmesg | grep docker-compose
You should see a snap.docker that was denied:
kernel: [ ] audit: type=1400 audit(....):
apparmor="DENIED" operation="exec" profile="snap.docker.dockerd"
name="/bin/kmod" pid=7213 comm="exe" requested_mask="x"
denied_mask="x" fsuid=0 ouid=0
To correct this, just go to apparmor config's tunables :
cd /etc/apparmor.d/tunables
And edit HOMEDIRS variables in the 'home' file, for example from :
#{HOMEDIRS}=/home/
to
#{HOMEDIRS}=/home/ /media/aUser/Linux/
hope that helps.

All the other answers didn't work for me.
docker --version
Docker version 20.10.17, build 100c701
docker-compose -v
docker-compose version 1.29.2, build unknown
Instead of
docker-compose up
please use
docker compose up