Develop Reference

How to perform a cd command and execute additional command in a new terminal from a python script?

I am trying to run a python script on ubuntu18, which simply will open a new terminal/tab,navigate to a specific folder and then execute a command. But this simple task is looking very daunting due to my lack of knowledge.
Expected:
In the python script
$cd /home/metabase
$java -jar metabase.jar
My code:
try1:
cmd = "gnome-terminal --tab 'cd /home/metabase/java -jar metabase.jar; read'"
os.system(cmd)
New tab opens but nothing happens
try2:
subprocess.call(['cd /home/metabase/', 'java -jar metabase.jar'])
Error:No such file or directory
I tried many other combinations. But results in either new tab not opening or new tab opens but in the same directory and does nothing.
I did some reading on the problem. It seems like i am creating these subprocess and therefore when i do the CD, it does nothing. Anyways,i looked into many similar stackoverflow threads but i am still lost. Any direction would be appreciated. Thank you

The command to open a new gnome-terminal window with a bash command is:
gnome-terminal -- bash -c 'your command'
In your case:
gnome-terminal -- bash -c 'cd /home/metabase; java -jar metabase.jar; read'
Make sure this works from a shell first. Then you can invoke it from Python:
subprocess.call(["gnome-terminal", "--", "bash", "-c", "cd /home/metabase; java -jar metabase.jar; read"])

Why do you want to change the directory ? If you want you can do it directly :
os.system('java -jar /home/metabase/metabase.jar') which will return the process exit value , 0 means success.
Keep in mind that os.system will execute the command passed as a string in a sub-shell.
If you do not want to spawn a new shell, you can use:
subprocess.call(['java', '-jar', '/home/metabase/metabase.jar']) in this way there is no system shell started up, so the first argument must be a path to an executable file.

Python script runs on command line but not from .sh file

I'm attempting to create a .sh file to batch a number of runs of a neural network on Python whilst on holidays.
At the moment I have been calling this from the command line:
python neural_network_trainer.py [args]
I now have a .sh script written:
#!/bin/bash
python neural_network_trainer.py [args]
# Repeated with varied args
That I am attempting to call in the same terminal as the original command line was running:
./august_hols.sh
I get the following error:
File "/data/Python-3.6.9/lib/python3.6/site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Where the Python install is in /data (for reasons).
Running which on the command line reports the correct Python directory set via an alias in ~/.bashrc:
alias python=/data/Python-3.6.9/bin/python3
But running which between the Bash shebang and the first python call reports /bin/python.
I've attempted to set the alias again at the start of the .sh script to no avail. I'm scratching my head as this is exact process I have used elsewhere, albeit not on this precise PC. I can copy the exact command from the top of the bash file into the terminal and it runs fine, try and call ./august_hols.sh and get the above Python error.
Where is Bash getting that path from, and why is it not using my expected route through ~/.bashrc?

Bash sub-shell does not inherit alias in the main shell
You can source the script (run in the main shell), instead of execute it (run in the sub-shell)
source script.sh
EDIT:
Solution 2:
Run bash as the login shell so ~/.bashrc is executed, so your alias is loaded before your script.
The subshell needs to be interactive to enable alias, because alias is enabled by default only for interactive shell, but script is non-interactive by default.
bash --login -i script.sh
Solution 3:
Similar to above, except alias is enabled explicitly
bash --login -O expand_aliases script.sh

Have you tried:
python=/data/Python-3.6.9/bin/python3 ./[your_bash].sh

In your .sh
Do this
#!/usr/bin/env bash
export PATH=/data/Python-3.6.9/bin:$PATH
exec python neural_network_trainer.py "$#"
Aliases are tricky.

A maybe more nasty solution
mapfile < <(declare -p | grep -m 1 BASH_ALIASES) && bash script.sh "${MAPFILE[#]}"
within your script you will need
shopt -s expand_aliases
eval $1
echo ${BASH_ALIASES[python]}
python --version

How about this:
#!/bin/bash
/data/Python-3.6.9/bin/python3 neural_network_trainer.py [args]
# Repeated with varied args

crontab won't run os.system python command

Using ubuntu's 16.04 crontab and #reboot to run python3 script. The script runs properly on reboot as I see the logged output. However, my script's os.system command is not running. It runs fine if ran outside of crontab. My scripts are all executable.
crontab -l output:
SHELL=/bin/bash
#reboot nohup /usr/bin/python3 -u /home/path/scheduler.py >> /path/log.out &
scheduler.py code:
#...(check if web server is running...if not restart)
os.system('nohup /usr/bin/python3 -u /path/webserver/main.py &')
print('this function ran')
When I logged the output of the os.system command , there was no output.
As a side note, I am running python schedule commands to check the general health of a webserver. crontab doesn't seem to be the right tool for this so I just use crontab to start my python scheduler on reboot.
I am using flask as the webserver, and would use gunicorn and systemctrl if I could get it to work... but it didn't so this is my workaround.

The point is that, the command called by os.system is not in default path.
For example, tcpdump is not in /usr/bin/.
So, you can solve the problem by adding the full path of the command.

I was facing the same issue when we try to run python script directly in crontab it just by passes the os.system() commands.
Make launcher.sh:
#!bin/bash
cd /home/pi/
sudo python example.py
Then, make your script executable:
chmod 755 launcher.sh
And at last, add your script to crontab:
crontab -e
and add this line at the end:
#reboot sh /home/pi/launcher.sh
(I set the program to run at each reboot)

Execute python program with bash script

I have following bash script:
export MYCONFIG_CONFIG=TEST
/home/bmwant/projects/test/venv/bin/python3 /home/bmwant/projects/test/script.py
My python script needs environment variable which I have set, but running this script as
bash run_python.sh
shows an error
/home/bmwant/projects/test/venv/bin/python3: can't open file 'home/bmwant/projects/tes': [Errno 2] No such file or directory
What is wrong? I have set script as executable with chmod u+x run_python.sh

Have made script.py executable?
chmod u+x /home/bmwant/projects/test/script.py
Try this script:
#!/bin/bash
export MYCONFIG_CONFIG=TEST
pushd /home/bmwant/projects/test/venv/bin/
./python3 /home/bmwant/projects/test/script.py
popd
You can run it via:
./run_python.sh

So, the problem was in file format. I have created it in Windows and then copy via FileZilla to remote server where I was trying to run it. So just run
apt-get install dos2unix
dos2unix run_python.sh
bash run_python.sh
and all will work well. Thanks #Horst for the hint.

Invoking shell script from a python script using root privileges

I'm trying to invoke a shell script shell_script.sh from a python script (python_script.py) using the call command. The shell_script.sh invokes a executable that requires root access to execute.
The python_script.py invokes shell_script.sh using subprocess.call().
See below:
subprocess.call(['/complete_path/shell_script.sh', 'param1', 'param2',
'param3'], shell=True)
When I try to execute the python script python_script.py it gives me permission denied.
I've tried different ways.
a) Invoke python with sudo - sudo python python_script.py
b) Invoke sudo into inside the call method - subprocess.call(['sudo' '/complete_path/shell_script.sh', 'param1', 'param2',
'param3'], shell=True)
What's the best way to resolve this.
Thanks.

I'd put logic in the python_script.py to check its UID and fail if is not executed as root. if os.getuid() != 0:. That will ensure it only runs as root, ether by a root login, or sudo.
If you're getting permission denied when trying to execute the python_script.py, you need to set the execute bit on it. chmod +x python_script.py