Develop Reference

How to pass Python variable into bash commands?

I've seen some answers about subprocess.call and popen, but I have a list of commands and I think it's not a good idea to have multiple calls or etc. Also I don't want to have a separate script.sh with these commands.
My code looks like
bash_code=r'''
echo "/common_home/{context['nickname']} /tmp/back/{context['nickname']} none bind 0 0" | sudo tee --append /etc/fstab
sudo mkdir /tmp/{context['nickname']} /tmp/back/{context['nickname']}
'''
subprocess.run(['bash', '-c', bash_code], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
But it has much more line with {context['nickname']} and I don't know best way how to parse this variable into bash commands.

You can use a so-called "f-string" to replace the variable references with their values.
context = {'nickname': 'foobar'}
bash_code = f'''
echo "/common_home/{context['nickname']} /tmp/back/{context['nickname']} none bind 0 0" | sudo tee --append /etc/fstab
sudo mkdir /tmp/{context['nickname']} /tmp/back/{context['nickname']}
'''
print(bash_code)
subprocess.run(['bash', '-c', bash_code], stdout=subprocess.PIPE, stderr=subprocess.PIPE)
Value printed:
echo "/common_home/foobar /tmp/back/foobar none bind 0 0" | sudo tee --append /etc/fstab
sudo mkdir /tmp/foobar /tmp/back/foobar

bash command wont run in python3

I made a python3 script and i need to run a bash command to make it work. i have tried os.system and subprocess but neither of them fully work to run the whole command, but when i run the command by itself in the terminal then it works perfect. what am i doing wrong?
os.system("fswebcam -r 640x480 --jpeg 85 -D 1 picture.jpg &> /dev/null")
os.system("echo -e "From: abc#gmail.com\nTo: abc1#gmail.com\nSubject: package for ryan\n\n"package for ryan|uuenview -a -bo picture.jpg|sendmail -t")
or
subprocess.run("fswebcam -r 640x480 --jpeg 85 -D 1 picture.jpg &> /dev/null")
subprocess.run("echo -e "From: abc#gmail.com\nTo: abc1#gmail.com\nSubject: package for ryan\n\n"package for ryan|uuenview -a -bo picture.jpg|sendmail -t")
This is supposed to take a picture and email it to me. With os.command it gives an error "the recipient has not been specified "(even though it works perfect in terminal by itself) and with subprocess it doesnt run anything

Best Practice: Completely Replacing the Shell with Python
The best approach is to not use a shell at all.
subprocess.run([
'fswebcam',
'-r', '640x480',
'--jpeg', '85',
'-D', '1',
'picture.jpg'],
stdout=subprocess.DEVNULL, stderr=subprocess.DEVNULL)
Doing this with a pipeline is more complicated; see https://docs.python.org/3/library/subprocess.html#replacing-shell-pipeline, and many duplicates already on this site.
Second Choice: Using sh-compatible syntax
echo is poorly defined by the POSIX sh standard (the standard document itself advises against using it, and also fully disallows -e), so the reliable thing to do is to use printf instead.
Passing the text to be sent as a literal command-line argument ($1) gets us out of the business of figuring out how to escape it for the shell. (The preceding '_' is to fill in $0).
subprocess.run("fswebcam -r 640x480 --jpeg 85 -D 1 picture.jpg >/dev/null 2>&1",
shell=True)
string_to_send = '''From: abc#gmail.com
To: abc1#gmail.com
Subject: package for ryan
package for ryan
'''
p = subprocess.run(
[r'''printf '%s\n' "$1" | uuenview -a -bo picture.jpg | sendmail -t''',
"_", string_to_send],
shell=True)

awk command doesn't work under /bin/sh -c

I'm using this command:
docker ps | awk '$2 ~ /^selenium/ { print $1 }'
which works fine in the shell, but when running it with sh -c it doesn't work and I'm getting this error:
awk: cmd. line:1: ~ /^selenium/ { print }
awk: cmd. line:1: ^ syntax error
The full command that I want to is part from a Python script:
os.popen("nsenter -t 1 -m -u -n -i sh -c \"docker ps | awk -F/ '\''$2 ~ /^selenium/ { print $1 }'\''\"")
It's probably some escaping problem but I couldn't solve any.

You've got several levels of quoting there that are causing problems. If you start by using Python triple-quotes on the outside (''' or """), you can reduce the amount of escaping you need to perform.
That gets us:
os.popen('''nsenter -t 1 -m -u -n -i sh -c "docker ps | awk -F/ '\$2 ~ /^selenium/ { print \$1 }'"''')
We still need to escape the $ because otherwise they would be escaped by the outer shell (the one that os.popen calls to run the command).
I'm a little supicious of the -F/ in your awk command, but I assume you've tested this out and confirmed it turns the output you want.
By using the subprocess module, which doesn't call /bin/sh by default, you can further reduce the escaping (at the expense of having to tokenize the command line yourself):
import subprocess
subprocess.check_output([
'nsenter', '-t', '1', '-m', '-u', '-n', '-i',
'sh', '-c', "docker ps | awk -F/ '$2 ~ /^selenium/ { print $1 }'"
])

Do you have trying: ... awk -F/ '\\$2 ~ /^s...
(double backslash before $2)

subprocess command execution

What is the best way to execute the below command in Python in a single line?
echo $(readlink /sys/dev/block/$(mountpoint -d /))
Tried using individual os.system(cmd) by separating - "mountpoint -d /" first and taking the output and appending to "readlink /sys/dev/block/${0}".format(out.strip()) and doing an echo works. Tried using subprocess and subprocess.Popen and subprocess.check_output but it raises raise CalledProcessError
cmd = "echo $(readlink /sys/dev/block/$(mountpoint -d /))"

You have to call the subcommand separately. And you can use python methods to read the link:
import subprocess
import os
path = "/"
device = subprocess.run(["mountpoint", "-d", path], stdout=subprocess.PIPE, encoding="utf8").stdout.strip()
link = os.readlink("/sys/dev/block/" + device)
print(link)

You probably want to use something like the following:
cmd = "bash -c 'echo $(readlink /sys/dev/block/$(mountpoint -d /))'"
echo doesn't substitute $() blocks, that's what your shell does, so you have to call the shell. os.system(cmd) should work then.

Subprocess.call for cmd

I have the following command that works in the shell:
$ pv itunes20140910.tbz | sudo tar xpj -C /tmp
However, when I try and do it in python, it doesn't work:
>>> import subprocess
>>> import shlex
>>> cmd=shlex.split('pv itunes20140910.tbz | sudo tar xpj -C /tmp')
>>> subprocess.call(cmd)
pv: invalid option -- 'C'
Try `pv --help' for more information.
1
What am I doing wrong here, and what would be the correct command to run in python?

The above answers didn't have the net effect of what I was looking for (the progress bar), though the command would run without error. Here is what worked for me:
>>> import shlex, subprocess
>>> p1 = subprocess.Popen(shlex.split('pv /tmp/itunes20140910.tbz'), stdout=subprocess.PIPE) #Set up the echo command and direct the output to a pipe
>>> subprocess.Popen(shlex.split('sudo tar xpj -C /tmp'), stdin=p1.stdout) #send p1's output to p2

Use shell=True argument. Otherwise | cannot be interpreted.
subprocess.call('pv itunes20140910.tbz | sudo tar xpj -C /tmp', shell=True)