Develop Reference

Self invocation of interactive shell through Python3 with bash

I am using python3 and subprocess.Popen to spawn a process of bash and invoking the Python3 interpreter again through the standard interpreter.
bash -i states:
-s If the -s option is present, or if no arguments remain after
option processing, then commands are read from the standard
input. This option allows the positional parameters to be
set when invoking an interactive shell.
This is a minimized example but it mainly bakes down to the following code:
import subprocess
import sys
p = subprocess.Popen(["bash", "-s"], stdin=subprocess.PIPE,stderr=sys.stderr, stdout=sys.stdout)
p.stdin.write(b"python3\n")
p.stdin.flush()
print("Done")
The output is simply "Done". Any suggestions how I need to handle the stdin pipes in order to let the interactive shell pop up inside the newly executed python3 interpreter?
Actual output
% python3 test.py
Done
Expected output:
% python3 test.py
Python 3.10.8 (main, Oct 13 2022, 10:17:43) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>

Running a python program stored in a shell variable

I want write a shell script that will run a Python Code stored in a shell variable $CODE.
#!/bin/bash
python3 $CODE
But this does not work, is there any way to do this? Any help is appreciated.The program that will be run:
print("Hello World")
export CODE='print("Hello World")'

Use the -c option.
python3 -c "$CODE"

python3 <<< "$CODE"

[Python.Docs]: Command line and environment - -c <command> states:
Execute the Python code in command. command can be one or more statements separated by newlines, with significant leading whitespace as in normal module code.
Example:
[064bit prompt]> CODE='import sys;print("Hello World");print(sys.version)'
[064bit prompt]> python -c "${CODE}"
Hello World
3.8.10 (default, Jun 22 2022, 20:18:18)
[GCC 9.4.0]
But this comes with some limitations. I'd save the code in a script and run that instead.

Load JSON from stdin before starting Python REPL

By loading JSON from stdin before starting the REPL, I can pipe JSON outputs from other commands and work with the JSON data in Python.
I've tried something like and it worked:
kubectl get po -o json | cp /dev/stdin /tmp/data.json && python -i -c "import json; data = json.load(open('/tmp/data.json'))" && rm /tmp/data.json
Now I want to make it a script, so I write the following script:
#!/bin/bash -i
tmp_file=$(mktemp /tmp/ipyjson.XXXXXX)
cp /dev/stdin $tmp_file
python -i -c "import json; data = json.load(open('$tmp_file')); print(data)"
rm $tmp_file
And change the previous command to kubectl get po -o json | new_script.
But this time the REPL quits immediately after starting up, I have no idea what's the difference here. Maybe something about the interactive mode?
Edit:
I've also tried reading sys.stdin directly and met the same issue:
❯ k get po -o json | ipython -i -c "import sys, json; data = json.load(sys.stdin)"
Python 3.10.2 (main, Feb 11 2022, 14:13:20) [Clang 13.0.0 (clang-1300.0.29.30)]
Type 'copyright', 'credits' or 'license' for more information
IPython 8.4.0 -- An enhanced Interactive Python. Type '?' for help.
In [1]: Do you really want to exit ([y]/n)?
As you can see, ipython quited immediately with the prompt "Do you really want to exit ([y]/n)?".

Is it possible to cross reference bash and python variables in python script

I can get a value n when I run a shell command using os.system in the python script, but I also need to sum it up to get a total number for subsequent computation in the python script,
total=0
for i in xrange(1,8):
os.system('n=$(qstat -n1 | grep -o node'+str(i)+' | wc -l) && echo $n')
Is it possible? Also is it possible to use python variable in shell command, something like
os.system('echo $total')

Use the shell's exportcommand:
$ export ABC=1 # Set and export var in shell
$ bash # Start another shell
$ echo $ABC # variable is still here
1
$ python # Start python still in the deeper shell
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from os import environ # import environnement
>>> environ['ABC'] # And here's the variable again (it's a string because the shell doesn't have types)
'1'

You can subprocess module's check_output method like this
import subprocess
print sum(int(subprocess.check_output(["/bin/sh", "-c", "n=`expr {} + 1` && echo $n".format(i)])) for i in range(10))
Output
55

python interative mode does not work when reading from std in

Given the following python script....
$ cat readStdin.py
#!/usr/bin/python
import sys
var = "".join(sys.stdin.readlines()).rstrip()
print var
... I get the follwing output:
$ echo hello | python -i readStdin.py
hello
>>>
$
... in other words it does not hang in the python console, but goes back to bash. Does anyone out there know how to make it stay in the python console???

Consider this -
$ echo print 4*2 | python -i
Python 2.7.2 (default, Jun 20 2012, 16:23:33)
Type "help", "copyright", "credits" or "license" for more information.
>>> 8
>>>
$
Echo produces print 4*2. Python even in interactive mode, considers this as input to be interpreted. Hence we see the 8 there. After this, the interpreter encounters an EOF, so it quits. Consider what you press to quit the interpreter - Ctrl+d or ^D. This is just another way to produce EOF on *nix.