While fooling around with the OS module in the python interpreter (run inside the shell on a Linux system), I noticed it is possible to do the following:
>>> os.system("python") #execute run python command in enclosing shell
Producing the following output, indicating a new python REPL session:
Python 2.7.9 (default, Apr 2 2015, 15:34:55)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> #*blinking cursor*
From here, its possible to once again make a system call to start a new session of Python, from which I can make the system call again, etc. These Python environments seem independent from each other in that variables aren't shared amongst sessions and that the system calls are treated equivalently.
These sessions seem to run inside each other, at least to some extent, rather than in parallel, as evidenced by the outcome of the quit() function:
>>> os.system("python")
Python 2.7.9 (default, Apr 2 2015, 15:34:55)
[GCC 4.9.2] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> quit()
0
>>> quit()
0
>>> quit()
0
>>> quit()
shaked#shaked-ThinkPad-X220:~/Desktop$ python
Yet, a quick examination (>>> os.system("ps -e")) of the ongoing processes from the shell reveals a new sh for each python interpreter running:
11802 ? 00:00:00 kworker/u16:0
11803 pts/3 00:00:00 sh
11804 pts/3 00:00:00 python
11806 pts/3 00:00:00 sh
11807 pts/3 00:00:00 python
11810 pts/3 00:00:00 sh
11811 pts/3 00:00:00 python
11813 pts/3 00:00:00 sh
11814 pts/3 00:00:00 ps
Can anyone explain this (seemingly) strange behaviour in terms of underlying system processes? That is, are these sessions running in parallel or from within each other?
Apologies if this question has come up before, but I wasn't sure how others might have presented it.
When you use os.system, it performs the command in a subshell, something like /bin/sh -c 'yourcommand'*. Thus, it's completely sensible that you would get the behavior you describe. It's not different from running:
/bin/sh -c 'python'
in any shell.
*with mild differences on Windows, please read the documentation.
As per the documentation of os.system():
Execute the command (a string) in a subshell.
os.system() forks, executes a subshell passing the argument to this subshell and waits for the subshell to exit.
The subshell executes the command and waits for it to terminate, hence the process tree is:
- python
\-- sh
\-- python
\-- sh
\-- python
Python's os.system starts a new system process. When you pass "python" to it, it starts a new Python interpreter, that is independent from the one os.system was called from.
Related
I am using python3 and subprocess.Popen to spawn a process of bash and invoking the Python3 interpreter again through the standard interpreter.
bash -i states:
-s If the -s option is present, or if no arguments remain after
option processing, then commands are read from the standard
input. This option allows the positional parameters to be
set when invoking an interactive shell.
This is a minimized example but it mainly bakes down to the following code:
import subprocess
import sys
p = subprocess.Popen(["bash", "-s"], stdin=subprocess.PIPE,stderr=sys.stderr, stdout=sys.stdout)
p.stdin.write(b"python3\n")
p.stdin.flush()
print("Done")
The output is simply "Done". Any suggestions how I need to handle the stdin pipes in order to let the interactive shell pop up inside the newly executed python3 interpreter?
Actual output
% python3 test.py
Done
Expected output:
% python3 test.py
Python 3.10.8 (main, Oct 13 2022, 10:17:43) [Clang 14.0.0 (clang-1400.0.29.102)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>
I'm on a Ubuntu 20.04 system, and I'm using Python 3.8 to write a script that does multiple things using configurable lines of bash, but one of them is that it creates desktop shortcuts.
This single-line command creates a desktop shortcut, and works flawlessly when I execute it directly in my terminal:
echo "[Desktop Entry]"$'\n'"Type=Application"$'\n'"Name[en_US]=Documents"$'\n'"Exec=pcmanfm ~/Documents"$'\n'"Icon=system-file-manager" > ~/Desktop/1;
However, when I execute it in Python, like so:
foobar.py
rl = """echo "[Desktop Entry]"$'\n'"Type=Application"$'\n'"Name[en_US]=Documents"$'\n'"Exec=pcmanfm ~/Documents"$'\n'"Icon=system-file-manager" > ~/Desktop/1;"""
subprocess.run(rl, shell=True)
...instead of creating a desktop shortcut with the proper name, icon, and action, it creates an empty file that contains the following text:
0.txt:
[Desktop Entry]$\nType=Application$\nName[en_US]=Sign Out$\nExec=/home/agent/QaSC/login/login.bin$\nIcon=system-switch-user
Is there any particular reason why Python would be handling the newline characters differently than the bash shell does, and if so, how can I resolve this problem?
$'...' is a bash extension, but the default shell used when shell=True is specified is sh. Use the executable option to specify an alternate shell.
subprocess.run(rl, shell=True, executable='/bin/bash')
Since the argument to echo has quotes, it could contain literal newlines at the command line, and therefore also in the Python process. I see no reason to use the Bash extension $'\n' syntax.
$ echo "foo
> bar"
foo
bar
$ python
Python 3.8.10 (default, Mar 15 2022, 12:22:08)
[GCC 9.4.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> subprocess.run('echo "foo\nbar"', shell=True)
foo
bar
CompletedProcess(args='echo "foo\nbar"', returncode=0)
>>>
(It's also unclear why the Python code doesn't just write a file in the normal way, but I assume the command in the OP is essentially a placeholder.)
Using a Python script, I'd like to get the email of a person who last committed changes to a specific file /path/to/file.py.
Sounds easy, right? I just need to somehow parse the following
git log -n 1 --pretty=format:%ae -- /path/to/file.py
Package sh is my preferred choice. Unfortunately, in Python
import sh
print(str(sh.git.log('-n 1 --pretty=format:%ae -- /path/to/file.py')))
print(str(sh.git.log('-n', '1', '--pretty=format:%ae', '--', /path/to/file.py')))
both print - (press RETURN).
So maybe I'm messing something up with the arguments.
Otherwise, str(sh.git.status()) correctly returns On branch master ..., and some other tested commands work as expected.
How to solve this?
The - (press RETURN) output sounds like it's something printed by a pager.
Remember, every Git command may (depending on options, arguments, configuration settings, and other environmental details such as whether stdin is a tty) run its output through a pager. The pager used depends on your personal configuration. How that pager acts depends on the pager used and on the input data.
One simple workaround is to run git --no-pager <git-command> to tell Git not to use a pager, even if the configuration and environment suggest that Git should use a pager.
This should work:
print(str(sh.git.log("-n 1", "--pretty=format:%ae", "/path/to/file")))
At least this shows how it works on my machine:
$ git log -n 1 --pretty=format:%ae -- README.md
foo#bar.com
$ python3
Python 3.6.4 (default, Jan 25 2018, 15:54:40)
[GCC 4.2.1 Compatible Apple LLVM 9.0.0 (clang-900.0.39.2)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import sh
>>> print(str(sh.git.log("-n 1", "--pretty=format:%ae", "README.md")))
foo#bar.com
I can get a value n when I run a shell command using os.system in the python script, but I also need to sum it up to get a total number for subsequent computation in the python script,
total=0
for i in xrange(1,8):
os.system('n=$(qstat -n1 | grep -o node'+str(i)+' | wc -l) && echo $n')
Is it possible? Also is it possible to use python variable in shell command, something like
os.system('echo $total')
Use the shell's exportcommand:
$ export ABC=1 # Set and export var in shell
$ bash # Start another shell
$ echo $ABC # variable is still here
1
$ python # Start python still in the deeper shell
Python 2.7.2 (default, Oct 11 2012, 20:14:37)
[GCC 4.2.1 Compatible Apple Clang 4.0 (tags/Apple/clang-418.0.60)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> from os import environ # import environnement
>>> environ['ABC'] # And here's the variable again (it's a string because the shell doesn't have types)
'1'
You can subprocess module's check_output method like this
import subprocess
print sum(int(subprocess.check_output(["/bin/sh", "-c", "n=`expr {} + 1` && echo $n".format(i)])) for i in range(10))
Output
55
I'm trying to, from a command line, open an instance of konsole and run a python script. I'm trying:
konsole -hold -e 'python -i hello.py'
The behaviour I'm getting is that a persistent konsole opens, and I am dropped into python, but the script does not run.
Python 2.7.2+ (default, Oct 4 2011, 20:03:08)
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>>
What do I need to do to get the python script to run in the konsole window?
jsbueno's solution is the correct one. However, as described here, you can also do something like this ...
konsole --hold -e /bin/sh -c "python -i hello.py"
P.S. you'll need to specify --workdir (before the -e arg), or provide the full path to the python script, if it's not always in the initial working dir of konsole. But, you probably already knew that.
The problem is the way "konsole" uses the parameters after the -e switch - it seems like it simply pass them in a call that does not interpret the space separators as parameter separators.
However, if you don't put your call parameters inside quotes it will work - that is, simply:
konsole --hold -e python -i hello.py
(I just tested it here)