Develop Reference

Catching full output from subprocess script using subprocess.run in Python

I'm trying to set up a master.py script that would call a set of individual Python scripts. For sake of simplicity, let's only consider one very basic such script: counting.py which just counts up with 1 second pauses:
# counting.py file
import time
for i in range(10000):
print(f"Counting: {i}")
time.sleep(1)
In the master.py, I use subprocess.run() to call counting.py which is located in the same folder. In the snippet below, sys.executable returns the path to the Python executable in the virtual environment. I also use the multiprocessing module to control timeouts: if counting.py runs longer than 60 seconds, process must be terminated. The code in master.py is as follows:
import subprocess
import multiprocessing
import sys
from pathlib import Path
def run_file(filename):
cmd = [sys.executable, str(Path('.').cwd() / f'{filename}.py')]
try:
result = subprocess.run(cmd, shell=True, text=True, stdout=subprocess.PIPE).stdout
print("Subprocess output:\n", result)
except Exception as e:
print(f"Error at {filename} when calling the command:\n\t{cmd}")
print(f"Full traceback:\n{e}")
if __name__ == '__main__':
p = multiprocessing.Process(target=run_file, args=("counting",))
p.start()
# Wait for 60 seconds or until process finishes
p.join(60)
if p.is_alive():
print("Timeout! Killing the process...")
p.terminate()
p.join()
The issue: Even though the code itself runs properly, I am unable to log any of the output while running master.py. Based on the documentation of the subprocess module, I had the impression that the shell and stdout arguments of subprocess.run() account for this exactly. I would like to see the same output as the one I get when only running counting.py, i.e.:
Counting 1
Counting 2
Counting 3
...

Run cmd file using python

I have a cmd file "file.cmd" containing 100s of lines of command.
Example
pandoc --extract-media -f docx -t gfm "sample1.docx" -o "sample1.md"
pandoc --extract-media -f docx -t gfm "sample2.docx" -o "sample2.md"
pandoc --extract-media -f docx -t gfm "sample3.docx" -o "sample3.md"
I am trying to run these commands using a script so that I don't have to go to a file and click on it.
This is my code, and it results in no output:
file1 = open('example.cmd', 'r')
Lines = file1.readlines()
# print(Lines)
for i in Lines:
print(i)
os.system(i)

You don't need to read the cmd file line by line. you can simply try the following:
import os
os.system('myfile.cmd')
or using the subprocess module:
import subprocess
p = subprocess.Popen(['myfile.cmd'], shell = True, close_fds = True)
stdout, stderr = proc.communicate()
Example:
myfile.cmd:
#ECHO OFF
ECHO Grettings From Python!
PAUSE
script.py:
import os
os.system('myfile.cmd')
The cmd will open with:
Greetings From Python!
Press any key to continue ...
You can debug the issue by knowing the return exit code by:
import os
return_code=os.system('myfile.cmd')
assert return_code == 0 #asserts that the return code is 0 indicating success!
Note: os.system works by calling system() in C can only take up to 65533 arguments after a command (so it is a 16 bit issue). Giving one more argument will result in the return code 32512 (which implies the exit code 127).

The subprocess module provides more powerful facilities for spawning new processes and retrieving their results; using that module is preferable to using this function (os.system('command')).
since it is a command file (cmd), and only the shell can run it, then shell argument must set to be true. since you are setting the shell argument to true, the command needs to be string form and not a list.
use the Popen method for spawn a new process and the communicte for waiting on that process (you can time it out as well). if you whish to communicate with the child process, provide the PIPES (see mu example, but you dont have to!)
the code below for python 3.3 and beyond
import subprocess
try:
proc=subprocess.Popen('myfile.cmd', shell=True, stderr=subprocess.PIPE, stdout=subprocess.PIPE)
outs, errs = proc.communicate(timeout=15) #timing out the execution, just if you want, you dont have to!
except TimeoutExpired:
proc.kill()
outs, errs = proc.communicate()
for older python versions
proc = subprocess.Popen('myfile.cmd', shell=True)
t=10
while proc.poll() is None and t >= 0:
print('Still waiting')
time.sleep(1)
t -= 1
proc.kill()
In both cases (python versions) if you dont need the timeout feature and you dont need to interact with the child process, then just, use:
proc = subprocess.Popen('myfile.cmd', shell=True)
proc.communicate()

Detemine if python script is called as subprocess and passing args

I have a script that gives the option to run a second script after completion. I am wondering if there is a good way for the second script to know if it was run on its own or as a subprocess. If it was called as a subprocess, pass args into the second script.
The end of the first script is below:
dlg = wx.MessageDialog(None, "Shall we format?",'Format Files',wx.YES_NO | wx.ICON_QUESTION)
result = dlg.ShowModal()
if result == wx.ID_YES:
call("Threading.py", shell=True)
else:
pass
The second script is a standalone script that takes 3 files and formats them into one. The args would just set file names in the second script.

So I would retrieve the parent process pid with os.getppid() and pass then this to the subprocess as arguments using Popen:
(parent.py)
#!/usr/bin/env python
import sys
import os
from subprocess import Popen, PIPE
output = Popen(['./child.py', str( os.getppid() )], stdout=PIPE)
print output.stdout.read()
and
(child.py)
#!/usr/bin/env python
import sys
import os
parent_pid = sys.argv[1]
my_pid = str(os.getppid())
print "Parent is %s child is %s " % ( parent_pid, my_pid )
So when you call the child from the parent
$ ./parent.py
Parent is 72297 child is 72346
At this point is easy to make a comparison and check the pid.

How to get application name from python?

In Wondows7 64.
I want to get application name from python.
here is code that I can get Processes name,but it's not I want.
from psutil import process_iter
process = filter(lambda p: p.name() == "main2.exe", process_iter())

On Windows, you could make a system call:
import subprocess
cmd = 'WMIC PROCESS get Caption'
proc = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE)
for line in proc.stdout:
print line
You could also get the 'Commandline' or 'Processid' if the 'Caption' isn't enough.

Maybe you mean this, which gives you the name of the Python script that is running:
import os
print(os.path.splitext(os.path.basename(__file__))[0])

Using input redirection with Popen in python

I need to use stream redirectiton in Popen call in python to use bat file with wine. I need make this:
wine32 cmd < file.bat
It works when I run it manually from terminal, however when I try to call it from python:
proc = Popen('wine32 cmd < file.bat',stdout = PIPE)
I got error: No such file or directory
How to manage with that?
Thanks

Try this:
import sys
#...
with open('file.bat', 'r') as infile:
subprocess.Popen(['wine32', 'cmd'],
stdin=infile, stdout=sys.stdout, stderr=sys.stderr)
Make sure that each argument to wine32 is a separate list element.

maybe you can check this thread.. https://stackoverflow.com/a/5469427/3445802
from subprocess import Popen
p = Popen("batch.bat", cwd=r"C:\Path\to\batchfolder")
stdout, stderr = p.communicate()