I'm generating a text file that is later processed by an external program. This must be done 1000 times, for this, i use a subprocess.call() inside a loop for each text file i want to process.
The first call of subprocess.call() works perfectly.
The second call fails and the python program exits with a []Stop.
- There is no debug output.
- Both remain stopped, but in the process list
I have tried subprocess.call(), subprocess.Popen() and the outcome is the same. I have tried to run it with the same textfile as the first execution and it also fails, so the culprit is the subprocess.call() function for sure.
This is the line that calls the external program
subprocess.call(['/bin/bash', '-i', '-c', 'nucplot textfile.txt']);
The program is a simple binary file, but it must use the ENV variables of its installation to work properly, hence the usage of /bin/bash with those options. If I try using a shell, it doesn´t work.
Is there anything else i need to do after calling subprocess.call() in order for it to flush its internal stuff?
Try using subprocess.check_output
https://docs.python.org/3/library/subprocess.html#subprocess.check_output
_ = subprocess.check_output(['/path/to/nucplot', '-i', '-c', 'textfile.txt'])
Related
I am making a virtual assistant that can start several programs using subprocess.Popen("path/to/app.exe"). But when I exit the python program, all of processes are killed. I want the processes (the applications started with Popen) to be independent and remain alive after main process is killed.
I have tried adding start_new_session=True as argument in subprocess.Popen() as some posts have suggested, but it's still not working.
I don't think showing the code is necessary, but still, here you go.
app_path = r'C:\Users\myusername\AppData\Local\Discord\app-1.0.9001\discord.exe'
subprocess.Popen(app_path) # also tried adding start_new_session=True as argument
Since you're on Windows, you can call the start command, which exists for this very purpose: to run another program independently of the one that starts it.
The start command is provided by the command-line interpreter cmd.exe. It is not an executable: there is no start.exe. It is a "shell command" (in Linux terminology), which is why shell=True must be passed when creating the subprocess.
You won't be able to communicate with the subprocess started in this way, that is, not via the pipe mechanism provided by the subprocess module. So instead of Popen, you may just use the convenience function run:
from subprocess import run
app = 'notepad'
run(['start', app], shell=True)
The example starts the Notepad editor (instead of Discord in the question) in order to make it easier to reproduce.
In cases where the full path to the app contains spaces, we can either call start like so
app = r'C:\Program Files (x86)\Microsoft\Edge\Application\msedge.exe'
run(f'start "" "{app}"', shell=True)
using the Edge browser in this example, or pass the directory separately:
folder = r'C:\Program Files (x86)\Microsoft\Edge\Application'
app = 'msedge.exe'
run(['start', '/d', folder, app], shell=True)
This is needed because start treats a single argument as the window title if that argument is in quotes. And only if not does it treat it as the command. See "Can I use the start command with spaces in the path?" (on SuperUser) for more details.
Answered here: https://stackoverflow.com/a/34718600/4355695
subprocess.Popen(full_command, shell=True, close_fds=True)
(In my linux system I have to put shell=True if I'm passing a full command as string instead of a split-up array of arguments. In windows it may differ, idk)
I am trying to use output of external program run using the run function.
this program regularly throws a row of data which i need to use in mine script
I have found a subprocess library and used its run()/check_output()
Example:
def usual_process():
# some code here
for i in subprocess.check_output(['foo','$$']):
some_function(i)
Now assuming that foo is already in a PATH variable and it outputs a string in semi-random periods.
I want the program to do its own things, and run some_function(i)every time foo sends new row to its output.
which boiles to two problems. piping the output into a for loop and running this as a background subprocess
Thank you
Update: I have managed to get the foo output onto some_function using This
with os.popen('foo') as foos_output:
for line in foos_output:
some_function(line)
According to this os.popen is to be deprecated, but I am yet to figure out how to pipe internal processes in python
Now just need to figure out how to run this function in a background
SO, I have solved it.
First step was to start the external script:
proc=Popen('./cisla.sh', stdout=PIPE, bufsize=1)
Next I have started a function that would read it and passed it a pipe
def foo(proc, **args):
for i in proc.stdout:
'''Do all I want to do with each'''
foo(proc).start()`
Limitations are:
If your wish t catch scripts error you would have to pipe it in.
second is that it leaves a zombie if you kill parrent SO dont forget to kill child in signal-handling
I'm running an application from within my code, and it rewrites files which I need to read later on in the code. There is no output the goes directly into my program. I can't get my code to wait until the subprocess has finished, it just goes ahead and reads the unchanged files.
I've tried subprocess.Popen.wait(), subprocess.call(), and subprocess.check_call(), but none of them work for my problem. Does anyone have any idea how to make this work? Thanks.
Edit: Here is the relevant part of my code:
os.chdir('C:\Users\Jeremy\Documents\FORCAST\dusty')
t = subprocess.Popen('start dusty.exe', shell=True)
t.wait()
os.chdir('C:\Users\Jeremy\Documents\FORCAST')
Do you use the return object of subprocess.Popen()?
p = subprocess.Popen(command)
p.wait()
should work.
Are you sure that the command does not end instantly?
If you execute a program with
t = subprocess.Popen(prog, Shell=True)
Python won't thrown an error, regardless whether the program exists or not. If you try to start an non-existing program with Popen and Shell=False, you will get an error. My guess would be that your program either doesn't exist in the folder or doesn't execute. Try to execute in the Python IDLE environment with Shell=False and see if you get a new window.
From what I can tell, execv overtakes the current process, and once the called executable finishes, the program terminates. I want to call execv multiple times within the same script, but because of this, that cannot be done.
Is there an alternative to execv that runs within the current process (i.e. prints to same stdout) and won't terminate my program? If so, what is it?
Yes, use subprocess.
os.execv* is not approporiate for your task, from doc:
These functions all execute a new program, replacing the current
process; they do not return. On Unix, the new executable is loaded
into the current process, and will have the same process id as the
caller.
So, as you want the external exe to print to the same output, this is what you might do:
import subprocess
output = subprocess.check_output(['your_exe', 'arg1'])
By default, check_output() only returns output written to standard output. If you want both standard output and error collected, use the stderr argument.
output = subprocess.check_output(['your_exe', 'arg1'], stderr=subprocess.STDOUT)
The subprocess module in the stdlib is the best way to create processes.
I have a bash script myscript.sh:
#!/bin/bash
while read line; do
myprog.py
done
calling a python program myprog.py
#!/usr/bin/env python
import subprocess
output = subprocess.check_output(['ssh', 'user#host', 'cmd'])
The ssh command that is called by subprocess executes without error, the output is correct. But when called like this the loop in myscript.sh only runs through the first line of input and then exits with status 0. If I replace the subprocess.check_output(...) call with a subprocess.Popen(...) and don't subsequently call Popen.wait() then the outer loop works as expected and the output from the ssh command is dumped to standard out some time after any output from the bash script. With the Popen.wait() behavior is the same as with check_output: bash loop only goes through one iteration before exiting without error.
If instead of ssh another command, e.g. ls, is called with check_output then the bash loop works as expected.
Can anyone help me understand why the code as shown isn't working as expected?
Note: this is a simplified version of what I am trying to do, though I do experience the same behavior with this code. In reality I am doing something with "$line" in the bash script and the subprocess call is wrapped in a try/except block.
As #larsmans guessed the ssh call was consuming stdin, breaking the outer bash loop. Adding the -n option to the ssh command resolved the issue:
output = subprocess.check_output(['ssh', '-n', 'user#host', 'cmd'])
The problem is that ssh reads from standard input, therefore it "eats" all the remaining lines in the loop. You can just connect its standard input to nowhere using the -n flag:
output = subprocess.check_output(['ssh', '-n', 'user#host', 'cmd'])
Look for the details on the man pages of ssh here https://linux.die.net/man/1/ssh and https://man.openbsd.org/ssh