I'm working on a tool for data entry at my job where it basically takes a report ID number, opens a PDF to that page of that report, allows you to input the information and then saves it.
I'm completely new to instantiating new processes in python; this is the first time that I've really tried to do it. so basically, I have a relevant function:
def get_report(id):
path = report_path(id)
if not path:
raise NameError
page = get_page(path, id)
proc = subprocess.Popen(["C:\Program Files (x86)\Adobe\Reader 11.0\Reader\AcroRd32.exe", "/A", "page={}".format(page),
path])
in order to open the report in Adobe Acrobat and be able to input information while the report is still open, I determined that I had to use multiprocessing. So, as a result, in the main loop of the program, where it iterates through data and gets the report ID, I have this:
for row in rows:
print 'Opening report for {}'.format(ID)
arg = ID
proc = Process(target=get_report, args=(arg,))
proc.start()
row[1] = raw_input('Enter the desired value: ')
rows.updateRow(row)
while proc.is_alive():
pass
This way, one can enter data without the program hanging on the subprocess.Popen() command. However, if it simply continues on to the next record without closing the Acrobat window that pops up, then it won't actually open the next report. Hence the while proc.is_alive():, as it gives one a chance to close the window manually. I'd like to kill the process immediately after 'enter' is hit and the value entered, so it will go on and just open the next report with even less work. I tried several different things, ways to kill processes through the pid using os.kill(); I tried killing the subprocess, killing the process itself, killing both of them, and also tried using subprocess.call() instead of Popen() to see if it made a difference.
It didn't.
What am I missing here? How do I kill the process and close the window that it opened in? Is this even possible? Like I said, I have just about 0 experience with processes in python. If I'm doing something horribly wrong, please let me know!
Thanks in advance.
To kill/terminate a subprocess, call proc.kill()/proc.terminate(). It may leave grandchildren processes running, see subprocess: deleting child processes in Windows
This way, one can enter data without the program hanging on the subprocess.Popen() command.
Popen() starts the command. It does not wait for the command to finish. There are .wait() method and convenience functions such as call()
Even if Popen(command).wait() returns i.e., if the corresponding external process has exited; it does not necessarily mean that the document is closed in the general case (the launcher app is done but the main application may persist).
i.e., the first step is to drop unnecessary multiprocessing.Process and call Popen() in the main process instead.
The second step is to make sure to start an executable that owns the opened document i.e., if it is killed the corresponding document won't stay opened: AcroRd32.exe might be already such program (test it: see whether call([r'..\AcroRd32.exe', ..]) waits for the document to be closed) or it might have a command-line switch that enables such behavior. See How do I launch a file in its default program, and then close it when the script finishes?
I tried killing the subprocess, killing the process itself, killing both of them, and also tried using subprocess.call() instead of Popen() to see if it made a difference.
It didn't.
If kill() and Popen() behave the same in your case then either you've made a mistake (they don't behave the same: you should create a minimal standalone code example with a dummy pdf that demonstrates the problem. Describe using words: what do you expect to happen (step by step) and what happens instead) or AcroRd32.exe is just a launcher app that I've described above (it just opens the document and immediately exits without waiting for the document to be closed).
Related
I have a Flask application where there are links that open Jupyter notebooks. In the function that handles the url, the Jupyter notebooks are opened by a call to subprocess.Popen. Especially on Windows, after some time, the notebooks seems to be dead, i.e. they have lost connection to the kernel, and I can only get them to work again by clicking on the Flask link again. I have not noticed this behavior on a Mac. This makes me think that maybe the subprocess is getting closed. It isn't stored in a variable or anything, so once the function exits, there is no scope for it to be in. Does anyone know if this happens, and if so what happens to the process that should be running?
Here is an example of what one of these functions looks like. When you click on a link it calls the open_lecture function, constructs a cmd and Popens it. Then the function exits.
#app.route("/lecture/<label>")
def open_lecture(label):
fname = 'lectures/{}.ipynb'.format(label)
# Now open the notebook.
cmd = [JUPYTER]
cmd += [fname]
print(cmd)
subprocess.Popen(cmd, stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
stdin=subprocess.PIPE)
return redirect(url_for('hello'))
Is there a way to keep this from happening? Or a better way to programmatically open a jupyter notebook?
executed processes should continue after any Popen object has been garbage collected, but in your case you have asked for the output to be returned to Python but then you aren't processing the output. this could do different things depending on how things get cleaned up in Python and how the invoked process handles errors.
as an example, if I run:
proc = subprocess.Popen(["yes"], stdout=subprocess.PIPE)
(yes writes lots of stuff to its stdout) it freezes after writing 64KiB of output, which is what my OS/kernel (Linux 5.3) gives to pipes between processes. you can confirm by waiting a second, calling proc.terminate() then print(len(proc.communicate()[0])).
as you're invoking Jupyter, it'll probably just write status and other informational messages to stdout, so will take a while to fill this buffer which is why you're seeing a sporadic timeout.
Python's garbage collector only works on its own heap (i.e. not other processes, each Python process is independent), so invoked processes will just run as they would according to the semantics of your OS
I want to store the output of the terminal command top into a file, using Python.
In the terminal, when I type top and hit enter, I get an output that is real time, so it keeps updating. I want to store this into a file for a fixed duration and then stop writing.
file=open("data.txt","w")
file.flush()
import os,time
os.system("top>>data.txt -n 1")
time.sleep(5)
exit()
file.close()
I have tried to use time.sleep() and then exit(), but it doesn't work, and the only way top can be stopped is in the terminal, by Control + C
The process keeps running and the data is continuously written onto the file, which is not ideal, as one would guess
For clarity: I know how to write the output on to the file, I just want to stop writing after a period
system will wait for the end of the child process. If you do not want that, the Pythonic way is to directly use the subprocess module:
import subprocess
timeout=60 # let top run for one minute
file=open("data.txt","w")
top = subprocess.Popen(["top", "-n", 1], stdout=file)
if top.wait(timeout) is None: # wait at most timeout seconds
top.terminate() # and terminate child
The panonoic way (which is highly recommended for robust code) would be to use the full path of top. I have not here, because it may depend on the actual system...
The issue you could be facing is that os.system starts the process as part of the current process. So the rest of your script will not be run until the command you run has completed execution.
I think what you want to be doing is executing your console command on another thread so that the thread running your python script can continue while the command runs in the background. See run a python program on a new thread for more info.
I'd suggest something like (this is untested):
import os
import time
import multiprocessing
myThread = multiprocessing.process(target=os.system, args=("top>>data.txt -n 1",))
myThread.start()
time.sleep(5)
myThread.terminate()
That being said, you may need to consider the thread safety of os.system(), if it is not thread safe you'll need to find an alternative that is.
Something else worth noting (and that I know little about) is that it may not be ideal to terminate threads in this way, see some of the answers here: Is there any way to kill a Thread?
I am trying to open an image with subprocess so it is visible to the user, then close the image so that it disapears.
This question has been asked before, but the answers I found have not worked for me. Here is what I have checked:
Killing a process created with Python's subprocess.Popen()
How to terminate a python subprocess launched with shell=True
How can I close an image shown to the user with the Python Imaging Library?
I need the code to open an image (with Preview (optional), default on my Mac), wait a second, then close the image.
openimg = subprocess.Popen(["open", mypath])
time.sleep(1)
openimg.terminate()
openimg.kill()
Everything is telling me to terminate() and kill(), but that still isn't closing the image.
This does not HAVE to use preview, I am open to hearing other options as well.
Edit: I have additionally tried the below code, still no success
print('openimg\'s pid = ',openimg.pid)
os.kill(openimg.pid, signal.SIGKILL)
OS X's open command is asynchronous - it spawns a Preview process (or whatever app it launches) and immediately exits. So by the time your Python code gets around to calling terminate() and kill(), the open process is done. It no longer exists.
You can force synchronous behavior, i.e. make open keep running until after Preview exits, by passing the -W option,
subprocess.Popen(["open", "-W", mypath])
This way, open will still be running when your code gets around to running terminate and kill. (I would suggest also passing the -n option to make sure Preview starts a new instance, in case you had another instance of Preview sitting around from before.) And when the open process ends, hopefully it will also end the underlying Preview process. You can check whether this actually happens using a process viewer such as ps or pgrep.
If terminating or killing open does not kill Preview, you'll need to either change the configuration so that the signal is delivered to all subprocesses of open when you call terminate() or kill(), for which this question or this one may be helpful, or you'll have to find a way to get the process ID of Preview and send signals to that directly, which will require you to go beyond Popen. I'm not sure of the best way to do that, but perhaps someone else can contribute an answer that shows you how.
I'm sure this is the hackiest way to do this, but hell, it works. If anyone stumbles on this and knows a better way, please let me know.
The Problem
As David Z described, I was not targeting the correct process. So I had to figure out what the correct process ID was in order to close it properly.
I started by opening my terminal and entering the command ps -A. This will show all of the current open processes and their ID. Then I just searched for Preview and found two open processes.
Previously, I was closing the first Preview pid in the list, let's call it 11329 but the second on the list was still open. The second Preview process, 11330, was always 1 digit higher then the first process. This second one was the pid I needed to target to close Preview.
The Answer
openimg = subprocess.Popen(["open", mypath]) ##Opens the image to view
newpid = openimg.pid + 1 ##Gets the pid 1 digit higher than the openimg pid.
os.kill(newpid, signal.SIGKILL) ##Kills the pid and closes Preview
This answer seems very fragile, but it works for me. I only just started learning about pids so if anyone could provide some knowledge, I would be grateful.
I have several python scripts that turn my TV on and off. Sometimes the TV does not respond the first time so I use a while loop to continue sending the command until the "success" response is sent. Up to 10 times.
I need to check if one of these programs are running when any of them are started and kill the first process.
This answer uses domain locks and I think this could work but I dont really understand whats happening there:
https://stackoverflow.com/a/7758075/2005444
What I dont know is what the process_name would be. The scripts are titles tvon.py, tvoff.py, and tvtoggle.py. Is it just the title? Would it include the extension? How do I get the pid so I can kill the process?
This is running on Ubuntu 14.04.1
EDIT: all I really need is to search for any of these running scripts first. Also, instead of killing the process maybe I could just wait for it to finish. I could just do a loop and break it if none of those processes are running.
The reason I need to do this is if the tv is off and the off script is run it will never succeed. The TV wont respond if it is already off. Which is why I built in the limit of 10 commands. It never really takes over 4 so 10 is overkill. The problem is if the off command is trying to run and I turn the TV on using the tvon script the TV will turn on and back off. Although the TV limits how often commands can be accepted, which reduces the chance of this happening I still want the to be as cleanly working as possible.
EDIT:
I found that I can not kill the process because it can lock the tty port up which requires a manual restart. So I think the smarter way is to have the second process wait until the first is done. Or find a way to tell the first process to stop at a specific point in the loop so I know its not transmitting.
If you have a socket, use it. Sockets provide full-blown bidirectional communication. Just write your script to kill itself if it receives anything on the socket. This can be most easily done by creating a separate thread which tries to do a socket.recv() (for SOCK_DGRAM) or socket.accept() (for SOCK_STREAM/SOCK_SEQPACKET), and then calls sys.exit() once that succeeds.
You can then use socket.send() (SOCK_DGRAM) or socket.connect() (SOCK_STREAM/SOCK_SEQPACKET) from the second script instance to ask the first instance to exit.
This function can kill a python script by name on *nix systems. It looks through a list of running processes, finds the PID of the one associated with your script, and issues a kill command.
import subprocess
def killScript(scriptName):
# get running processes with the ps aux command
res = subprocess.check_output(["ps","aux"], stderr=subprocess.STDOUT)
for line in res.split("\n"):
# if one of the lines lists our process
if line.find(scriptName) != -1:
info = []
# split the information into info[]
for part in line.split(" "):
if part.strip() != "":
info.append(part)
# the PID is in the second slot
PID = info[1]
#kill the PID
subprocess.check_output(["kill",PID], stderr=subprocess.STDOUT)
At the beginning of your tv script you could run something like:
killList = ["tvon.py", "tvoff.py", "tvtoggle.py"]
for script in killList:
killScript(script)
What I would like to do, is some kind of communication between two applications:
first application would run and call the second application (first application would be closed) then if the second application has finished its job I would like to sent some kind of signal to the first application and launch it again (second application would be closed this time).
The only one idea I got is to write to a file, when the second application has finished its job and check in the first application if the file exists... is there any other way to do that?
It's a little unclear what you're trying to do. Are you simply trying to chain applications together, i.e. when the first application finishes it calls the second application, and when the second finishes it calls the first, etc.? If this is true, then you simply have to have one application spawn the other and exit immediately instead of waiting. Take a look at subprocess.Popen, which lets you do this (subprocess.call always waits).
However, it sounds like maybe you want the first application to continue running, but to reset itself when the second one finishes. In this case the second application is in fact the child of the first. In the first, you can check if the second has finished after calling Popen by using poll; then, when the second has finished, have the first app respawn itself and then exit, just as described above using Popen. If you want it to work asynchronously, you can create a separate thread in the first app that calls wait, and then exits; but you have to be careful with synchronization in this case. I can't say more because I don't know what the first app is doing while the second runs.
If you want the second application to keep running but the send a signal back to the first application, which is also running, create a pipe between the two, so that when the child writes to its stdout, it's actually writing to a pipe, which the parent can read. If your parent (the first application) is able to block waiting for the second application to finish doing what it's doing, then you just do this:
p = subprocess.Popen('myprogram.exe', stdout = subprocess.PIPE)
str = p.stdout.readline()
Then, the call to readline will block until the second program prints something out (e.g. a line just containing "done"), at which point str in the first program will contain the line printed out by the second program.
If you need for the first program to do something else at the same time, and periodically poll the pipe in a non-blocking fashion, it gets trickier. Take a look at this answer for a solution that works on both Unix and Windows:
Non-blocking read on a subprocess.PIPE in python