Develop Reference

Break Loop in python without pressing Ctr-C, "agnostic" to OS

An easy way to end an infinite for loop in python is by using Ctrl-C, e.g.:
try:
while True:
print '-',
except KeyboardInterrupt:
print 'done'
However, I want to use some other than Ctrl-C key because on other colleagues machines that might be interpreted by their gui environment (e.g. pycharm) as copy. Is there a way to do this? I need a solution that works both in windows and linux...

Ctrl+C on Linux sends a kill signal to the Python process. I don't think the Python script will be able to override the default behavior of the operating system.

Python3 Non-blocking input or killing threads

Reading through posts of similar questions I strongly suspect there is no way to do what I'm trying to do but figured I'd ask. I have a program using python3 that is designed to run headless, receiving commands from remote users that have logged in. One of the commands of course is a shutdown so that the program can be ended cleanly. This section is working correctly.
However while working on this I realized an option to be able to enter commands directly, without a remote connection, would be useful in the event something unusual happened to prevent remote access. I added a local_control function that runs in it's own thread so that it doesn't interfere with the main loop. This works great for all commands except for the shutdown command.
I have a variable that both loops monitor so that they can end when the shutdown command is sent. Sending the shutdown command from within local_control works fine because the loop ends before getting back to input(). however when sending the shutdown command remotely the program doesn't end until someone presses the enter key locally because that loop remains stuck at input(). As soon as enter is pressed the program continues, successfully breaks the loop and continues with the shutdown as normal. Below is an example of my code.
import threading
self.runserver = True
def local_control(): #system to control server without remote access
while self.runserver:
raw_input = input()
if raw_input == "shutdown":
self.runserver = False
mythread = threading.Thread(target=local_control)
mythread.start()
while self.runserver:
some_input = get_remote_input() #getting command from remote user
if some_input == "shutdown":
self.runserver = False
sys.exit(0) #server is shutdown cleanly
Because the program runs primarily headless GUI options such as pygame aren't an option. Other solutions I've found online involve libraries that are not cross-platform such as msvcrt, termios, and curses. Although it's not as clean an option I'd settle for simply killing the thread to end it if I could however there is no way to do that as well. So is there a cross-platform, non-GUI option to have a non-blocking input? Or is there another way to break a blocked loop from another thread?

Your network-IO thread is blocking the processing of commands while waiting for remote commands, so it will only evaluate the state of runserver after get_remote_input() returns (and it's command is processed).
You will need three threads:
One which loops in local_control(), sending commands to the processing thread.
One which loops on get_remote_input(), also sending commands to the processing thread.
A processing thread (possibly the main thread).
A queue will probably be helpful here, since you need to avoid the race condition caused by unsynchronized access as currently present with regards to runserver.

Not a portable solution, but in *nix, you might be able send yourself an interrupt signal from the local_control function to break the blocking input(). You'll need the pthread ID (pthread_self and save it somewhere readable from local_control) for the network control thread so you can call pthread_kill.

Python: GUI for continuously running script

I am writing a script which will run continuously on a computer. As it has to run on a computer without python installation, I am planning to convert it to executable. I also want to have a GUI to start and stop this application but I don't want this GUI to be opened all the time. I mean if the GUI is closed, I don't want the executable to stop running. It should stop only if user presses stop button on GUI. This GUI is just a interface for users to start and stop the executable.
How can I achieve this behavior?

The obvious solution is to have two separate programs: a backgrounder/daemon/agent/service that just chugs along in the background detached from user input and output, and a GUI program to control it. A nice advantage of this design is that you can also have a command-line program to control it, if you ever want to ssh in remotely, or control it from a script.
The traditional Unix way of handling this is to use a daemon designed like a system service (even if it's run like a normal user): it writes its pid to a file when it starts up, and the control program reads that file and sends a signal to the pid that it finds to kill it.
So, the control program has functions something like this:
def is_running():
try:
with open(PID_PATH) as f:
pid = int(f.read())
os.kill(pid, 0)
except Exception:
return False
else:
return True
def stop():
with open(PID_PATH) as f:
pid = int(f.read())
os.kill(pid, signal.SIGUSR1)
def start():
subprocess.check_call(DAEMON_PATH)
Of course in real life, you'll want some better error handling. Also, which signal you use depends on whether you want the daemon to die hard and instantly, or to gracefully shut down. And so on.
An alternative is to have the background process listen on a socket—whether TCP with a known port, or a Unix socket with a known filename—and communicate with it that way. This allows you to do fancier things that just start and stop.
On Windows, the details aren't quite the same, but you can do something similar.
Finally, Windows, OS X, and various linux distros also all have platform-specific ways of wrapping this kind of thing up at a higher level, so you might want to build a Windows Service, LaunchAgent, etc.

Thanks #abarnert.I used your method and converted your code for windows. Please see below my solution which works. It's starting and stopping helloworld.exe. I have removed error handling to keep it simple.
import subprocess
import time
def startprocess():
#start helloworld.exe
process = subprocess.Popen(['helloworld.exe'])
#Write down the prog id into a file for later use
f = open('progid.txt','w')
f.writelines(str(int(process._handle)))
f.close()
def endprocess():
f = open('progid.txt','r')
progid = int(f.read())
f.close()
# Kill the process using pywin32
import win32api
win32api.TerminateProcess(progid, -1)
startprocess()
time.sleep(60) #wait for 60 second before kill
endprocess()

Shell hangs after killing subprocess

I know there are a bunch of similar questions on SO like this one or this one and maybe a couple more, but none of them seem to apply in my particular situation. My lack of understanding on how subprocess.Popen() works doesn't help either.
What i want to achieve is: launch a subprocess (a command line radio player) that also outputs data to the terminal and can also receive input -- wait for a while -- terminate the subprocess -- exit the shell. I am running python 2.7 on OSX 10.9
Case 1.
This launches the radio player (but audio only!), terminates the process, exits.
import subprocess
import time
p = subprocess.Popen(['/bin/bash', '-c', 'mplayer http://173.239.76.147:8090'],
stdin=subprocess.PIPE, stdout=subprocess.PIPE, shell=False,
stderr=subprocess.STDOUT)
time.sleep(5)
p.kill()
Case 2.
This launches the radio player, outputs information like radio name, song, bitrate, etc and also accepts input. It terminates the subprocess but it never exists the shell and the terminal becomes unusable even after using 'Ctrl-C'.
p = subprocess.Popen(['/bin/bash', '-c', 'mplayer http://173.239.76.147:8090'],
shell=False)
time.sleep(5)
p.kill()
Any ideas on how to do it? I was even thinking at the possibility of opening a slave-shell for the subprocess if there is no other choice (of course it is also something that I don't have a clue about). Thanks!

It seems like mplayer uses the curses library and when kill()ing it or terminate()ing it, for some reason, it doesn't clean the library state correctly.
To restore the terminal state you can use the reset command.
Demo:
import subprocess, time
p = subprocess.Popen(['mplayer', 'http://173.239.76.147:8090'])
time.sleep(5)
p.terminate()
p.wait() # important!
subprocess.Popen(['reset']).wait()
print('Hello, World!')
In principle it should be possible to use stty sane too, but it doesn't work well for me.
As Sebastian points out, there was a missing wait() call in the above code (now added). With this wait() call and using terminate() the terminal doesn't get messed up (and so there shouldn't be any need for reset).
Without the wait() I sometimes do have problems of mixed output between the python process and mplayer.
Also, a solution specific to mplayer, as pointed out by Sebastian, is to send a q to the stdin of mplayer to quit it.
I leave the code that uses reset because it works with any program that uses the curses library, whether it correctly tears down the library or not, and thus it might be useful in other situations where a clean exit isn't possible.

What i want to achieve is: launch a subprocess (a command line radio player) that also outputs data to the terminal and can also receive input -- wait for a while -- terminate the subprocess -- exit the shell. I am running python 2.7 on OSX 10.9
On my system, mplayer accepts keyboard commands e.g., q to stop playing and quit:
#!/usr/bin/env python
import shlex
import time
from subprocess import Popen, PIPE
cmd = shlex.split("mplayer http://www.swissradio.ch/streams/6034.m3u")
p = Popen(cmd, stdin=PIPE)
time.sleep(5)
p.communicate(b'q')
It starts mplayer tuned to public domain classical; waits 5 seconds; asks mplayer to quit and waits for it to exit. The output is going to terminal (the same place where the python script's output goes).
I've also tried p.kill(), p.terminate(), p.send_signal(signal.SIGINT) (Ctrl + C). p.kill() creates the impression that the process hangs. Possible explanation: p.kill() leaves some pipes open e.g., if stdout=PIPE then your Python script might hang at p.stdout.read() i.e., it kills the parent mplayer process but there might be a child process that holds the pipes open. Nothing hangs with p.terminate(), p.send_signal(signal.SIGINT) -- mplayer exits in an orderly manner. None of the variants I've tried require reset.
how should I go about having both input from Python and keyboard? Do I need two different subprocesses and how to redirect the keyboard input to PIPE?
It would be much simpler just to drop stdin=PIPE and call p.terminate(); p.wait() instead of p.communicate(b'q').
If you want to keep stdin=PIPE then the general principle is: read from sys.stdin, write to p.stdin until timeout happens. Given that mplayer expects one letter commands, you need to be able to read one character at at time from sys.stdin. The write part is easy: p.stdin.write(c) (set bufsize=0 to avoid buffering on Python side. mplayer doesn't buffer its stdin so you don't need to worry about it).
You don't need two different subprocesses. To implement timeout, you could use threading.Timer(5, p.stdin.write, [b'q']).start() or select.select on sys.stdin with timeout.
I guess something using the good old raw_input has nothing to do with it, or?
raw_input() is not suitable for mplayer because it reads the full lines but mplayer expects one character at a time.

How can I create a Python script that runs in the background and can be stopped cleanly?

I have a python script that constantly runs (it has an infinite loop), but I want it to be able to still accept input while running. It will run in the background and then at any time I want to be able to type
scriptname stop
and stop it (or something like that). That way it can call a shutdown method to save information and quit.
Currently it runs in the foreground in the terminal, and can't be stopped by a keyboard interrupt, so the only way to kill it is to close the terminal or kill python.
How can I do something like this?

Use supervisord. It exists to manage processes, and provides a command interface to start and stop them.
When supervisor kills a process, it sends SIGTERM (or any other signal you choose). So, to shutdown cleanly, you need to handle that signal.
See this question on how to handle SIGTERM: Python - Trap all signals
Processes can still listen on their own pipes for input, and send output that way.

If you are in Windows then You are at right point...
Just Rename your file: script.py to script.pyw and Use It Normally.
Your Script will run in background.
To close that script:
Go to Task Manager , click on Process Tab , look out for python , End Task.
If You need more information I am Ready to Provide to you...
I am Not Sure About Linux or Ubuntu.
Thanks.