I recently worked on using multithreading but ran into an issue where it seems that multiprocessing would be the better way to go. When I run a simple loop counter function as a process, why doesn't it iterate through the loop and print out the output? Instead the code waits for a set amount of time before producing the output. Is there a way this can be solved or am I stuck dealing with processes this way?
import multiprocessing, time
def loop_process(process_name):
loopCnt = 0
print "\nstarting {}".format(process_name)
for loopCnt in range(15):
print("value of loopCnt = {}".format(loopCnt))
loopCnt += 1
time.sleep(1)
print('stopping {}'.format(process_name))
if __name__ == '__main__':
L00P_process = multiprocessing.Process(target=loop_process, args=('L00P_process',))
L00P_process.start()
L00P_process.join()
print('processes stopped')
print "Exiting Main"
I'm not clear on what you're seeing. On a Windows box just now, running the program from an interactive console ("DOS box"), I saw value of loopCnt = ... once per second until the program ended. That's what I expected.
Which OS are you running under, how are you running the program, and what exactly are you seeing?
On most (all?) machines, standard output (stdout) is line-buffered if it's attached to an interactive terminal. Which means output is forced to display each time a line boundary is hit. For other kinds of output device, it may use other kinds of buffering, and that could account for a delay.
Something to try: first add
import sys
near the top, then add
sys.stdout.flush()
after your
print("value of loopCnt = {}".format(loopCnt))
That should tell us whether you are, or are not, experiencing a problem with output buffering.
Related
I am using a python library I wrote to interact with a custom USB device. The library needs to send and receive data. I also need to interactively call methods. At the moment I utilize two shells, one receiving only and the other sending only. The latter is in the (i)python REPL. It works but it is clumsy, so I want to consolidate the two things in a single shell, which will have the advantage to have access to data structures from both sides in one context. That works fine. The problem is in the UI.
In fact, the receiving part needs to asynchronously print some information. So I wrote something like the following:
#!/usr/bin/python
import threading
import time
import blessings
TOTAL=5
def print_time( thread_id, delay):
count = 0
t=blessings.Terminal()
while count < TOTAL:
time.sleep(delay)
count += 1
stuff = "Thread " + str(thread_id) + " " + str(time.ctime(time.time())) + " -- " + str(TOTAL - count) + " to go"
with t.location(t.width - len(stuff) - 1, thread_id):
print (stuff, end=None )
print("", end="") # just return the cursor
try:
t1 = threading.Thread( target = print_time, args = (1, 2, ) )
t1.start()
print ("Thread started")
except:
print ("Error: unable to start thread")
That is my __init__.py file for the module. It somewhat works, but it has two problems:
While the thread is running, you cannot exit the REPL neither with CTRL-D nor with sys.exit() (that is the reason I am using TOTAL=5 above, so your life is easier if you try this code). This is a problem since my actual thread needs to be an infinite loop. I guess one solution could be to exit via a custom call which will cause a break into that infinite loop, but is there anything better?
The cursor does not return correctly to its earlier position
if I remove the end="" in the line with the comment # just return the cursor, it sort of works, but obviously print an unwanted newline in the place the cursor was (which messes us other input and/or output which might be happening there, in addition to add that unwanted newline)
if I leave the end="" it does not return the cursor, not even if I add something to print, e.g. print(".", end="") -- the dots . are printed at the right place, but the blinking cursor and the input is printed at the top
I know these are two unrelated problem and I could have asked two separate questions, but I need an answer to both, or otherwise it's a moot point. Or alternatively, I am open to other solutions. I thought of a separate GTK window, and that might work, but it's a subpar solution, since I really would like this to work in CLI only (to keep it possible in a ssh-without-X-tunneling setup).
Using blessed instead of blessing does not have the problem with the cursor not returning to the previous position, even without anything outside of the with context.
Making the thread a daemon solves the other problem.
There's a little app named logivew that I'm writing a script to monitor, along with some other tasks. In the main while loop (which will exit when the app I'm most concerned about closes), I check to see if logview needs restarting. The code I have presently is roughly as follows:
#a good old global
logview = "/usr/bin/logview"
#a function that starts logview:
port = 100
log_file = "/foo/bar"
logview_process = subprocess.Popen([logview, log_file, port],
stdout = subprocess.DEVNULL,
stderr = subprocess.STDOUT)
#a separate function that monitors in the background:
while True:
time.sleep(1)
logview_status = 0
try:
logview_status = psutil.Process(logview_process.pid).status()
except psutil.NoSuchProcess:
pass
if(logview_status == psutil.STATUS_STOPPED or
logview_status == psutil.STATUS_ZOMBIE or
logview_status == psutil.STATUS_DEAD or
logview_status == 0):
print("Logview died; restarting")
logview_cli_list = [logview]
logview_cli_list.extend(logview_process.args)
logview_process = subprocess.Popen(logview_cli_list,
stdout = subprocess.DEVNULL,
stderr = subprocess.STDOUT)
if(some_other_condition): break
However, if I test-kill logview, the condition triggers and I do see the printed message, but then I see it again, and again, and again. It seems that the condition triggers every single iteration of the loop if logview does die. And, it never does get restarted properly.
So clearly... I'm doing something wrong. =)
Any help (or better methods!) would be greatly appreciated.
I don't know your logview program but the problem is here:
logview_cli_list = [logview]
logview_cli_list.extend(logview_process.args)
When you're creating the argument list, you're putting logview twice in your command, because logview_process.args also contains the name of the launched command, so the program probably fails immediately because of bad args, and is run again and again...
The fix is then obvious:
logview_cli_list = logview_process.args
a better fix would be to create the process in the loop if a given flag is set and set the flag at the start.
When process dies, set the flag to trigger the process creation again. Would have avoided this copy/almost paste mistake.
I wrote a little code which scans the wifi signals in the air and according to the amount plays one or another audio file.
import time
import subprocess
import os
while True:
output = subprocess.Popen("airport en1 -s", shell=True, stdout=subprocess.PIPE).stdout.read()
lines = output.split('\n')
n = len(lines)
if n < 10:
print os.path.exists('/Users/shirin/Desktop/wifi/1.mp3')
p = subprocess.Popen(['/Applications/VLC.app/Contents/MacOS/VLC','/Users/shirin/Desktop/wifi/1.mp3'])
elif n > 10:
print os.path.exists('/Users/shirin/Desktop/wifi/1.mp3')
p = subprocess.Popen(['/Applications/VLC.app/Contents/MacOS/VLC','/Users/shirin/Desktop/wifi/1.mp3'])
Now my problem is that, when I run the file through the terminal it keeps running the code over and over again and I do not know how to stop it. It also does not stop when I close the terminal.
So my question:
How can I tell python to run the code once and then stop?
already super thanks!
Two things:
You have a while True: and no condition to break the loop, insert a break after reproducing the audio file so the loop could stop.
Even if you close the program VLC would be running until you manually stop it. This is because you are just calling an external command. If you want to reproduce the audio from python you can use another library like pygame
You can call p.terminate() if you want to terminate your application.
You can also call p.kill() if your application is not responsive.
Now, let's say you want to stop the application and stop VLC.
Have except KeyboardInterrupt: in your code and call p.terminate() when you stop the script.
while True:
try:
... #code goes here.
except KeyboardInterrupt:
p.terminate()
Im new to python and was wondering how to make a loading animation while my program runs. I need this because I don't want users thinking that the program is caught in a dead loop. I prefer a something like...
Loading...(with the dots disappearing and reappearing one by one)
Thanks!
If your output window supports the carriage return character, you can print it to make the cursor return to the beginning of the current line (provided you end your print statement with a comma, so a newline character isn't automatically printed). Then subsequent prints will overwrite what was already printed. You can use this to do very simple one line animation. Example:
import time
print "Starting program."
print "Loading ",
time.sleep(1) #do some work here...
print "\rLoading. ",
time.sleep(1) #do some more work here...
print "\rLoading.. ",
time.sleep(1) #do even more work...
print "\rLoading...",
time.sleep(1) #gratuitious amounts of work...
print "\rLoading ",
... Where time.sleep(1) is a placeholder representing the actual work you want to do.
Result:
Starting program.
Loading
Then, one second later:
Starting program.
Loading.
Then, one second later:
Starting program.
Loading..
Then, one second later:
Starting program.
Loading...
Then, one second later:
Starting program.
Loading
etc.
Compatibility note: in 3.X, print is no longer a statement, and the "end with a comma" trick no longer works. Instead, specify the end parameter:
print("\rLoading...", end="")
The most proper way I can think of to do it would be using threading.
You would initiate a thread that starts displaying some indication that the program is doing something and then open a new thread that actually does the work.
When the thread doing the work finished then you can move on with whatever else the program does.
This looks ok when ran in windows command prompt, not sure how linux will like it:
import threading
import time
import os
import queue
q = queue.Queue()
q.put(False)
class counter(object):
def __init__(self):
wait_label = "Loading"
self.stop_flag = q.get()
while not self.stop_flag:
try:
self.stop_flag = q.get_nowait()
except:
pass
os.system('cls') # might need to change this command for linux
wait_label += "."
print(wait_label)
time.sleep(1)
class other(counter):
def __init__(self):
time.sleep(15)
q.put(True)
counter_thread = threading.Thread(None, counter)
counter_thread.start()
other_thread = threading.Thread(None, other)
other_thread.start()
To reduce the code length, we can loop it.
import time
# flag variable to print the dots and it's value increases inside the while loop.
flag = 1
# To print the dots we use while loop. In total, 4 dots will be printed.
while flag < 5:
print("\rLoading, Please Wait " + ("." * flag), end=" ")
time.sleep(1)
flag = flag + 1
I know how to drop into an interpreter with pdb and IPython, but this requires me knowing beforehand exactly where I want to stop. However, I often run number crunching scripts that take minutes to hours, and I would like to know exactly what it's progress is. One solution is to simply put lots of logging statements everywhere, but then I either inundate myself with too much information or fail to log exactly what I want to know.
Is there a way to initialize a listener loop that under some key combination will drop me into the code wherever it currently is? Think CTRL+Z but leaving me in Python rather than Bash.
You can use the signal module to setup a handler that will launch the debugger when you hit control-C or control-Z or whatever.. SIGINTR, SIGSUSP.
For example, define a module instant_debug.py that overrides SIGQUIT,
import signal
import pdb
def handler(signum, frame):
pdb.set_trace()
signal.signal(signal.SIGQUIT, handler)
Then make a script
import instant_debug
import time
for i in xrange(1000000):
print i
time.sleep(0.1)
At any point during execution, you can jump into the code by typing CTRL+\, examine the stack with u and d as in normal pdb, then continue with c as if nothing ever happened. Note that you will only jump in at the end of the next "atomic" operation -- that means no stopping in the middle of a giant C module.
You could do this
def main():
i = 1000
while True:
print "Count Down %s" % i
time.sleep(1)
i -= 1
try:
main()
except KeyboardInterrupt:
pass # Swallow ctrl-c
finally:
code.interact("Dropped into interpreter", local=globals())