Develop Reference

print information in a corner when using python/ipython REPL -- and quitting it while thread is running

I am using a python library I wrote to interact with a custom USB device. The library needs to send and receive data. I also need to interactively call methods. At the moment I utilize two shells, one receiving only and the other sending only. The latter is in the (i)python REPL. It works but it is clumsy, so I want to consolidate the two things in a single shell, which will have the advantage to have access to data structures from both sides in one context. That works fine. The problem is in the UI.
In fact, the receiving part needs to asynchronously print some information. So I wrote something like the following:
#!/usr/bin/python
import threading
import time
import blessings
TOTAL=5
def print_time( thread_id, delay):
count = 0
t=blessings.Terminal()
while count < TOTAL:
time.sleep(delay)
count += 1
stuff = "Thread " + str(thread_id) + " " + str(time.ctime(time.time())) + " -- " + str(TOTAL - count) + " to go"
with t.location(t.width - len(stuff) - 1, thread_id):
print (stuff, end=None )
print("", end="") # just return the cursor
try:
t1 = threading.Thread( target = print_time, args = (1, 2, ) )
t1.start()
print ("Thread started")
except:
print ("Error: unable to start thread")
That is my __init__.py file for the module. It somewhat works, but it has two problems:
While the thread is running, you cannot exit the REPL neither with CTRL-D nor with sys.exit() (that is the reason I am using TOTAL=5 above, so your life is easier if you try this code). This is a problem since my actual thread needs to be an infinite loop. I guess one solution could be to exit via a custom call which will cause a break into that infinite loop, but is there anything better?
The cursor does not return correctly to its earlier position
if I remove the end="" in the line with the comment # just return the cursor, it sort of works, but obviously print an unwanted newline in the place the cursor was (which messes us other input and/or output which might be happening there, in addition to add that unwanted newline)
if I leave the end="" it does not return the cursor, not even if I add something to print, e.g. print(".", end="") -- the dots . are printed at the right place, but the blinking cursor and the input is printed at the top
I know these are two unrelated problem and I could have asked two separate questions, but I need an answer to both, or otherwise it's a moot point. Or alternatively, I am open to other solutions. I thought of a separate GTK window, and that might work, but it's a subpar solution, since I really would like this to work in CLI only (to keep it possible in a ssh-without-X-tunneling setup).

Using blessed instead of blessing does not have the problem with the cursor not returning to the previous position, even without anything outside of the with context.
Making the thread a daemon solves the other problem.

Python: How to make a non-blocking while-loop?

How do I make a non-blocking while-loop? I am making a program to control RGB led-strips with my phone using mqtt package. An mqtt command should make Led_Desk[1] False but the while-loop blocks everything else including mqtt-subscription.
I've tried threading, async-await, but nothing's worked so far.
while Led_Desk[1]:
for x in range(1,255):
GREEN.ChangeDutyCycle(round(x/2.55,1)*brightness)
time.sleep(speed)
for x in range(1,255):
RED.ChangeDutyCycle(round(100-x/2.55,1)*brightness)
time.sleep(speed)
for x in range(1,255):
BLUE.ChangeDutyCycle(round(x/2.55,1)*brightness)
time.sleep(speed)
for x in range(1,255):
GREEN.ChangeDutyCycle(round(100-x/2.55,1)*brightness)
time.sleep(speed)
for x in range(1,255):
RED.ChangeDutyCycle(round(x/2.55,1)*brightness)
time.sleep(speed)
for x in range(1,255):
BLUE.ChangeDutyCycle(round(100-x/2.55,1)*brightness)
time.sleep(speed)

One option is to use threads, where each thread handles one LED. Not an unreasonable solution, but probably overkill for what you are trying to accomplish. I think this can be done in another way, using 1 thread.
import time
Leds = [[GREEN, time.clock_gettime_ns(0)], [RED, time.clock_gettime_ns(0)], [BLUE, time.clock_gettime_ns(0)]]
while Led_Desk[1]:
now = time.clock_gettime_ns(0)
for led in Leds:
if led[1] + speed > now:
led[1] = now
led[0].ChangeDutyCycle(round(x/2.55,1)*brightness)
The idea here is to keep track of how long each LED has been in it's current state. Then continuously check those values, and when they exceed the threshold, toggle them.
You will probably need to tweak both the logic in the if statement and the way the times are being assigned to create the flashing pattern you want. But hopefully this is clear enough to serve as a starting point.
Also, the reason (you might have figured this out already) that the original code is preventing the mqtt subscription is because calling sleep in those nested for loops in a single threaded program is going to end up sleeping many, many times, sequentially.

I solved it by wrapping the while loop in a thread and defining 'thread1.daemon = True'. I'm not sure what it does exactly but it allows threads and main loop to run parrallel without blocking.

Using time.sleep() in komodo edit?

I am trying to make a simple program in komodo edit using python that when it runs will print out 10 seconds worth of time in the command output.
The code is as follows:
import time
seconds = 0
while seconds != 10:
time.sleep(1)
seconds += 1
print(">", seconds)
When I run this in komodo edit, it doesn't print out the numbers as wanted.
I want the number 1 to be printed after one second of time, the number 2 to be printed after two seconds of time, and etc.
Instead, it prints out all the numbers (1-10) after 10 seconds of time.
I have run this exact same program in the python IDLE and it works as it should, printing one number per second.
What am I doing wrong or what do I not understand/know about?

The program is likely running in an environment where it does not believe its output is connected to a terminal, so stdout defaults to block buffering, not line buffering. Since you're outputting very little data, the buffer never fills, and is only flushed just before the program exits.
The simplest fix is to add the flush=True argument to your print call, so the buffer is explicitly flushed after each print:
print(">", seconds, flush=True)

Does using print() too much cause it to fail?

TL;DR:
The print() result is not updating in a Windows Console. Executes fine in IDLE. Program is executing even though Windows Console is not updating.
Background
I have a file, test.py that contains:
Edit: Included the conditions that I used to see if the Console was updating. Eventually the series of X values never prints again in Console and the Console never scrolls back up (as it normally does when output is being generated at the bottom).
count = 0
while True:
print ("True")
count += 1
if count == 10:
print ("XXXXXXXXX")
count = 0
When I run this in cmd.exe it obviously prints a very large number of True.
However, after about 25 seconds of running, it stops printing any more, though the program is still running and can be seen in the Task Manager.
I have a program with some progress indicators that end up stay at say 50% even though they are moving well beyond 50% simply because print() is not showing in the Console output.
Edit: The true use case problem.
The above code was just a test file to see if printing in Console stopped in all programs, not the one I was running. In practice, my program prints to Console and looks like:
line [10] >> Progress 05%
Where line [10] isn't real but I merely typed here to show you that print() sends to that line in the Console window. As my program continues it increments:
line [10] >> Progress 06%
line [10] >> Progress 11%
.
.
.
line [10] >> Progress 50%
Each time line [10] is overwritten. I use ANSI escape characters and colorama to move the Console cursor accordingly:
print('\x1b[1000D\x1b[1A')
This moves the cursor 1000 columns left and 1 row up (so the start of the previous line).
Something is happening where the print("Progress " + prog + "%") is not showing up anymore in Console because eventually the next bit of Python gets executed:
line [11] >> Program Complete...
I verified the resultants which get put into a folder. So the program continued to run while the Console did not update.
Edit: Here is the script running the updates to the stdout.
def check_queue(q, dates, dct):
out = 0
height = 0
# print the initial columns and rows of output
# each cell has a unique id
# so they are stored in a dictionary
# then I convert to list to print by subscripting
for x in range(0, len(list(dct.values())), 3):
print("\t\t".join(list(dct.values())[x:x+3]))
height +=1 # to determine where the top is for cursor
while True:
if out != (len(dates) * 2):
try:
status = q.get_nowait()
dct[status[1]] = status[2]
print('\x1b[1000D\x1b[' + str(height + 1) + 'A')
# since there was a message that means a value was updated
for x in range(0, len(list(dct.values())), 3):
print("\t\t".join(list(dct.values())[x:x+3]))
if status[0] == 'S' or 'C' or 'F':
out += 1
except queue.Empty:
pass
else:
break
In short, I pass a message to the queue from a thread. I then update a dictionary that holds unique cell IDs. I update the value, move the cursor in Console to the upper left position of the printed list, and print over it.
Question:
When using stdout, is there a limit to how many times you can print to it in a period of time?

That may well be an illusion (maybe because there's a maximum limit of lines in the console and new ones just replace the first ones then).
There's definetly no limit how much you can print. You could verify this with something that changes each iteration, for example a loop that counts the number of iterations:
import itertools
for i in itertools.count():
print(i, "True")

I cannot reproduce the problem in Windows 10 using 64-bit Python 3.6.2 and colorama 0.3.9. Here's the simple example that I tested:
import colorama
colorama.init()
def test(M=10, N=1000):
for x in range(M):
print('spam')
for n in range(N):
print('\x1b[1000D\x1b[' + str(M + 1) + 'A')
for m in range(M):
print('spam', m, n)
Each pass successfully overwrites the previous lines. Here's the final output from looping N (1000) times:
>>> test()
spam 0 999
spam 1 999
spam 2 999
spam 3 999
spam 4 999
spam 5 999
spam 6 999
spam 7 999
spam 8 999
spam 9 999
If this example fails for you, please update your question and include the versions of Windows, Python, and colorama that you're testing.

Sounds like it might be a system limitation, not a Python process issue? I've never run across a 'hang' related to print statements (or any built-in function), however you may want to look at mapping performance and memory usage:
High Memory Usage Using Python Multiprocessing
As far as how many times you can print in a period of time, that is almost exclusively based on the speed the system executes the code. You could run some benchmark tests (execution time / number of executions) across several platforms to test performance with specific system specs, but I'd say the likely cause of your issue is system / environment related.

python sharing variables between active threads

I have a program which runs a camera for a set number of exposures and a set length of exposures through a serial port. The program talks to a server which gives error values, if the error values are within an acceptable range, then the exposure starts and a countdown clock runs until the exposure ends. The following variables are used to make this happen.
T="Length of exposure in seconds"
N="Number of exposures"
then a while loop runs the program using the following
def countdown(n):
for i in range(int(n)):
c = int(n) - int(i)
print c ,'seconds left','\r',
time.sleep(1)
While x < T loop:
countdown(n)
I would like to run a thread which is constantly probing the error number from the server, and if the error number grows to large, it changes to the value of i to equal n.
def errortest():
test=struct.pack("B",10)
s.send(test)
data = s.recv(BUFFER_SIZE)
if ord(data) < (75) and ord(data) > 0:
print ord(data)
time.sleep(5)
else:
print ("error! Stopping exposure")
i=n
My problem is that the variables aren't being shared between the functions. I had some success with setting "i" and "n" as global variables, but this has caused other problems as well, depending on the order in which I invoke or write the different functions. I've also tried return i=n, but n is not shared. It is especially a problem as both threads are running concurrently and not sharing variables. This is why I cannot get queue to work because it pauses the for loop at each instance of q.get().
Is there a better way than using global variables to share variable values between functions running as concurrent threads?

You might want to try sharing your variables within a class. You'll find many tutorials if you google for them. If you want to distribute your tasks into different programs, xmlrpc is a simple possibility.