TL;DR:
The print() result is not updating in a Windows Console. Executes fine in IDLE. Program is executing even though Windows Console is not updating.
Background
I have a file, test.py that contains:
Edit: Included the conditions that I used to see if the Console was updating. Eventually the series of X values never prints again in Console and the Console never scrolls back up (as it normally does when output is being generated at the bottom).
count = 0
while True:
print ("True")
count += 1
if count == 10:
print ("XXXXXXXXX")
count = 0
When I run this in cmd.exe it obviously prints a very large number of True.
However, after about 25 seconds of running, it stops printing any more, though the program is still running and can be seen in the Task Manager.
I have a program with some progress indicators that end up stay at say 50% even though they are moving well beyond 50% simply because print() is not showing in the Console output.
Edit: The true use case problem.
The above code was just a test file to see if printing in Console stopped in all programs, not the one I was running. In practice, my program prints to Console and looks like:
line [10] >> Progress 05%
Where line [10] isn't real but I merely typed here to show you that print() sends to that line in the Console window. As my program continues it increments:
line [10] >> Progress 06%
line [10] >> Progress 11%
.
.
.
line [10] >> Progress 50%
Each time line [10] is overwritten. I use ANSI escape characters and colorama to move the Console cursor accordingly:
print('\x1b[1000D\x1b[1A')
This moves the cursor 1000 columns left and 1 row up (so the start of the previous line).
Something is happening where the print("Progress " + prog + "%") is not showing up anymore in Console because eventually the next bit of Python gets executed:
line [11] >> Program Complete...
I verified the resultants which get put into a folder. So the program continued to run while the Console did not update.
Edit: Here is the script running the updates to the stdout.
def check_queue(q, dates, dct):
out = 0
height = 0
# print the initial columns and rows of output
# each cell has a unique id
# so they are stored in a dictionary
# then I convert to list to print by subscripting
for x in range(0, len(list(dct.values())), 3):
print("\t\t".join(list(dct.values())[x:x+3]))
height +=1 # to determine where the top is for cursor
while True:
if out != (len(dates) * 2):
try:
status = q.get_nowait()
dct[status[1]] = status[2]
print('\x1b[1000D\x1b[' + str(height + 1) + 'A')
# since there was a message that means a value was updated
for x in range(0, len(list(dct.values())), 3):
print("\t\t".join(list(dct.values())[x:x+3]))
if status[0] == 'S' or 'C' or 'F':
out += 1
except queue.Empty:
pass
else:
break
In short, I pass a message to the queue from a thread. I then update a dictionary that holds unique cell IDs. I update the value, move the cursor in Console to the upper left position of the printed list, and print over it.
Question:
When using stdout, is there a limit to how many times you can print to it in a period of time?
That may well be an illusion (maybe because there's a maximum limit of lines in the console and new ones just replace the first ones then).
There's definetly no limit how much you can print. You could verify this with something that changes each iteration, for example a loop that counts the number of iterations:
import itertools
for i in itertools.count():
print(i, "True")
I cannot reproduce the problem in Windows 10 using 64-bit Python 3.6.2 and colorama 0.3.9. Here's the simple example that I tested:
import colorama
colorama.init()
def test(M=10, N=1000):
for x in range(M):
print('spam')
for n in range(N):
print('\x1b[1000D\x1b[' + str(M + 1) + 'A')
for m in range(M):
print('spam', m, n)
Each pass successfully overwrites the previous lines. Here's the final output from looping N (1000) times:
>>> test()
spam 0 999
spam 1 999
spam 2 999
spam 3 999
spam 4 999
spam 5 999
spam 6 999
spam 7 999
spam 8 999
spam 9 999
If this example fails for you, please update your question and include the versions of Windows, Python, and colorama that you're testing.
Sounds like it might be a system limitation, not a Python process issue? I've never run across a 'hang' related to print statements (or any built-in function), however you may want to look at mapping performance and memory usage:
High Memory Usage Using Python Multiprocessing
As far as how many times you can print in a period of time, that is almost exclusively based on the speed the system executes the code. You could run some benchmark tests (execution time / number of executions) across several platforms to test performance with specific system specs, but I'd say the likely cause of your issue is system / environment related.
Related
I'm trying to make a script to calculate pi, but by increasing the steps in the main for loop, it drastically increases in the time it takes to calculate. Sometimes its hard to tell if its doing anything at all. So to fix this I put the following in the main for loop:
# prints the progress percentage (or pp)
pp = (i/rolls)*100
print(pp.__round__(pp_decimals))
rolls in the total number of times the loop will execute.
But this presents a new problem, this ends up printing ALOT of text, and all the print calls end up lagging my pc as well as clogging up the terminal. So my question is, how can I edit the text or delete previous text in order to clear up the output?
It is best to approach your problem in a different way. I assume that i is loop counter so you can print progress percentage in every t times instead clearing the output:
# prints the progress percentage (or pp)
if i % 10 == 0: # t is 10 in this example
pp = (i/rolls)*100
print(pp.__round__(pp_decimals))
I don't understand why the second part of this code is not printing one number by one in steps of 1 second but all at the end of the script (in this case 3 seconds after the beginning of the second part of the code)?
#!/usr/bin/python2.7
import time
x = 0
maxi = 999999 #I chose a big number to show them scroll
#this part will print all the number on the same line
while x <= maxi:
print '<{}>\r'.format(x),
x+=1
print
#this part will print the numbers with a sleep of 1 second for each print
x = 999998 #I initialized it at this number to only wait 3 seconds
while x <= maxi:
print '<{}>\r'.format(x),
x+=1
time.sleep(1)
print
That's probably because of buffering.
You can try running your program with python -u on the command line, as the man page says
-u Force stdin, stdout and stderr to be totally unbuffered. [...]
Or you can use import sys and put a sys.stdout.flush() just after the print().
If you're wondering why that's needed on the second print and not on the first one, it's most likely because of the sleep. It blocks execution, so nothing is updated / flushed. I think it's actually also happening on the first one but the updates are much more / faster, so you don't notice it.
I'm using the Enthought Canopy IDE to write Python and the the output of print commands do not reach the ipython output window at the time when I would expect them to. The clearest way to explain is to give an example:
import time
print 1
print 2
time.sleep(1)
print 3
for i in range(10):
print i
time.sleep(0.5)
The initial 1, 2 and 3 are displayed simultaneously after a second, and then there is a half second delay before the next 0 is displayed.
(as opposed to the expected instantaneous and simultaneous display of 1 and 2, then a 1 second delay before a simultaneous display of 3 and 0, then half second delays)
This is causing me issues as I am looking to get readouts of variables before a time consuming portion of script runs, and I cannot get it to display these readouts without terminating the program after calling the print command.
Any ideas/solutions?
I'm on Windows 7, let me know if you need any other specs/config details...
Thanks!
You can start the Python interpreter with an additional -u parameter, which gives you unbuffered output (i.e. instant).
Or you can call sys.stdout.flush() after every print. Maybe even wrapping it in a function like this:
from __future__ import print_function
import sys
def print_flush(*args):
print(*args)
sys.stdout.flush()
My overall goal of the scripts is to do a efficient ping of a /8 network. To do this I am running 32 scripts at the same time to scan a /13 each.
the basic of the script is
import ipaddress
import time
import fileinput
import pingpack
set = 0
mainset =0
while mainset < 8:
while set < 256:
net_addr = ("10.{}.{}.0/24".format(mainset,set))
ip_net = ipaddress.ip_network(net_addr)
all_hosts = list(ip_net.hosts())
f_out_file=('{}_Ping.txt'.format(mainset))
for i in range(len(all_hosts)):
output = pingpack.ping("{}".format(all_hosts[i]))
if output != None:
with open(f_out_file,'a',newline="\n") as file:
file.write("{}, Y\r\n".format(all_hosts[i]))
print ("{}".format("Subnet Complete"))
set = set + 1
set=0
The script it self works and runs and gives me a good output when ran by it self. The issue i am running into is when i get 32 of these running for each subnet they run for about 8 set loops before the python process locks up and stops writing.
The script I am using to start the 32 is as follows
from subprocess import Popen, PIPE
import time
i = 0
count=0
while i < 32:
process = Popen(['ping{}.py'.format(i),"{}".format(count)], stdout=PIPE, stderr=PIPE, shell=True)
print(count)
print(i)
i = i + 1
count=count+8
time.sleep(1)
In this case; Yes; I do have 32 duplicate scrips each with a 2 lines changed for the different /13 subents. It may be as effect as some; but it gets them started and running.
How would I go about finding the reason for the stop for these scripts?
Side note: Yes I know I can do this with something like NMAP or Angry IP Scanner; but they both take 90+ hours to scan a entire /8; I am trying to shorten this down to something that can be ran in a more reasonable timeframe.
Your first problem is that set is never set back to zero when you move on to the next mainset. Your second problem is that mainset never increments. Rather than a pair of obscure while-loops, why not:
for mainset in xrange(8):
for set in xrange(256):
Also, in range(len(all_hosts)) is a code smell (and it turns out that you never use i except to write all_hosts[i]). Why not:
for host in all_hosts:
Short version: I have a program that prints to the terminal successive integer numbers in an infinite loop. At some point, the terminal became black, and no output is visible, but I can still execute commands.
Details:
I read this answer in PCG and wanted to try it in Python. Here it is:
#!/bin/python2
class Dier(object):
def __init__(self):
global ct
ct += 1
print ct
def __del__(self):
Dier()
ct = 0
Dier()
This program loops indefinitely, printing the number of iterations on each step. Left overnight (we get to the tenths of millions in a matter of minutes) executed from an Ubuntu gnome terminal, the terminall just shows black. The program is still running, and new lines appear, but nothing is visible. I killed the program, but the terminal, including the command prompt, is black. I can input commands, and they work, but no output is visible.
Why is this happening?
Some information I provided in the comments:
The memory used by the program (reported by top) remains constant and low.
I can print absurdly large numbers to the terminal without getting this behaviour. The largest number I have printed has 10^10^6 digits (of course, the terminal has forgotten the beginning of it, and only shows the last digits, but its log10 is 1246124). My program couldn't have gone that far, that would take millions of times the age of the universe.
On an added note, if I try to print something ever bigger than that, it just seems to freeze, using the CPU but without any output (while the original output was there, but invisible).
Seems like the output is being buffered. You should try and change the print to
print ct, "\n"
or
print ct
sys.stdout.flush()
alternatively you can tell python to not buffer the output by calling it with -u parameter (see https://docs.python.org/2/using/cmdline.html#cmdoption-u) or setting PYTHONUNBUFFERED=1 env variable (see https://docs.python.org/2/using/cmdline.html#envvar-PYTHONUNBUFFERED)