I am running the following on Mac OSX and Python 3. I am running the snippet using Sublime Text 3's default build system for Python 3 and displaying the output in the same program's output window.
print('Starting test.')
for i in range(1,100):
print('This is a test. ' + str(i), end='\r')
print('Test done!')
My desired output is basically a repetition of that same line, overwriting the previous line in the console each time. By the end, I should ideally only see the following in my output window.
Starting test.
This is a test. 99
Test done!
I would expect the end='\r' gets me to this behavior, but it doesn't. Instead, I see:
Starting test.
This is a test. 1
This is a test. 2
This is a test. 3
[...]
This is a test. 99
Test done!
For background, if you use View Package File from the command palette to view Default/exec.py you can examine the code that's being used to execute programs and display their output into the panel.
As a short synopsis, the program is executed in a background thread, and the output of stdout and stderr are read from using this code:
def read_fileno(self, fileno, execute_finished):
decoder_cls = codecs.getincrementaldecoder(self.listener.encoding)
decoder = decoder_cls('replace')
while True:
data = decoder.decode(os.read(fileno, 2**16))
if len(data) > 0:
if self.listener:
self.listener.on_data(self, data)
else:
try:
os.close(fileno)
except OSError:
pass
if execute_finished and self.listener:
self.listener.on_finished(self)
break
That is, whenever they read any data, they invoke on_data() in the object that's listening for results, which in this case is the exec command itself. The implementation of that method looks like this:
def on_data(self, proc, data):
# Normalize newlines, Sublime Text always uses a single \n separator
# in memory.
data = data.replace('\r\n', '\n').replace('\r', '\n')
self.append_string(proc, data)
Internally Sublime uses \n as the line separator, so on windows \r\n is interpreted as \n, and on MacOS \r is interpreted as \n as well.
So on the one hand this doesn't do what you expect because your \r is effectively treated the same as \n is.
Aside of that, if you trace the call to append_string() through the guard code that keeps it thread safe and allows it to interact with the Sublime core, it ends up in a line that looks like this:
self.output_view.run_command(
'append',
{'characters': characters, 'force': True, 'scroll_to_end': True})
That is, whatever characters you provide, they're directly appended to the output buffer without regard to what they might contain.
If you were to provide a custom build target (a somewhat advanced plugin but still possible) this behaviour could be modified, although it might be tricky to get it to work on MacOS since there would be no way to distinguish what \r means on that platform.
The code displays correctly in a normal command line window. The problem is with Sublime's output window, which doesn't seem to be displaying the carriage return correctly (it replaces it with a newline).
Actually you need to add '\r' at the beginning of the string.
print('Starting test.')
for i in range(1,100):
print('\rThis is a test.'+str(i), end="")
print('\nTest done!')
Output
Starting test.
This is a test.99
Test done!
Related
num = 0
for i in range(5):
ask = int(input())
if abs(ask) > 0:
num = ask
print(num)
When I run the code, it lets me input the first string. But once I enter the second string, the program crashes and says "pythonProject\main.py", line 3, in
ask = int(input())
ValueError: invalid literal for int() with base 10: ''"
What's going on?
My input: I just type 1, press enter, type 2, press enter and then it crashes. I am sure it is not an error where I click enter too quickly, or I accidentally type in an empty string because I've ran the code multiple times.
What have I tried so far?
Creating a new project and pasting the code -> didn't work
Asking my friend to copy the code onto his PyCharm and run it -> worked fine on his computer
'Edit configurations', uncheck 'emulate code in output console' -> didn't work, it was already unchecked
Checked that I was running the correct file in the project -> didn't work, I was running the right file
EDIT:
FIXED, just needed to check 'Emulate code in output console' rather than uncheck it. Not sure why this works though, or how I can keep it checked for all future projects - rather than having to manually check it every time.
FIXED, just needed to check 'Emulate code in output console' rather than uncheck it. Not sure why this works though, or how I can keep it checked for all future projects - rather than having to manually check it every time.
The problem is with the input, as you are either pressing enter without any input (empty string) or you are entering a float value. This thread might help you in this case. The code is working fine when I input an integer and gives the same error when entered empty string or a float value.
To get it work you need to check "Emulate code in output console".
I've answered in the comments section and I'm glad it worked out, here is an explanation:
You need to know a concept of "terminal emulator" to understand why and how this works. When a program is ran (at least on UNIX-like operating systems), it has three I/O streams: stdin, stdout and stderr. The stdin is used to input data, and two others are for output.
Input or output stream is just a buffer used to communicate with the program back and forth. Once something is written to the buffer, it can be read from there. If the buffer is empty, an attempt to read from there will cause stall until the buffer has something in it. More about stdio: https://en.wikipedia.org/wiki/Standard_streams
When the program is ran through the terminal emulator, I/O streams are connected to this emulator, so whatever you type in the terminal window is written to your stdin by default. Whatever your program writes to the stdout and stderr is displayed on the screen. (However, this behavior may be changed using pipes, so you can pass data from some file to the stdin and also you can redirect the output to the file)
Here is the history behind terminal emulators, to understand, why is it implemented this way: https://en.wikipedia.org/wiki/Terminal_emulator#Computer_terminals
For example, you have a simple program:
s = input('Enter string: ')
print(f'stdout: {s}')
If you run it from the terminal and type "TEST":
$ python3 test.py
TEST
stdout: TEST
But you also can, for example, pass data directly to stdin, and redirect output to the file:
$ echo "ABCDEF" | python3 test.py > OUTPUT.txt
there will be no text in the terminal, but OUPUT.txt will appear. It will contain:
stdout: ABCDEF
Now, about PyCharm:
By default, when it runs your script, it does not automatically emulate terminal in the output window. It simply does not send anything to the stdin and it won't react to pressed keys. When your program gets to the line with input(), it starts to read the stdin stream until it gets \n character from the stream (indicating that user has pressed Return key). As nothing gets sent to the stream, it will wait infinitely.
Useful tip: for testing, instead of just typing something into the terminal every time, you can also check "Redirect input from:" and choose an input file.
I have a python script that performs a simulation. It takes a fairly long, varying time to run through each iteration, so I print a . after each loop as a way to monitor how fast it runs and how far it went through the for statement as the script runs. So the code has this general structure:
for step in steps:
run_simulation(step)
# Python 3.x version:
print('.', end='')
# for Python 2.x:
# print '.',
However, when I run the code, the dots do not appear one by one. Instead, all the dots are printed at once when the loop finishes, which makes the whole effort pointless. How can I print the dots inline as the code runs?
This problem can also occur when iterating over data fed from another process and trying to print results, for example to echo input from an Electron app. See Python not printing output.
The issue
By default, output from a Python program is buffered to improve performance. The terminal is a separate program from your code, and it is more efficient to store up text and communicate it all at once, rather than separately asking the terminal program to display each symbol.
Since terminal programs are usually meant to be used interactively, with input and output progressing a line at a time (for example, the user is expected to hit Enter to indicate the end of a single input item), the default is to buffer the output a line at a time.
So, if no newline is printed, the print function (in 3.x; print statement in 2.x) will simply add text to the buffer, and nothing is displayed.
Outputting in other ways
Every now and then, someone will try to output from a Python program by using the standard output stream directly:
import sys
sys.stdout.write('test')
This will have the same problem: if the output does not end with a newline, it will sit in the buffer until it is flushed.
Fixing the issue
For a single print
We can explicitly flush the output after printing.
In 3.x, the print function has a flush keyword argument, which allows for solving the problem directly:
for _ in range(10):
print('.', end=' ', flush=True)
time.sleep(.2) # or other time-consuming work
In 2.x, the print statement does not offer this functionality. Instead, flush the stream explicitly, using its .flush method. The standard output stream (where text goes when printed, by default) is made available by the sys standard library module, and is named stdout. Thus, the code will look like:
for _ in range(10):
print '.',
sys.stdout.flush()
time.sleep(.2) # or other time-consuming work
For multiple prints
Rather than flushing after every print (or deciding which ones need flushing afterwards), it is possible to disable the output line buffering completely. There are many ways to do this, so please refer to the linked question.
I'd like to use Python as an external process and communicate with it by standard input/standard output. Specifically, I want to
for line in sys.stdin:
result = compute_something(line)
print result
sys.stdout.flush()
The output flush is to get the result back right away, without buffering. I want to do the same with the input--- to process each line right away, without buffering. However, the above code does not respond to each line individually; it waits until a large amount of data is accumulated in the standard input and then processes everything at once.
This is true even if the calling program flushes its standard output with every line. It's also true even if I'm running the above directly on a console. The buffer is in Python, not the calling program.
Moreover, I found that control-D on the console makes Python flush its standard input buffer. (And then I can continue to send more input afterward!) However, that's not useful to me because the calling program can't send the equivalent of control-D at the end of each line.
One more thing: for line in sys.stdin.xreadlines() appears to be equivalent to for line in sys.stdin: they both buffer.
So my question is, how can I write a Python script that does not buffer its input, so that it processes each line of input right away?
(I solved the problem before posting the question, but I think I should share it anyway--- others might encounter this problem and I'm still interested in any comments on why this is happening and what we should all know about how to control Python's or Linux's implicit buffering.)
Here's one way to avoid input buffering:
while True:
line = sys.stdin.readline()
result = compute_something(line)
print result
sys.stdout.flush()
Apparently, .readline() avoids the input buffer while direct iteration and .xreadlines() do not.
I have a python script that performs a simulation. It takes a fairly long, varying time to run through each iteration, so I print a . after each loop as a way to monitor how fast it runs and how far it went through the for statement as the script runs. So the code has this general structure:
for step in steps:
run_simulation(step)
# Python 3.x version:
print('.', end='')
# for Python 2.x:
# print '.',
However, when I run the code, the dots do not appear one by one. Instead, all the dots are printed at once when the loop finishes, which makes the whole effort pointless. How can I print the dots inline as the code runs?
This problem can also occur when iterating over data fed from another process and trying to print results, for example to echo input from an Electron app. See Python not printing output.
The issue
By default, output from a Python program is buffered to improve performance. The terminal is a separate program from your code, and it is more efficient to store up text and communicate it all at once, rather than separately asking the terminal program to display each symbol.
Since terminal programs are usually meant to be used interactively, with input and output progressing a line at a time (for example, the user is expected to hit Enter to indicate the end of a single input item), the default is to buffer the output a line at a time.
So, if no newline is printed, the print function (in 3.x; print statement in 2.x) will simply add text to the buffer, and nothing is displayed.
Outputting in other ways
Every now and then, someone will try to output from a Python program by using the standard output stream directly:
import sys
sys.stdout.write('test')
This will have the same problem: if the output does not end with a newline, it will sit in the buffer until it is flushed.
Fixing the issue
For a single print
We can explicitly flush the output after printing.
In 3.x, the print function has a flush keyword argument, which allows for solving the problem directly:
for _ in range(10):
print('.', end=' ', flush=True)
time.sleep(.2) # or other time-consuming work
In 2.x, the print statement does not offer this functionality. Instead, flush the stream explicitly, using its .flush method. The standard output stream (where text goes when printed, by default) is made available by the sys standard library module, and is named stdout. Thus, the code will look like:
for _ in range(10):
print '.',
sys.stdout.flush()
time.sleep(.2) # or other time-consuming work
For multiple prints
Rather than flushing after every print (or deciding which ones need flushing afterwards), it is possible to disable the output line buffering completely. There are many ways to do this, so please refer to the linked question.
I've developed a Python script that performs several tasks in a row (mainly connecting to servers and retrieving information).
There are many steps, and for each of them I would like to display a dot, so that the user knows there is something going on.
At the end of each step, I do:
print('.', end='')
And in the final step, I write:
print('Done!')
It works, except nothing is displayed until the final print is executed, so it kind of defeats its original purpose :)
Basically, nothing is displayed on the screen, and at the very last moment, this pops up:
.......Done!
How can I force Python to print on the same line step after step?
By default, stdout is line buffered, meaning the buffer won't be flushed until you write a newline.
Flush the buffer explicitly each time you print a '.':
print('.', end='', flush=True)
The flush keyword was added in Python 3.3; for older versions, use sys.stdout.flush().
From the print() function documentation:
Whether output is buffered is usually determined by file, but if the flush keyword argument is true, the stream is forcibly flushed.
and from the sys.stdout documentation (the default value for the file argument of the print() function):
When interactive, standard streams are line-buffered. Otherwise, they are block-buffered like regular text files.
Instead of using print, you can write directly to stdout, (unbuffered):
import sys
import time
for i in range (10):
time.sleep (0.5)
sys.stdout.write('.')
print 'Done!'
for python 2.7.3 you can left a trailing comma which tells the idle to not insert a line after the statement is printed for example
print "hello",
print "world"
would return
>>> hello world