Develop Reference

Python's print statement followed by sleep(...) function gives unexpected output

Here's my Python 3 code:
from time import sleep
s='what is your name'
for x in s:
print(x,end='')
sleep(1)
What I expect is that, each letter will be printed one by one after a delay of one second. But when I run this program, it takes a time of len(s) seconds and then prints the value of s.
Can anyone explain to me what is actually happening.

Use flush argument to print function:
print(x, end='', flush=True)
From docs:
Whether output is buffered is usually determined by file, but if the
flush keyword argument is true, the stream is forcibly flushed.

subprocess call avoid stopping after error

I call a program a subprocess.call() in a for-loop and then it does something. However, after the called program crashes (no, I can't avoid that) the python script terminates.
The code:
with open("runtests.output", "w") as output:
for i in range(1,5):
for j in range(2,5):
subprocess.call(["runprogram", "with", "arguments"])
output.write(str(i) + str(j))
output.write(str(resource.getrusage(resource.RUSAGE_CHILDREN)))
output.write("\n")
In this case, runprogram with arguments fails at some point, i.e. at the third loop in the inner for-loop, and then the whole script stops. I, however, want the script to just continue calling subrpocesses until all the loops have looped.
So how to avoid this behavior?
EDIT:
Tried
try:
call and writes here
except:
print "Ohai"
The actual output is two times an ABORT which is negated, which are in the main process of runprogram, but then an ABORT comes which is from a child process of runprogram and then the whole script fails. Maybe that has something to do with it?
Also, "Ohai" isn't printed...

taking multiline input with sys.stdin

I have the following function:
def getInput():
# define buffer (list of lines)
buffer = []
run = True
while run:
# loop through each line of user input, adding it to buffer
for line in sys.stdin.readlines():
if line == 'quit\n':
run = False
else:
buffer.append(line.replace('\n',''))
# return list of lines
return buffer
which is called in my function takeCommands(), which is called to actually run my program.
However, this doesn't do anything. I'm hoping to add each line to an array, and once a line == 'quit' it stops taking user input. I've tried both for line in sys.stdin.readlines() and for line sys.stdin, but neither of them register any of my input (I'm running it in Windows Command Prompt). Any ideas? Thanks

So, took your code out of the function and ran some tests.
import sys
buffer = []
while run:
line = sys.stdin.readline().rstrip('\n')
if line == 'quit':
run = False
else:
buffer.append(line)
print buffer
Changes:
Removed the 'for' loop
Using 'readline' instead of 'readlines'
strip'd out the '\n' after input, so all processing afterwards is much easier.
Another way:
import sys
buffer = []
while True:
line = sys.stdin.readline().rstrip('\n')
if line == 'quit':
break
else:
buffer.append(line)
print buffer
Takes out the 'run' variable, as it is not really needed.

I'd use itertools.takewhile for this:
import sys
import itertools
print list(itertools.takewhile(lambda x: x.strip() != 'quit', sys.stdin))
Another way to do this would be to use the 2-argument iter form:
print list(iter(raw_input,'quit'))
This has the advantage that raw_input takes care of all of the line-buffering issues and it will strip the newlines for you already -- But it will loop until you run out of memory if the user forgets to add a quit to the script.
Both of these pass the test:
python test.py <<EOF
foo
bar
baz
quit
cat
dog
cow
EOF

There are multiple separate problems with this code:
while run:
# loop through each line of user input, adding it to buffer
for line in sys.stdin.readlines():
if line == 'quit':
run = False
First, you have an inner loop that won't finish until all lines have been processed, even if you type "quit" at some point. Setting run = False doesn't break out of that loop. Instead of quitting as soon as you type "quit", it will keep going until it's looked at all of the lines, and then quit if you typed "quit" at any point.
You can fix this one pretty easily by adding a break after the run = False.
But, with or without that fix, if you didn't type "quit" during that first time through the outer loop, since you've already read all input, there's nothing else to read, so you'll just keep running an empty inner loop over and over forever that you can never exit.
You have a loop that means "read and process all the input". You want to do that exactly once. So, what should the outer loop be? It should not be anyway; the way to do something once is to not use a loop. So, to fix this one, get rid of run and the while run: loop; just use the inner loop.
Then, if you type "quit", line will actually be "quit\n", because readlines does not strip newlines.
You fix this one by either testing for "quit\n", or stripping the lines.
Finally, even if you fix all of these problems, you're still waiting forever before doing anything. readlines returns a list of lines. The only way it can possibly do that is by reading all of the lines that will ever be on stdin. You can't even start looping until you've read all those lines.
When standard input is a file, that happens when the file ends, so it's not too terrible. But when standard input is the Windows command prompt, the command prompt never ends.* So, this takes forever. You don't get to start processing the list of lines, because it takes forever to wait for the list of lines.
The solution is to not use readlines(). Really, there is never a good reason to call readlines() on anything, stdin or not. Anything that readlines works on is already an iterable full of lines, just like the list that readlines would give you, except that it's "lazy": it can give you the lines one at a time, instead of waiting and giving you all of them at once. (And even if you really need the list, just do list(f) instead of f.readlines().)
So, instead of for line in sys.stdin.readlines():, just do for line in sys.stdin: (Or, better, replace the explicit loop completely and use a sequence of iterator transformations, as in mgilson's answer.)
The fixes JBernardo, Wing Tang Wong, etc. proposed are all correct, and necessary. The reason none of them fixed your problems is that if you have 4 bugs and fix 1, your code still doesn't work. That's exactly why "doesn't work" isn't a useful measure of anything in programming, and you have to debug what's actually going wrong to know whether you're making progress.
* I lied a bit about stdin never being finished. If you type a control-Z (you may or may not need to follow it with a return), then stdin is finished. But if your assignment is to make it quit as soon as the user types "quit"< turning in something that only quits when the user types "quit" and then return, control-Z, return again probably won't be considered successful.

Keyboard input between select() in Python

I write some codes to get the input from keyboard and also check something is alive or not:
import sys
from select import select
timeout = 10
while is_alive(): # is_alive is a method to check some stuffs, might take 5 secs
rlist, _, _ = select([sys.stdin], [], [], timeout)
if rlist:
s = sys.stdin.readline()
print repr(s)
handle(s) # handle is a method to handle and react according to input s
I found that when the keyboard input ends outside of the waiting in select() (usually it ends during the 5 secs of is_alive()), the if rlist: will get false.
I can understand why but I don't know how to solve it.
And there is still another question related to the situation mentioned above, sometimes readline() will return the last line of my input when some inputs are located across different select() waiting.
That means, if I enter 'abc\n' and unfortunately the '\n' located outside of wating in select() (that means, when I press Enter, the program are executing other parts, such as is_alive()), and then if I enter 'def\n' and this time the Enter pressed successfully located within select(), I'll see the s from readline() becomes 'def\n' and the first line is disappeared.
Is there any good solution to solve two issues above? I'm using FreeBSD 9.0.

As your code in is_alive() calls ssh, this will eat up the stdin.
Try starting ssh with the -n option or with a re-directed stdin.
The latter would work with
sp = subprocess.Popen(..., stdin=subprocess.PIPE)
sp.stdin.close()

Usage of sys.stdout.flush() method

What does sys.stdout.flush() do?

Python's standard out is buffered (meaning that it collects some of the data "written" to standard out before it writes it to the terminal). Calling sys.stdout.flush() forces it to "flush" the buffer, meaning that it will write everything in the buffer to the terminal, even if normally it would wait before doing so.
Here's some good information about (un)buffered I/O and why it's useful:
http://en.wikipedia.org/wiki/Data_buffer
Buffered vs unbuffered IO

Consider the following simple Python script:
import time
import sys
for i in range(5):
print(i),
#sys.stdout.flush()
time.sleep(1)
This is designed to print one number every second for five seconds, but if you run it as it is now (depending on your default system buffering) you may not see any output until the script completes, and then all at once you will see 0 1 2 3 4 printed to the screen.
This is because the output is being buffered, and unless you flush sys.stdout after each print you won't see the output immediately. Remove the comment from the sys.stdout.flush() line to see the difference.

As per my understanding, When ever we execute print statements output will be written to buffer. And we will see the output on screen when buffer get flushed(cleared). By default buffer will be flushed when program exits. BUT WE CAN ALSO FLUSH THE BUFFER MANUALLY by using "sys.stdout.flush()" statement in the program. In the below code buffer will be flushed when value of i reaches 5.
You can understand by executing the below code.
chiru#online:~$ cat flush.py
import time
import sys
for i in range(10):
print i
if i == 5:
print "Flushing buffer"
sys.stdout.flush()
time.sleep(1)
for i in range(10):
print i,
if i == 5:
print "Flushing buffer"
sys.stdout.flush()
chiru#online:~$ python flush.py
0 1 2 3 4 5 Flushing buffer
6 7 8 9 0 1 2 3 4 5 Flushing buffer
6 7 8 9

import sys
for x in range(10000):
print "HAPPY >> %s <<\r" % str(x),
sys.stdout.flush()

As per my understanding sys.stdout.flush() pushes out all the data that has been buffered to that point to a file object.
While using stdout, data is stored in buffer memory (for some time or until the memory gets filled) before it gets written to terminal. Using flush() forces to empty the buffer and write to terminal even before buffer has empty space.

You can see the differences b/w these two
import sys
for i in range(1,10 ):
sys.stdout.write(str(i))
sys.stdout.flush()
for i in range(1,10 ):
print i
In the first case, the characters are output one by one after each is written, because of the flush. In the second case, the characters are buffered by Python until it thinks it's got something worth the effort to write, and then written all in a batch.
If you add, say, a time.sleep(0.2) in the loops, this becomes more obvious.

Imagine you have a toy box where you keep all your toys. When you want to play with a toy, you usually take it out of the box, right?
In a similar way, when you write something on the computer, it gets stored in a temporary place called a buffer. Think of it like a toy box for computer data. The computer waits until the toy box is full before it takes the data out of the box and shows it to you on the screen.
However, sometimes you might want to play with a toy right away, without waiting for the toy box to fill up with other toys. This is like when you call sys.stdout.flush() in Python. It tells the computer to take the data out of the buffer and show it to you on the screen right away, without waiting for the buffer to fill up.