taking multiline input with sys.stdin - python

I have the following function:
def getInput():
# define buffer (list of lines)
buffer = []
run = True
while run:
# loop through each line of user input, adding it to buffer
for line in sys.stdin.readlines():
if line == 'quit\n':
run = False
else:
buffer.append(line.replace('\n',''))
# return list of lines
return buffer
which is called in my function takeCommands(), which is called to actually run my program.
However, this doesn't do anything. I'm hoping to add each line to an array, and once a line == 'quit' it stops taking user input. I've tried both for line in sys.stdin.readlines() and for line sys.stdin, but neither of them register any of my input (I'm running it in Windows Command Prompt). Any ideas? Thanks

So, took your code out of the function and ran some tests.
import sys
buffer = []
while run:
line = sys.stdin.readline().rstrip('\n')
if line == 'quit':
run = False
else:
buffer.append(line)
print buffer
Changes:
Removed the 'for' loop
Using 'readline' instead of 'readlines'
strip'd out the '\n' after input, so all processing afterwards is much easier.
Another way:
import sys
buffer = []
while True:
line = sys.stdin.readline().rstrip('\n')
if line == 'quit':
break
else:
buffer.append(line)
print buffer
Takes out the 'run' variable, as it is not really needed.

I'd use itertools.takewhile for this:
import sys
import itertools
print list(itertools.takewhile(lambda x: x.strip() != 'quit', sys.stdin))
Another way to do this would be to use the 2-argument iter form:
print list(iter(raw_input,'quit'))
This has the advantage that raw_input takes care of all of the line-buffering issues and it will strip the newlines for you already -- But it will loop until you run out of memory if the user forgets to add a quit to the script.
Both of these pass the test:
python test.py <<EOF
foo
bar
baz
quit
cat
dog
cow
EOF

There are multiple separate problems with this code:
while run:
# loop through each line of user input, adding it to buffer
for line in sys.stdin.readlines():
if line == 'quit':
run = False
First, you have an inner loop that won't finish until all lines have been processed, even if you type "quit" at some point. Setting run = False doesn't break out of that loop. Instead of quitting as soon as you type "quit", it will keep going until it's looked at all of the lines, and then quit if you typed "quit" at any point.
You can fix this one pretty easily by adding a break after the run = False.
But, with or without that fix, if you didn't type "quit" during that first time through the outer loop, since you've already read all input, there's nothing else to read, so you'll just keep running an empty inner loop over and over forever that you can never exit.
You have a loop that means "read and process all the input". You want to do that exactly once. So, what should the outer loop be? It should not be anyway; the way to do something once is to not use a loop. So, to fix this one, get rid of run and the while run: loop; just use the inner loop.
Then, if you type "quit", line will actually be "quit\n", because readlines does not strip newlines.
You fix this one by either testing for "quit\n", or stripping the lines.
Finally, even if you fix all of these problems, you're still waiting forever before doing anything. readlines returns a list of lines. The only way it can possibly do that is by reading all of the lines that will ever be on stdin. You can't even start looping until you've read all those lines.
When standard input is a file, that happens when the file ends, so it's not too terrible. But when standard input is the Windows command prompt, the command prompt never ends.* So, this takes forever. You don't get to start processing the list of lines, because it takes forever to wait for the list of lines.
The solution is to not use readlines(). Really, there is never a good reason to call readlines() on anything, stdin or not. Anything that readlines works on is already an iterable full of lines, just like the list that readlines would give you, except that it's "lazy": it can give you the lines one at a time, instead of waiting and giving you all of them at once. (And even if you really need the list, just do list(f) instead of f.readlines().)
So, instead of for line in sys.stdin.readlines():, just do for line in sys.stdin: (Or, better, replace the explicit loop completely and use a sequence of iterator transformations, as in mgilson's answer.)
The fixes JBernardo, Wing Tang Wong, etc. proposed are all correct, and necessary. The reason none of them fixed your problems is that if you have 4 bugs and fix 1, your code still doesn't work. That's exactly why "doesn't work" isn't a useful measure of anything in programming, and you have to debug what's actually going wrong to know whether you're making progress.
* I lied a bit about stdin never being finished. If you type a control-Z (you may or may not need to follow it with a return), then stdin is finished. But if your assignment is to make it quit as soon as the user types "quit"< turning in something that only quits when the user types "quit" and then return, control-Z, return again probably won't be considered successful.

Related

Why isn't python printing statement above infinite loop?

The output of this code is just bb. I want to know why cc is not printed, meaning that the third line is not executed which is print("cc").
This is the link - https://ideone.com/M7LyS3
print("bb")
s2=input()
print("cc")
while 0>-50:
pass
print("qq")
Input = 5
print does not flush by default. Here, your first print in queued, and then flushed when you call input(). The second print is queued, but is never flushed.
If you add the optional flush argument, you'll get the behavior you expect:
print("bb", flush = True)
s2=input()
print("cc"", flush = True)
while 0>-50:
pass
print("qq"", flush = True) # Will never happen because of the infinite loop, though
Try copying your code in this online compiler here:
https://www.onlinegdb.com/online_python_compiler
It works as you expected.
The reason it's not printing cc on your machine might be because of some issues or race condition of the buffer that prints output. The program gets busy in that infinite loop and is holding the buffer to be written to the terminal/console.
The code is running fine:
print("bb")
s2=input()
print("cc")
while 0>-50:
pass
print("qq")
Output:
bb
my_input
cc

What is CLI-Loop ? What's the difference with normal loop?

I was doing some research about Python.
And I saw something like this.
# Start CLI-Loop
while True:
try:
text = raw_input()
except:
text = error()
if text == condition_1:
do_Some_Other_Things_1()
break
elif text == condition_2:
do_Some_Other_Things_2()
Is CLI-Loop stands for "Command Line Interface Loop" ?
If not, what does it mean?
What's so special about it?
There is nothing special about the loop; the author simply introduces the code block, stating that it'll interpret commands.
Which is exactly what the loop does; using raw_input(), it asks for user input from the terminal, then executes functions based on the input. In other words, it takes commands, interfacing with the user.
CLI indeed stands for Command Line Interface. There's nothing special about this loop, it's just called "the CLI loop" to indicate it's a loop handling the input taken from the command line.

Piping in Python 2.6 to a file with while loop

I'm completely new to Python piping but I think it's the answer to my problem. I've got a file cleverbot.py that takes endless input from raw_input and prints out the results to the command line. This is done while staying in a while loop the whole time, and when you say "bye" the script exits.
Now, can someone give me an example where I can pipe inputs to this script and then receive the output, without it ending the loop? The loop acts as a session and it's important that the script continues to run constantly inside this while loop while being able to have inputs piped to it.
Here is a snippet of the code. For anyone that wants to get the full thing its the pycleverbot library.
def main():
import sys
cb = Session()
q = ''
while q != 'bye':
try:
q = raw_input("> ")
except KeyboardInterrupt:
print
sys.exit()
print cb.Ask(q)
The cb = Session() can't be recreated at all after it's first made. That's why the while loop is in place. I lack the necessary logic to see a way of not using a while loop.
Hope this makes sense, thanks.
The trick is to not use a while loop in the first place.
for line in sys.stdin:
if line.lower().strip() == 'bye':
break
sys.stdout.write(do_something(line))
sys.stdout.flush()

Can't kill my python code. What's wrong?

Okay, so I'm writing a very simplistic password cracker in python that brute forces a password with alphanumeric characters. Currently this code only supports 1 character passwords and a password file with a md5 hashed password inside. It will eventually include the option to specify your own character limits (how many characters the cracker tries until it fails). Right now I cannot kill this code when I want it to die. I have included a try and except snippit, however it's not working. What did I do wrong?
Code: http://pastebin.com/MkJGmmDU
import linecache, hashlib
alphaNumeric = ["a","b","c","d","e","f","g","h","i","j","k","l","m","n","o","p","q","r","s","t","u","v","w","x","y","z","A","B","C","D","E","F","G","H","I","J","K","L","M","N","O","P","Q","R","S","T","U","V","W","X","Y","Z",1,2,3,4,5,6,7,8,9,0]
class main:
def checker():
try:
while 1:
if hashlib.md5(alphaNumeric[num1]) == passwordHash:
print "Success! Your password is: " + str(alphaNumeric[num1])
break
except KeyboardInterrupt:
print "Keyboard Interrupt."
global num1, passwordHash, fileToCrack, numOfChars
print "What file do you want to crack?"
fileToCrack = raw_input("> ")
print "How many characters do you want to try?"
numOfChars = raw_input("> ")
print "Scanning file..."
passwordHash = linecache.getline(fileToCrack, 1)[0:32]
num1 = 0
checker()
main
The way to allow a KeyboardInterrupt to end your program is to do nothing. They work by depending on nothing catching them in an except block; when an exception bubbles all the way out of a program (or thread), it terminates.
What you have done is to trap the KeyboardInterrupts and handle them by printing a message and then continuing.
As for why the program gets stuck, there is nothing that ever causes num1 to change, so the md5 calculation is the same calculation every time. If you wanted to iterate over the symbols in alphaNumeric, then do that: for symbol in alphaNumeric: # do something with 'symbol'.
Of course, that will still only consider every possible one-character password. You're going to have to try harder than that... :)
I think you're also confused about the use of classes. Python does not require you to wrap everything inside a class. The main at the end of your program does nothing useful; your code runs because it is evaluated when the compiler tries to figure out what a main class is. This is an abuse of syntax. What you want to do is put this code in a main function, and call the function (the same way you call checker currently).
Besides printing, you need to actually exit your program when capturin KeyboardInterrupt, you're only printing a message.
This is what worked for me...
import sys
try:
....code that hangs....
except KeyboardInterrupt:
print "interupt"
sys.exit()
Well, when you use that try and except block, the error is raised when that error occurs. In your case, KeyboardInterrupt is your error here. But when KeyboardInterrupt is activated, nothing happens. This due to having nothing in the except part. You could do this after importing sys:
try:
#Your code#
except KeyboardInterrupt:
print 'Put Text Here'
sys.exit()
sys.exit() is an easy way to safely exit the program. This can be used for making programs with passwords to end the program if the password is wrong or something like that. That should fix the except part. Now to the try part:
If you have break as the end of the try part, nothing is going to happen. Why? Because break only works on loops, most people tend to do it for while loops. Let's make some examples. Here's one:
while 1:
print 'djfgerj'
break
The break statement will stop and end the loop immediately unlike its brother continue, which continues the loop. That's just extra information. Now if you have break in a something like this:
if liners == 0:
break
That's going to depend where that if statement is. If it is in a loop, it is going to stop the loop. If not, nothing is going to happen. I am assuming you made an attempt to exit the function which didn't work. It looks like the program should end, so use sys.exit() like I showed you above. Also, you should group that last piece of code (in the class) into a seperate function. I hope this helps you!

What's the benefit of using generator in this case?

I'm learning Python's generator from this slide: http://www.dabeaz.com/generators/Generators.pdf
There is an example in it, which can be describe like this:
You have a log file called log.txt, write a program to watch the content of it, if there are new line added to it, print them. Two solutions:
1. with generator:
import time
def follow(thefile):
while True:
line = thefile.readline()
if not line:
time.sleep(0.1)
continue
yield line
logfile = open("log.txt")
loglines = follow(logfile)
for line in loglines:
print line
2. Without generator:
import time
logfile = open("log.txt")
while True:
line = logfile.readline()
if not line:
time.sleep(0.1)
continue
print line
What's the benefit of using generator here?
If all you have is a hammer, everything looks like a nail
I'd almost just like to answer this question with just the above quote. Just because you can does not mean you need to all the time.
But conceptually the generator version separates functionality, the follow function serves the purpose of encapsulating the continuous reading from a file while waiting for new input. Which frees you to do anything in your loop with the new line that you want. In the second version the code to read from the file, and to print out is intermingled with the control loop. This might not be really an issue in this small example but that is something you might want to think about.
One benefit is the ability to pass your generator around (say to different functions) and iterate manually by calling .next(). Here is a slightly modified version of your initial generator example:
import time
def follow(file_name):
with open(file_name, 'rb') as f:
for line in f:
if not line:
time.sleep(0.1)
continue
yield line
loglines = follow(logfile)
first_line = loglines.next()
second_line = loglines.next()
for line in loglines:
print line
First of all I opened the file with a context manager (with statement, which auto-closes the file when you're done with it, or on exception). Next, at the bottom I've demonstrated using the .next() method, allowing you to manually step through. This can be useful sometimes if you need to break logic out from a simple for item in gen loop.
A generator function is defined like a normal function, but whenever it needs to generate a value, it does so with the yield keyword rather than return. Its main advantage is it allows its code to produce a series of values over time, rather than computing them at once and sending them back like a list.For example
# A Python program to generate squares from 1
# to 100 using yield and therefore generator
# An infinite generator function that prints
# next square number. It starts with 1
def nextSquare():
i = 1;
# An Infinite loop to generate squares
while True:
yield i*i
i += 1 # Next execution resumes
# from this point
# Driver code to test above generator
# function
for num in nextSquare():
if num > 100:
break
print(num)
Return sends a specified value back to its caller whereas Yield can produce a sequence of values. We should use yield when we want to iterate over a sequence, but don’t want to store the entire sequence in memory.
Ideally most loops are roughly of the form:
for element in get_the_next_value():
process(element)
However sometimes (as in your example #2), the loop is actually more complex as you sometimes get an element and sometimes don't. That means in your example without the element you have mixed up code for generating an element with the code for processing it. It doesn't show too clearly in the example because the code to generate the next value isn't actually too complex and the processing is just one line, but example number 1 is separating these two concepts more cleanly.
A better example might be one that processes variable length paragraphs from a file with blank lines separating each paragraph: try writing code for that with and without generators and you should see the benefit.
While your example might be a bit simple to fully take advantage of generators, I prefer to use generators to encapsulate the generation of any sequence data where there is also some kind of filtering of the data. It keeps the 'what I'm doing with the data' code separated from the 'how I get the data' code.

Categories

Resources