This happens if a prompt is written using sys.stdout.write or print instead of raw_input. The following script demonstrates it:
$ cat overwrite.py
import readline, sys
if 'libedit' in readline.__doc__:
readline.parse_and_bind('bind ^I rl_complete')
else:
readline.parse_and_bind('tab: complete')
def set_completer(choices):
choices = sorted(map(str,choices))
def completer(txt, state):
if state == 0:
completer.options = [c for c in choices if c.startswith(txt)]
if state < len(completer.options):
return completer.options[state]
return None
readline.set_completer(completer)
set_completer(['foo','flup'])
sys.stdout.write('input: ')
x = raw_input()
print x
If you run python overwrite.py, you get the expected prompt: "input: ". If you hit backspace once, nothing gets deleted (readline thinks it's already at the beginning of the line, I guess). If you hit 'f' then backspace, however, the whole line, prompt included, is wiped.
It would be very inconvenient to have to go through and replace all the places I write to stdout and am expecting to get input from the user with calls to raw_input, so I hope it's not necessary to use raw_input. The python docs are uncharacteristically sparse concerning readline.
There is no other real way to fix this; while readline has the rl_already_prompted variable, it still requires the prompt to be passed in so that readline's functions can manage the input line properly.
Related
num = 0
for i in range(5):
ask = int(input())
if abs(ask) > 0:
num = ask
print(num)
When I run the code, it lets me input the first string. But once I enter the second string, the program crashes and says "pythonProject\main.py", line 3, in
ask = int(input())
ValueError: invalid literal for int() with base 10: ''"
What's going on?
My input: I just type 1, press enter, type 2, press enter and then it crashes. I am sure it is not an error where I click enter too quickly, or I accidentally type in an empty string because I've ran the code multiple times.
What have I tried so far?
Creating a new project and pasting the code -> didn't work
Asking my friend to copy the code onto his PyCharm and run it -> worked fine on his computer
'Edit configurations', uncheck 'emulate code in output console' -> didn't work, it was already unchecked
Checked that I was running the correct file in the project -> didn't work, I was running the right file
EDIT:
FIXED, just needed to check 'Emulate code in output console' rather than uncheck it. Not sure why this works though, or how I can keep it checked for all future projects - rather than having to manually check it every time.
FIXED, just needed to check 'Emulate code in output console' rather than uncheck it. Not sure why this works though, or how I can keep it checked for all future projects - rather than having to manually check it every time.
The problem is with the input, as you are either pressing enter without any input (empty string) or you are entering a float value. This thread might help you in this case. The code is working fine when I input an integer and gives the same error when entered empty string or a float value.
To get it work you need to check "Emulate code in output console".
I've answered in the comments section and I'm glad it worked out, here is an explanation:
You need to know a concept of "terminal emulator" to understand why and how this works. When a program is ran (at least on UNIX-like operating systems), it has three I/O streams: stdin, stdout and stderr. The stdin is used to input data, and two others are for output.
Input or output stream is just a buffer used to communicate with the program back and forth. Once something is written to the buffer, it can be read from there. If the buffer is empty, an attempt to read from there will cause stall until the buffer has something in it. More about stdio: https://en.wikipedia.org/wiki/Standard_streams
When the program is ran through the terminal emulator, I/O streams are connected to this emulator, so whatever you type in the terminal window is written to your stdin by default. Whatever your program writes to the stdout and stderr is displayed on the screen. (However, this behavior may be changed using pipes, so you can pass data from some file to the stdin and also you can redirect the output to the file)
Here is the history behind terminal emulators, to understand, why is it implemented this way: https://en.wikipedia.org/wiki/Terminal_emulator#Computer_terminals
For example, you have a simple program:
s = input('Enter string: ')
print(f'stdout: {s}')
If you run it from the terminal and type "TEST":
$ python3 test.py
TEST
stdout: TEST
But you also can, for example, pass data directly to stdin, and redirect output to the file:
$ echo "ABCDEF" | python3 test.py > OUTPUT.txt
there will be no text in the terminal, but OUPUT.txt will appear. It will contain:
stdout: ABCDEF
Now, about PyCharm:
By default, when it runs your script, it does not automatically emulate terminal in the output window. It simply does not send anything to the stdin and it won't react to pressed keys. When your program gets to the line with input(), it starts to read the stdin stream until it gets \n character from the stream (indicating that user has pressed Return key). As nothing gets sent to the stream, it will wait infinitely.
Useful tip: for testing, instead of just typing something into the terminal every time, you can also check "Redirect input from:" and choose an input file.
I'm writing a Python script in which at a certain point, the user is asked to enter a key combination (by simultaneously pressing those keys) and to confirm with enter. Grabbing the key combination and the enter key is done using the keyboard module and thus it works whether the command line is in focus or not. After that, an input() prompt is shown in which they need to enter some text.
The problem is that this input prompt gets 'skipped'. I assume this is because there are some characters and an enter stored which get entered into the input as soon as it appears.
How could I fix this? There's an os.system('cls') before the input, but that doesn't help. I've also tried adding a throwaway input() before the main prompt, this sort of worked but I'm looking for more of a general solution that also works if the characters aren't followed by an enter.
Found it, looks like I wasn't using the correct search term.
This function clears the input buffer on both Windows and UNIX:
def flush_input():
try:
import msvcrt
while msvcrt.kbhit():
msvcrt.getch()
except ImportError:
import sys, termios #for linux/unix
termios.tcflush(sys.stdin, termios.TCIOFLUSH)
Source
Since I'm already importing the module at the top of my file and I only need Windows support for now, this is what I'm using:
def flush_input():
while msvcrt.kbhit():
msvcrt.getch()
I am running the following on Mac OSX and Python 3. I am running the snippet using Sublime Text 3's default build system for Python 3 and displaying the output in the same program's output window.
print('Starting test.')
for i in range(1,100):
print('This is a test. ' + str(i), end='\r')
print('Test done!')
My desired output is basically a repetition of that same line, overwriting the previous line in the console each time. By the end, I should ideally only see the following in my output window.
Starting test.
This is a test. 99
Test done!
I would expect the end='\r' gets me to this behavior, but it doesn't. Instead, I see:
Starting test.
This is a test. 1
This is a test. 2
This is a test. 3
[...]
This is a test. 99
Test done!
For background, if you use View Package File from the command palette to view Default/exec.py you can examine the code that's being used to execute programs and display their output into the panel.
As a short synopsis, the program is executed in a background thread, and the output of stdout and stderr are read from using this code:
def read_fileno(self, fileno, execute_finished):
decoder_cls = codecs.getincrementaldecoder(self.listener.encoding)
decoder = decoder_cls('replace')
while True:
data = decoder.decode(os.read(fileno, 2**16))
if len(data) > 0:
if self.listener:
self.listener.on_data(self, data)
else:
try:
os.close(fileno)
except OSError:
pass
if execute_finished and self.listener:
self.listener.on_finished(self)
break
That is, whenever they read any data, they invoke on_data() in the object that's listening for results, which in this case is the exec command itself. The implementation of that method looks like this:
def on_data(self, proc, data):
# Normalize newlines, Sublime Text always uses a single \n separator
# in memory.
data = data.replace('\r\n', '\n').replace('\r', '\n')
self.append_string(proc, data)
Internally Sublime uses \n as the line separator, so on windows \r\n is interpreted as \n, and on MacOS \r is interpreted as \n as well.
So on the one hand this doesn't do what you expect because your \r is effectively treated the same as \n is.
Aside of that, if you trace the call to append_string() through the guard code that keeps it thread safe and allows it to interact with the Sublime core, it ends up in a line that looks like this:
self.output_view.run_command(
'append',
{'characters': characters, 'force': True, 'scroll_to_end': True})
That is, whatever characters you provide, they're directly appended to the output buffer without regard to what they might contain.
If you were to provide a custom build target (a somewhat advanced plugin but still possible) this behaviour could be modified, although it might be tricky to get it to work on MacOS since there would be no way to distinguish what \r means on that platform.
The code displays correctly in a normal command line window. The problem is with Sublime's output window, which doesn't seem to be displaying the carriage return correctly (it replaces it with a newline).
Actually you need to add '\r' at the beginning of the string.
print('Starting test.')
for i in range(1,100):
print('\rThis is a test.'+str(i), end="")
print('\nTest done!')
Output
Starting test.
This is a test.99
Test done!
On Windows (specifically Win Server 2008 R2), I need to repeatedly execute an existing python script that comes with our product repeated. The intention of this script was to be called occasionally and the input is expected to be manual. However, I end up having to call this script hundreds of times.
So, I'm trying to automate the calls to this script (and other related scripts) with an additional python script. Where I'm getting hung up is that the "out of the box" script I am calling uses getpass.getpass() for password input.
In my automation script, I've tried using subrocess pipe.communicate to pass the password values to the base script. But I can't get it to work. Here's the relevant code in my automation script:
p = Popen(coreScriptCmd, stdout=PIPE, stdin=PIPE, stderr=STDOUT)
x = p.stdout.readline().rstrip()
print x #for debugging
x = p.communicate(args.pwd1+"\n"+args.pwd2)[0].rstrip()
print x #for debugging
As I said though, this doesn't work when the subprocess being called is using getpass.getpass() to ask for it's input. Here's the if statement in the core code where I'm running into trouble:
elif cmd == 'update-user':
if 'password1' not in globals():
password1 = getpass.getpass(mgmtusername + " password:")
if 'dbpassword' not in globals():
dbpassword = getpass.getpass(dbusername + " password:")
checkAccessDb(hostname, database, mgmtusername, password1, dbusername, dbpassword)
Does anyone have any suggestion on how to programmaticly pass values to getpass() in the subscript?
Alright, so I'm not sure what the original script looks like. But, in the case that it will still need to be usable from the command line, I would recommend this.
I would modify the original script to accept an argument. For example, let's say that the getpass is inside a function like this...
def run_script():
paswd = getpass.getpass("Please enter the password:")
Try modifying it to something like this:
def run_script(cmdlin = True):
if cmdlin:
paswd = getpass.getpass("Please enter the password:")
else:
# get password using another method
The other method could be anything you choose, pass it as an argument, grab it from a file, etc..
Once it is setup like this, just call it passing in "cmdlin" argument as false.
Edit: Using the subprocess module you should be able to use communicate to send the password over
Also, I found the pexpect library that might help in your situation
This could be possible because of your code
x = p.stdout.readline().rstrip() . stdout.readline() is a blocking call and it will block as long as there is nothing to output. Try commenting out that line and see if it works.Also sharing the content of "coreScriptCmd" would help to find the root cause in a better way.
I have the following Python script that reads numbers and outputs an error if the input is not a number.
import fileinput
import sys
for line in (txt.strip() for txt in fileinput.input()):
if not line.isdigit():
sys.stderr.write("ERROR: not a number: %s\n" % line)
If I get the input from stdin, I have to press Ctrl + D twice to end the program. Why?
I only have to press Ctrl + D once when I run the Python interpreter by itself.
bash $ python test.py
1
2
foo
4
5
<Ctrl+D>
ERROR: not a number: foo
<Ctrl+D>
bash $
In Python 3, this was due to a bug in Python's standard I/O library. The bug was fixed in Python 3.3.
In a Unix terminal, typing Ctrl+D doesn't actually close the process's stdin. But typing either Enter or Ctrl+D does cause the OS read system call to return right away. So:
>>> sys.stdin.read(100)
xyzzy (I press Enter here)
(I press Ctrl+D once)
'xyzzy\n'
>>>
sys.stdin.read(100) is delegated to sys.stdin.buffer.read, which calls the system read() in a loop until either it accumulates the full requested amount of data; or the system read() returns 0 bytes; or an error occurs. (docs) (source)
Pressing Enter after the first line caused the system read() to return 6 bytes. sys.stdin.buffer.read called read() again to try to get more input. Then I pressed Ctrl+D, causing read() to return 0 bytes. At this point, sys.stdin.buffer.read gave up and returned just the 6 bytes it had collected earlier.
Note that the process still has my terminal on stdin, and I can still type stuff.
>>> sys.stdin.read() (note I can still type stuff to python)
xyzzy (I press Enter)
(Press Ctrl+D again)
'xyzzy\n'
OK. This is the part that was busted when this question was originally asked. It works now. But prior to Python 3.3, there was a bug.
The bug was a little complicated --- basically the problem was that two separate layers were doing the same work. BufferedReader.read() was written to call self.raw.read() repeatedly until it returned 0 bytes. However, the raw method, FileIO.read(), performed a loop-until-zero-bytes of its own. So the first time you press Ctrl+D in a Python with this bug, it would cause FileIO.read() to return 6 bytes to BufferedReader.read(), which would then immediately call self.raw.read() again. The second Ctrl+D would cause that to return 0 bytes, and then BufferedReader.read() would finally exit.
This explanation is unfortunately much longer than my previous one, but it has the virtue of being correct. Bugs are like that...
Most likely this has to do with Python the following Python issues:
5505: sys.stdin.read() doesn't return after first EOF on Windows, and
1633941: for line in sys.stdin: doesn't notice EOF the first time.
I wrote an explanation about this in my answer to this question.
How to capture Control+D signal?
In short, Control-D at the terminal simply causes the terminal to flush the input. This makes the read system call return. The first time it returns with a non-zero value (if you typed something). The second time, it returns with 0, which is code for "end of file".
The first time it considers it to be input, the second time it's for keeps!
This only occurs when the input is from a tty. It is likely because of the terminal settings where characters are buffered until a newline (carriage return) is entered.
Using the "for line in file:" form of reading lines from a file, Python uses a hidden read-ahead buffer (see http://docs.python.org/2.7/library/stdtypes.html#file-objects at the file.next function). First of all, this explains why a program that writes output when each input line is read displays no output until you press CTRL-D. Secondly, in order to give the user some control over the buffering, pressing CTRL-D flushes the input buffer to the application code. Pressing CTRL-D when the input buffer is empty is treated as EOF.
Tying this together answers the original question. After entering some input, the first ctrl-D (on a line by itself) flushes the input to the application code. Now that the buffer is empty, the second ctrl-D acts as End-of-File (EOF).
file.readline() does not exhibit this behavior.