Develop Reference

Python and C pipeline: Python does not write from buffer when I flush

This is part of a university assignment but the problem I have does not relate to the assignment itself.
I have a C file that is waiting for two inputs from terminal, there are two gets(info1) and later a gets(info2). I have a buffered writed that I use to do the following inside python:
sys.stdout.buffer.write("Hello 1".encode("ascii"))
sys.stdout.flush()
sys.stdout.buffer.write("Hello 2".encode("ascii"))
sys.stdout.flush()
I then have a pipeline that sends these outputs from the python code to the input of the C program using the terminal. When I run the command I can see on the prints that both "Hello 1" and "Hello 2" is sent to gets(info1). Why is this happening? Since I flush the buffer should it not send the first input to the terminal, getting catched by the C first gets(info1) and "Hello 2" getting catched in the second gets(info2)? I even introduced a sleep function after the first flush but it sleeps then sends both the outputs to the first gets(info1). The pipeline obviously works since the C program is able to get the output from terminal produced by the python program. But why am I only gettings inputs to the first function even though I flush the buffer after the first string is written?
When I do
sys.stdout.buffer.write("Hello 1".encode("ascii"))
sys.stdout.buffer.write("\n".encode("ascii"))
sys.stdout.flush()
sys.stdout.buffer.write("Hello 2".encode("ascii"))
sys.stdout.flush()
It sends it properly. However, I need the output to be very specific

I got it working. See stdout buffering. I needed to feed stdout a newline instead of flushing, all good! :)

Why is Pycharm failing to convert string to integer?

num = 0
for i in range(5):
ask = int(input())
if abs(ask) > 0:
num = ask
print(num)
When I run the code, it lets me input the first string. But once I enter the second string, the program crashes and says "pythonProject\main.py", line 3, in
ask = int(input())
ValueError: invalid literal for int() with base 10: ''"
What's going on?
My input: I just type 1, press enter, type 2, press enter and then it crashes. I am sure it is not an error where I click enter too quickly, or I accidentally type in an empty string because I've ran the code multiple times.
What have I tried so far?
Creating a new project and pasting the code -> didn't work
Asking my friend to copy the code onto his PyCharm and run it -> worked fine on his computer
'Edit configurations', uncheck 'emulate code in output console' -> didn't work, it was already unchecked
Checked that I was running the correct file in the project -> didn't work, I was running the right file
EDIT:
FIXED, just needed to check 'Emulate code in output console' rather than uncheck it. Not sure why this works though, or how I can keep it checked for all future projects - rather than having to manually check it every time.

FIXED, just needed to check 'Emulate code in output console' rather than uncheck it. Not sure why this works though, or how I can keep it checked for all future projects - rather than having to manually check it every time.

The problem is with the input, as you are either pressing enter without any input (empty string) or you are entering a float value. This thread might help you in this case. The code is working fine when I input an integer and gives the same error when entered empty string or a float value.

To get it work you need to check "Emulate code in output console".
I've answered in the comments section and I'm glad it worked out, here is an explanation:
You need to know a concept of "terminal emulator" to understand why and how this works. When a program is ran (at least on UNIX-like operating systems), it has three I/O streams: stdin, stdout and stderr. The stdin is used to input data, and two others are for output.
Input or output stream is just a buffer used to communicate with the program back and forth. Once something is written to the buffer, it can be read from there. If the buffer is empty, an attempt to read from there will cause stall until the buffer has something in it. More about stdio: https://en.wikipedia.org/wiki/Standard_streams
When the program is ran through the terminal emulator, I/O streams are connected to this emulator, so whatever you type in the terminal window is written to your stdin by default. Whatever your program writes to the stdout and stderr is displayed on the screen. (However, this behavior may be changed using pipes, so you can pass data from some file to the stdin and also you can redirect the output to the file)
Here is the history behind terminal emulators, to understand, why is it implemented this way: https://en.wikipedia.org/wiki/Terminal_emulator#Computer_terminals
For example, you have a simple program:
s = input('Enter string: ')
print(f'stdout: {s}')
If you run it from the terminal and type "TEST":
$ python3 test.py
TEST
stdout: TEST
But you also can, for example, pass data directly to stdin, and redirect output to the file:
$ echo "ABCDEF" | python3 test.py > OUTPUT.txt
there will be no text in the terminal, but OUPUT.txt will appear. It will contain:
stdout: ABCDEF
Now, about PyCharm:
By default, when it runs your script, it does not automatically emulate terminal in the output window. It simply does not send anything to the stdin and it won't react to pressed keys. When your program gets to the line with input(), it starts to read the stdin stream until it gets \n character from the stream (indicating that user has pressed Return key). As nothing gets sent to the stream, it will wait infinitely.
Useful tip: for testing, instead of just typing something into the terminal every time, you can also check "Redirect input from:" and choose an input file.

Get STDIN before user hits enter key

I'm relatively new to socket programming and stdin / stdout and I need to know if there is a way to take input from STDIN before the user hits enter.
For example, if they type 'F' but don't hit enter, I need to see that 'F'.
I need this for a basic chatting app, and I need the user's typing to go to the next line so I can print incoming messages without breaking their text.
I've tried freezing STDOUT until the user is done typing, but this just freezes the minute the code reaches the input code block.
Current code:
### Main Handle ###
def Handler():
print("\nConnected.\n")
print(f"{server.recv(1028).decode()}\n")
def _send():
while True:
sys.stdout = buffer = io.StringIO()
text = input()
sys.stdout = old_stdout
print(buffer.getvalue())
try:
server.sendall(text.encode())
except:
pass
def _get():
while True:
print(stdin)
try:
message = server.recv(1028).decode()
if message == "":
print("\nYou got disconnected from the server, sorry bud. :C")
break
except ConnectionResetError:
print("\nYou got disconnected from the server, sorry bud. :C")
break
print(f"\n{message}")
_gthread = Thread(target = _get)
_gthread.start()
_sthread = Thread(target = _send)
_sthread.start()
while True:
pass
Any help at all is greatly appreciated. :)

The shell or the terminal emulator, or both, would ask, as part of their startup sequence, to put the terminal into "cooked" mode, which buffers input until it sees a new line. This buffering happens in the terminal driver, before it even reaches your program, which is why your program won't see individual characters in stdin.
You need to have your program put the terminal into it's raw (uncooked) mode, or a variation of raw mode that doesn't do buffering.
Python example.
Note that in the "real" raw mode, CTRL+C and friends don't work, so you probably don't want tty.setraw() but rather tty.setcbreak().

Executing a Python file from within Python, feeding it input and retrieving output

I'm writing a web application where users may attempt to solve various programming problems. The user uploads an executable .py file and the application feeds it some pre-determined input and checks whether the output is correct. (similar to how Codeforce works)
Assuming I have uploaded the user's script, how do I execute the user's script from within my own Python script, feeding the user's script input that would be captured by a normal input() function, and retrieving the output from print() functions so that I can verify its correctness?

Figured it out.
Note: If you are going to use this in a production environment, make sure you place restrictions on what the user can execute.
executor.py
import subprocess
# create process
process = subprocess.Popen(
['python3.4','script_to_execute.py'], # shell command
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE
)
# encode string into bytes, since communicate function only accepts bytes
input_data = "Captured string".encode(encoding='UTF-8')
# send the input to the process, retrieve output of print() function
stdoutdata, stderrdata = process.communicate(input=input_data)
# decode output back into string
output_data = stdoutdata.decode(encoding='UTF-8')
print(output_data)
script_to_execute.py
input_data = input()
print("Input recieved!\nData: " + input_data)
Output of executor.py
Input recieved!
Data: Captured string

Mysterious behavior with pySerial

I got the following problem:
I have a Arduino here which is connected to my Mac.
The Arduino is reading and responding to serial data.
I made a little python3-script which takes user input and uses
pySerial to write commands like 'set(13, 255)~' to the Arduino.
That is just working fine:
while 1:
cmd = input("<<< ")
cmd += '~'
if cmd != "~":
serialPort.write(cmd.encode())
output = str(serialPort.readline()).upper()[2:-5]
if output != " ":
print(">>> ", output)
Here's the Problem:
If I want to send some "automated" commands within the script, e.g. in a loop, the Arduino is not responding anymore, even the commands send are
the same, also encoded and of type "byte".
for i in range[0, 256]:
cmd = 'set(13, '
cmd += str(i)
cmd += ')~'
serialPort.write(cmd.encode())
time.sleep(0.1)
The strange thing is, if i first declare the variable (cmd) as an input, its also working fine, but that means a manual interrupt every time the command is send.
cmd = input()
cmd = 'set(13, '
cmd += str(i)
cmd += ')~'
After some hours of research i absolutely have no idea what the problem might be. As I said, the datatype is always byte after encoding.

I don't know about PySerial, but I suspect it's because the write call don't send the data immediately.
When you use the input, it will add a newline to the buffer, which may trigger the write call to flush automatically.
In the other piece of code you have, you don't add a newline to the buffer and it won't flush the data, before the buffer is big enough.
So with a flush() call it should force send it.
Hope it makes sense

Well, I just had to make a longer interrupt after connecting to the port.
(Has not been a problem since I had to give some input so far)