Develop Reference

ser.inWaiting() always returns 0 when reading a virtual port

I'm having difficulties getting pyserial to play nicely with a virtual port. I know this is an area which a few others have written about, but I couldn't find anything which solved my problem in those answers. Forgive me if I'm just being dense, and the solution exists ready-made elsewhere.
This is what I'm trying to achieve: I want to set up a virtual port, to which I can write data in one .py file, and from which I can then read data in another .py file. This is for the purposes of development and testing; I don't always have access to the device around which my current project is built.
This is my code so far:
dummy_serial.py
import os, pty, serial, time
master, slave = pty.openpty()
m_name = os.ttyname(master)
s_name = os.ttyname(slave)
# This tells us which ports "openpty" has happened to choose.
print("master: "+m_name)
print("slave: "+s_name)
ser = serial.Serial(s_name, 9600)
message = "Hello, world!"
encoded = message.encode("ascii")
while True:
ser.write(encoded)
time.sleep(1)
reader.py
import serial, time
# The port will change, depending on what port "openpty" (in the other file)
# happens to choose.
ser = serial.Serial("/dev/pts/1", 9600)
while True:
time.sleep(1)
incoming_bytes = ser.inWaiting()
# This print statement gives us an idea of what's going on.
print(incoming_bytes)
if incoming_bytes != 0:
data = ser.read(incoming_bytes)
print(data)
At present, dummy_serial.py seems to run okay. However, reader.py just keeps saying that there are no bytes waiting to be read, and hence reads no data.
What I would like:
An explanation of why ser.inWaiting() keeps returning 0, and a solution which makes ser.read(x) actually spit out "Hello, world!"
Or an explanation of why what I'm trying to do is fundamentally silly, and a better means of creating a writeable/readable virtual port.

Turn off Windows 10 console "Mark" mode from my application

I've got a console application written in Python. Windows 10 console's "Mark" mode is frustrating me to no end as users accidentally click in the application while doing something as simple as switching windows. Do I have any way to control and stop this?
For those who are unaware of Mark mode, it is when a user selects some text in the console window. When the program next writes to stdout, the entire program is paused, which is very annoying.

Automatic "Mark" mode using the mouse in windows 10, is "Quick Edit" mode from prior versions. I believe the only difference is that it is now on by default. Quick Edit Mode can be enabled/disabled from code:
import time
import win32console
ENABLE_QUICK_EDIT_MODE = 0x40
ENABLE_EXTENDED_FLAGS = 0x80
def quick_edit_mode(turn_on=None):
""" Enable/Disable windows console Quick Edit Mode """
screen_buffer = win32console.GetStdHandle(-10)
orig_mode = screen_buffer.GetConsoleMode()
is_on = (orig_mode & ENABLE_QUICK_EDIT_MODE)
if is_on != turn_on and turn_on is not None:
if turn_on:
new_mode = orig_mode | ENABLE_QUICK_EDIT_MODE
else:
new_mode = orig_mode & ~ENABLE_QUICK_EDIT_MODE
screen_buffer.SetConsoleMode(new_mode | ENABLE_EXTENDED_FLAGS)
return is_on if turn_on is None else turn_on
quick_edit_enabled = quick_edit_mode()
while True:
print('Quick edit is %s' % ('on' if quick_edit_enabled else 'off'))
time.sleep(3)
quick_edit_enabled = quick_edit_mode(not quick_edit_enabled)

I'm a little short on reputation to comment on Stephen's answer so I'm posting a separate answer.
To make this multi-platform/environment friendly add some conditional checks to skip over the code when not running on windows or when no console is attached to the process such as when running inside an IDE or when built via pyinstaller etc:
if os.name == "nt" and sys.stdout.isatty():
# stephen's code here...
This prevents errors from being raised in several cases. In my opinion you should also add a try/catch-all around the code block since os-implementations of the methods involved are unknown and are known to raise exceptions and be finicky. In the worst case scenario I would rather my code continue to run with QuickEdit enabled than fail because of it.

How to make multiple python programs communicate in this situation?

I'm a bit new at Python and I am working on a robotics project. The short form of my question is that I am trying to find the best way (for my situation) to run multiple python programs at once.
A little bit of context, my robot is a platform for a service robot that is capable of following markers and paths using image algorithms and also receive commands from a remote computer. I want to have separate programs for the image processing, the driving, and so on, and then manage all of them through a main program. I know I can't use anything basic like functions or classes, because each of these processes must be looping continuously, and I don't want to combine all the code to run in a single while loop, because it runs very slowly and it is significantly harder to manage.
So, in short, how do I make two separate, looping programs "talk"? Like I want the imaging program to send information about what it sees to the driving and steering program, etc.
I did some research and I found some information on multithreading and API's and stuff like that, though I can't really tell which one is actually the thing I'm looking for.
To clarify, I just need to be pointed in the right direction. This doesn't seem like a very high-level thing, and I know there are definitely tutorials out there, I'm just really confused as to where to start as I am teaching myself this as I go.

After some sniffing around, I found that using IPC was a good solution. The process I used wasn't too difficult, I just made some very simple server and client classes and had them communicate over the Localhost IP. There's undoubtedly a better way to do this, but for a beginner like myself, it was a simple way to make two programs talk without modifying code too much. For those who are trying to do a similar thing as I did, here's the classes I made for myself. Fair warning, they're not exactly pristine or even very complex, but they got the job done.
Here's the class I made for the server:
import socket
from random import random
from time import sleep
class ServerObject:
def __init__(self,host_address,port):
self._host_address = host_address
self._s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._s.bind((self._host_address,port))
def handshake(self):
print "Server Started. Awaiting Connection"
while True:
_data, _addr = self._s.recvfrom(1024)
if str(self._s.recvfrom(1024)[0]) == 'marco':
break
print 'marco recieved. sending polo...'
while True:
self._s.sendto('polo',_addr)
if str(self._s.recvfrom(1024)[0]) == 'confirm':
break
sleep(.5)
print 'connection verified'
self._addr = _addr
return True
def send(self,data):
self._s.sendto(str(data),self._addr)
def recieve(self,mode = 0):
_data, _addr = self._s.recvfrom(1024)
if mode == 0:
return str(_data)
if mode == 1:
return int(_data)
if mode == 2:
return float(_data)
if mode == 3:
return tuple(_data)
def change_port(self,port):
self._s.bind((self._host_address,port))
def close(self):
self._s.close()
print '_socket closed_'
if __name__ == '__main__':
host = '127.0.0.1'
talk = ServerObject(host,6003)
talk.handshake()
And here's the class I made for the client:
import socket
from time import sleep
class ClientObject:
def __init__(self,host_address,server_port,port = 0):
self._server = (host_address,server_port)
self._s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
self._s.bind((host_address,port))
def handshake(self):
print ' sending marco'
self._s.sendto('marco',self._server)
sleep(.1)
self._s.sendto('marco',self._server)
while True:
if str(self._s.recvfrom(1024)[0]) == 'polo':
break
#self._s.sendto('marco',self._server)
#self._s.sendto('marco',self._server)
print ' connection verified'
self._s.sendto('confirm',self._server)
self._s.setblocking(0)
return True
def recieve(self,mode = 0):
_data, _addr = self._s.recvfrom(1024)
if mode == 0:
return str(_data)
if mode == 1:
return int(_data)
if mode == 2:
return float(_data)
if mode == 3:
return tuple(_data)
def send(self,data):
self._s.sendto(str(data),self._server)
def close(self):
self._s.close()
print '_socket closed_'
if __name__ == '__main__':
host = '127.0.0.1'
port = 0
talk = ClientObject(host,24603,port)
talk.handshake()
#while True:
#print talk.recieve()
Use the ServerObject class on the program that will primarily send data and the ClientObject class on the program that will primarily recieve data. These can be flipped around in many situations, but I found it's best to do it this way to take advantage of UDP. The client class has an optional port variable that is set to 0 by default. This is because for UDP the client needs another port to establish itself on. 0 means it will pick an available port, but if you specify one, it's possible to re-establish a connection if the client goes offline without needing to restart both programs.
Use the handshake first on both programs being sure to use the same IP and port (not referring to the last variable on the client) and then use the send and receive functions to pass data back and forth.
again, these aren't that good, in fact there's many problems that cab arise with using this method, but for a simple task, they got the job done. I set up the handshake to print verifications of what is happening, but if those get annoying, you can just remove those lines.
Hope this helps!

I think multiproccessing library could be a solution.
You will be able to run several processes in parallel when each process could perform it specific work, while sending data to each other.
You can check this example
This is generic directory walker, which have process that scans directory tree and passes the data to other process, which scans files in already discovered folders. All this done in parallel.

This is probably a little bit outside the scope of your project, but have you considered using ROS? It lets you run a bunch of different nodes (can be Python scripts) at the same time that communicate by publishing and subscribing to topics. They can be on the same system (i.e. one or more nodes on the robot) or different systems (i.e. one node on the robot, multiple nodes on the PC). ROS also has a lot of awesome built in tools and libraries that are specifically made for robotic systems such as visualization, mapping, odometry, etc. Here's a bit of starting info:
https://en.wikipedia.org/wiki/Robot_Operating_System
http://wiki.ros.org/ROS/StartGuide
It's usually used for much larger frameworks than you seem to be describing, and beware that it takes quite a bit of time (in my experience) to implement, but it is very easy to expand once its up and running. Like I said, it all depends on the scope of your project!
Good luck!

Python Run Shell Without Enter [duplicate]

This question already has answers here:
How to read a single character from the user?
(26 answers)
Closed 6 years ago.
This is my first post to stackoverflow, so please let me know if I don't follow the correct etiquette.
I am also new to Python and would like to incorporate it into a project. I currently have a Emotiv EEG headset and would like to run a homebuilt 3-D Printer from brain signals. To do so, the EEG headset is read by EmoKey 2.0.0.20 (see attached photo), EmoKey will then send keystrokes to the Python Shell, Python interprets this and sends commands to my Arduino run printer through a COM PORT.
This may seem to be an indirect way of doing things, but it works except for one hitch. Lets pretend I think left, this is sent to EmoKey which types 'L' and the keystroke Enter into the Python Shell. In theory this would then move the printer head left. However, when EmoKey sends Enter it only creates a new line in the Shell, it doesn't actually execute. I then have to press enter by hand, which defeats the whole point.
.getch() hasn't worked because I don't think there is an actual key stroke for it to read. The link below also seems like it would be useful, but It hasn't worked thus far.
How to run a shell script without having to press enter/confirm s.th. inbetween
My question is: How can I get Python to execute what is written in the shell when only something like L or R is written? I don't think I can have it wait for a keystroke, Python would have to wait and automatically execute when it sees a specific command.
I understand that this may seem like a duplicate of the link below. However, .getch hasn't worked with EmoKey so far (maybe it's just a mistake on my part). Also, I wan't to find a way for my Python script to read what is put into the shell. Although I've started this project with just the "L" and "R" commands for simplicity and prototyping, I will move into using G-code so I can communicate with other printers or CNC equipment. That is another reason why .getch won't work in my case since it only grabs a single character (A single G-code command will be a few characters long).
Python read a single character from the user
I'm using Windows 10 and Python 2.7.11.
import serial
ser = 0
def init_serial():
COMNUM = 3 #Enter Your COM Port Number Here.
global ser #Must be declared in Each Function
ser = serial.Serial()
ser.baudrate = 9600
ser.port = COMNUM - 1 #COM Port Name Start from 0
#ser.port = '/dev/ttyUSB0'
#Specify the TimeOut in seconds, so that SerialPort
#Doesn't hangs
ser.timeout = 10
ser.open() #Opens SerialPort
# print port open or closed
if ser.isOpen():
print 'Open:' + ser.portstr
init_serial()
while 1:
temp = raw_input('Send command + Enter:\r\n')
ser.write(temp) #Writes to the SerialPort
#bytes = ser.readline() #Read from Serial Port
#print 'Response: ' + bytes #Print What is Read from Port
EmoKey Interface

I think you are looking for something like readchar:
import readchar
ch = readchar.readkey()

This definition is kind of clunky and unsatisfying, but I think it solves your problem without having to install any new packages. If I create a module defined in this link (I put it in a file called "getch.py"), we can check if the input key matches anything in a list of strings.
from getch import getch
while True:
if getch() in ['l', 'r', 'L', 'R']:
print('This was the key I was looking for!')
Technically you only need the section of the class I linked to that is relevant to your OS, but this example getch() function that I linked to is nice in that it is cross platform.

PySerial non-blocking read loop

I am reading serial data like this:
connected = False
port = 'COM4'
baud = 9600
ser = serial.Serial(port, baud, timeout=0)
while not connected:
#serin = ser.read()
connected = True
while True:
print("test")
reading = ser.readline().decode()
The problem is that it prevents anything else from executing including bottle py web framework. Adding sleep() won't help.
Changing "while True"" to "while ser.readline():" doesn't print "test", which is strange since it worked in Python 2.7. Any ideas what could be wrong?
Ideally I should be able to read serial data only when it's available. Data is being sent every 1,000 ms.

Using a separate thread is totally unnecessary. Just follow the example below for your infinite while loop instead.
I use this technique in my eRCaGuy_PyTerm serial terminal program here (search the code for inWaiting() or in_waiting).
Notes:
To check your python3 version, run this:
python3 --version
My output when I first wrote and tested this answer was Python 3.2.3.
To check your pyserial library (serial module) version, run this--I first learned this here:
python3 -c 'import serial; \
print("serial.__version__ = {}".format(serial.__version__))'
This simply imports the serial module and prints its serial.__version__ attribute.
My output as of Oct. 2022 is: serial.__version__ = 3.5.
If your pyserial version is 3.0 or later, use property in_waiting in the code below. If your pyserial version is < 3.0, use function inWaiting() in the code below. See the official pyserial documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
Non-blocking, single-threaded serial read example
import serial
import time # Optional (required if using time.sleep() below)
ser = serial.Serial(port='COM4', baudrate=9600)
while (True):
# Check if incoming bytes are waiting to be read from the serial input
# buffer.
# NB: for PySerial v3.0 or later, use property `in_waiting` instead of
# function `inWaiting()` below!
if (ser.inWaiting() > 0):
# read the bytes and convert from binary array to ASCII
data_str = ser.read(ser.inWaiting()).decode('ascii')
# print the incoming string without putting a new-line
# ('\n') automatically after every print()
print(data_str, end='')
# Put the rest of your code you want here
# Optional, but recommended: sleep 10 ms (0.01 sec) once per loop to let
# other threads on your PC run during this time.
time.sleep(0.01)
This way you only read and print if something is there. You said, "Ideally I should be able to read serial data only when it's available." This is exactly what the code above does. If nothing is available to read, it skips on to the rest of your code in the while loop. Totally non-blocking.
(This answer originally posted & debugged here: Python 3 non-blocking read with pySerial (Cannot get pySerial's "in_waiting" property to work))
pySerial documentation: http://pyserial.readthedocs.io/en/latest/pyserial_api.html
UPDATE:
27 Dec. 2018: added comment about in_waiting vs inWaiting(). Thanks to #FurkanTürkal for pointing that out in the comments below. See documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
27 Oct. 2018: Add sleep to let other threads run.
Documentation: https://docs.python.org/3/library/time.html#time.sleep
Thanks to #RufusV2 for bringing this point up in the comments.
Note on multi-threading:
Even though reading serial data, as shown above, does not require using multiple threads, reading keyboard input in a non-blocking manner does. Therefore, to accomplish non-blocking keyboard input reading, I've written this answer: How to read keyboard input?.
References:
Official pySerial serial.Serial() class API - https://pyserial.readthedocs.io/en/latest/pyserial_api.html

Put it in a separate thread, for example:
import threading
import serial
connected = False
port = 'COM4'
baud = 9600
serial_port = serial.Serial(port, baud, timeout=0)
def handle_data(data):
print(data)
def read_from_port(ser):
while not connected:
#serin = ser.read()
connected = True
while True:
print("test")
reading = ser.readline().decode()
handle_data(reading)
thread = threading.Thread(target=read_from_port, args=(serial_port,))
thread.start()
http://docs.python.org/3/library/threading

I would warn against using blocking IO in a thread. Remember Python has a GIL and at one time only one thread can execute. Now please note that pyserial module is a wrapper over an OS implementation of accessing the serial port. That means it calls code external to the Python. If that code blocks, then the interpreter also get blocked and nothing will execute in the Python program, even the main thread.
This can even happen when using non-blocking IO or timeout based polling if the underlying device driver does not implement timeout well.
A more robust approach is to use multiprocessing module with a queue. Run serial read code in a separate process. This will make sure main and other threads don't block and the program can exit in clean way.

Use a timer driven event to test and read the serial port.
Untested example:
import threading
class serialreading():
def __init__(self):
self.active = True
self.test()
def test(self):
n_in =comport.in_waiting()
if n_in> 0:
self.data = self.data + comport.read(size=n_in)
if len(self.data) > 0:
print(self.data)
self.data=""
if self.active:
threading.Timer(1, test).start() # start new timer of 1 second
def stop(self):
self.active = False