Develop Reference

ser.inWaiting() always returns 0 when reading a virtual port

I'm having difficulties getting pyserial to play nicely with a virtual port. I know this is an area which a few others have written about, but I couldn't find anything which solved my problem in those answers. Forgive me if I'm just being dense, and the solution exists ready-made elsewhere.
This is what I'm trying to achieve: I want to set up a virtual port, to which I can write data in one .py file, and from which I can then read data in another .py file. This is for the purposes of development and testing; I don't always have access to the device around which my current project is built.
This is my code so far:
dummy_serial.py
import os, pty, serial, time
master, slave = pty.openpty()
m_name = os.ttyname(master)
s_name = os.ttyname(slave)
# This tells us which ports "openpty" has happened to choose.
print("master: "+m_name)
print("slave: "+s_name)
ser = serial.Serial(s_name, 9600)
message = "Hello, world!"
encoded = message.encode("ascii")
while True:
ser.write(encoded)
time.sleep(1)
reader.py
import serial, time
# The port will change, depending on what port "openpty" (in the other file)
# happens to choose.
ser = serial.Serial("/dev/pts/1", 9600)
while True:
time.sleep(1)
incoming_bytes = ser.inWaiting()
# This print statement gives us an idea of what's going on.
print(incoming_bytes)
if incoming_bytes != 0:
data = ser.read(incoming_bytes)
print(data)
At present, dummy_serial.py seems to run okay. However, reader.py just keeps saying that there are no bytes waiting to be read, and hence reads no data.
What I would like:
An explanation of why ser.inWaiting() keeps returning 0, and a solution which makes ser.read(x) actually spit out "Hello, world!"
Or an explanation of why what I'm trying to do is fundamentally silly, and a better means of creating a writeable/readable virtual port.

Does Fusion 360 not have access to USB ports by default?

I'm working on making my own 3D mouse, sort of like 3D Connexion's own model. I'm using Autodesk's Fusion 360 as my application, and Fusion's API(Application Program Interface) to interact with the mouse. I have a gyroscope + accelerometer that spits raw measurements into an Arduino, which then sends those values(through the serial port using the Serial library). I'm then using the PySerial library(because Fusion's API uses python for it's language) to read that data coming from the Arduino, and spit it into Fusion, where I can control the camera position. The issue that I'm running into is that, for testing, I can print the values of the gyro + accelerometer to the Command Prompt just fine with a Python script. But when I try the exact same script in Fusion's code editor(Spyder), it doesn't work. I know that the Serial library is being imported, but the script doesn't work. Is it possible that Fusion 360 doesn't have access to the USB ports? If so, how could I fix it?
Here is a little snipet of my code:
#This is a python script that when run in the CMD, it works just fine.
import serial
ser = serial.Serial()
ser.baudrate = 9600
ser.port = 'COM3'
ser.open()
while True:
result = ser.readline()
result = str(result)
print (result)
All that this code does, is it reads the value of a potentiometer hooked up to an Arduino. It then prints those values into the CMD.
Here is the code that comes from the Fusion 360 API:
#This is the Fusion 360 code that doesn't work.
import adsk.core, adsk.fusion, adsk.cam, traceback
import serial
def get(app):
try:
ui = app.UserInterface
ser = serial.win32.Serial()
ser.baudrate = 9600
ser.port = 'COM3'
ser.open()
value = ser.read()
value = str(value)
adsk.doEvents()
ui.messageBox(value)
except:
ui.messageBox('Failed:\n{}'.format(traceback.format_exc()))
def Main():
try:
app = adsk.core.Application.Get()
ui = app.UserInterface
ui.messageBox("Everything is working till this point")
adsk.doEvents()
get(app)
except:
ui.messageBox('Failed:\n{}'.format(traceback.format_exc()))
Main()
exit()
Thanks in advance!

In answer to your question, it may be so that Fusion360 does not have access to serial ports directly.
Fusion360 has a JavaScript API available and had to add custom calls to allow access to local machine resources like files.
(JavaScript does not have that natively).
That may be the nature of the problem you are up against, even though your program is written in Python. (I think F360 uses its JavaScript API to display its own User Interface).
Also, I see a difference between the code snippets:
ser = serial.win32.Serial()
...
ser = serial.Serial()
Perhaps you can call the first program from F360, and simply write to a file or something like that, to test it.

Python reading and writing to tty

BACKGROUND: If you want, skip to the problem section
I am working on a front end for test equipment. The purpose of the front end is to make it easier to write long test scripts. Pretty much just make them more human readable and writable.
The equipment will be tested using a Prologix GPIB-USB Controller (see prologix.biz). We found a tutorial at http://heliosoph.mit-links.info/gpib-on-debian-linux-the-easy-way/ and did all of the steps, and it worked!
As we don't have the test equipment yet, we wanted to write an emulator in Python using openpty. We do have the GPIB-USB Controller, just not what gets connected to that. I got the emulator working as a perfect replacement for the GPIB-USB. This means that I would follow the "GPIB on Debian ..." tutorial (above) and get output that I programmed the emulator to return. The input and output were done in the same manner as the tutorial just reading and writing to/from a pty device (ie /dev/pts/2) instead of the tty (ie /dev/ttyUSB0).
Now that the emulator works, we want to write a front end that can be used to write scripts easily. The goal is to make a kind of macro system that writes a bunch of commands when we call a function.
PROBLEM: exists using both the emulator and the device
I am using the following Python functions to read, write, and open the tty/pty devices, but I am not getting the same result that I get if I just use echo and cat in bash.
tty = os.open(tty_path, os.O_RDWR)
os.read(tty, 100)
os.write(tty, "++ver")
for example, I would expect the following to be equivalent
$ cat < /dev/pty/2 & # According to the tutorial, this must be run in parallel
$ echo "++ver" > /dev/pty/2
Prologix GPIB Version 1.2.3.4 ...
and
tty = os.open("/dev/pyt/2", os.o_RDWR)
os.read(tty, 100) # In separate Thread to be run in parallel
os.write(tty, "++ver") # in main thread
The output is very different, please explain why and how I can fix it.
FULL CODE is here: http://pastebin.com/PWVsMjD7

Well, I asked too soon. I hope someone benefits from this self answer.
So this works to read and write from both the emulator and the actual device. I am not exactly sure why, and would appreciate an explanation, but this does work in all of my tests
import serial
class VISA:
def __init__(self, tty_name):
self.ser = serial.Serial()
self.ser.port = tty_name
# If it breaks try the below
#self.serConf() # Uncomment lines here till it works
self.ser.open()
self.ser.flushInput()
self.ser.flushOutput()
self.addr = None
self.setAddress(0)
def cmd(self, cmd_str):
self.ser.write(cmd_str + "\n")
sleep(0.5)
return self.ser.readline()
def serConf(self):
self.ser.baudrate = 9600
self.ser.bytesize = serial.EIGHTBITS
self.ser.parity = serial.PARITY_NONE
self.ser.stopbits = serial.STOPBITS_ONE
self.ser.timeout = 0 # Non-Block reading
self.ser.xonxoff = False # Disable Software Flow Control
self.ser.rtscts = False # Disable (RTS/CTS) flow Control
self.ser.dsrdtr = False # Disable (DSR/DTR) flow Control
self.ser.writeTimeout = 2
def close(self):
self.ser.close()

You do not actually have to use any special module to read from TTY.
Option O_NOCTTY solved my problems with CDCACM example MCU app.
I'm sure it will work for you (as you work on Linux too).
#!/usr/bin/env python3
import io, os
tty = io.TextIOWrapper(
io.FileIO(
os.open(
"/dev/ttyACM1",
os.O_NOCTTY | os.O_RDWR),
"r+"))
for line in iter(tty.readline, None):
print(line.strip())

Stumbled on this while looking into pty/tty usage in python.
I think the original code did not work because echo will add a newline and the python os.write will not.
This is shown in your self answer here self.ser.write(cmd_str + "\n")
So the original code may have worked if it were os.write(tty, "++ver\n")

PySerial non-blocking read loop

I am reading serial data like this:
connected = False
port = 'COM4'
baud = 9600
ser = serial.Serial(port, baud, timeout=0)
while not connected:
#serin = ser.read()
connected = True
while True:
print("test")
reading = ser.readline().decode()
The problem is that it prevents anything else from executing including bottle py web framework. Adding sleep() won't help.
Changing "while True"" to "while ser.readline():" doesn't print "test", which is strange since it worked in Python 2.7. Any ideas what could be wrong?
Ideally I should be able to read serial data only when it's available. Data is being sent every 1,000 ms.

Using a separate thread is totally unnecessary. Just follow the example below for your infinite while loop instead.
I use this technique in my eRCaGuy_PyTerm serial terminal program here (search the code for inWaiting() or in_waiting).
Notes:
To check your python3 version, run this:
python3 --version
My output when I first wrote and tested this answer was Python 3.2.3.
To check your pyserial library (serial module) version, run this--I first learned this here:
python3 -c 'import serial; \
print("serial.__version__ = {}".format(serial.__version__))'
This simply imports the serial module and prints its serial.__version__ attribute.
My output as of Oct. 2022 is: serial.__version__ = 3.5.
If your pyserial version is 3.0 or later, use property in_waiting in the code below. If your pyserial version is < 3.0, use function inWaiting() in the code below. See the official pyserial documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
Non-blocking, single-threaded serial read example
import serial
import time # Optional (required if using time.sleep() below)
ser = serial.Serial(port='COM4', baudrate=9600)
while (True):
# Check if incoming bytes are waiting to be read from the serial input
# buffer.
# NB: for PySerial v3.0 or later, use property `in_waiting` instead of
# function `inWaiting()` below!
if (ser.inWaiting() > 0):
# read the bytes and convert from binary array to ASCII
data_str = ser.read(ser.inWaiting()).decode('ascii')
# print the incoming string without putting a new-line
# ('\n') automatically after every print()
print(data_str, end='')
# Put the rest of your code you want here
# Optional, but recommended: sleep 10 ms (0.01 sec) once per loop to let
# other threads on your PC run during this time.
time.sleep(0.01)
This way you only read and print if something is there. You said, "Ideally I should be able to read serial data only when it's available." This is exactly what the code above does. If nothing is available to read, it skips on to the rest of your code in the while loop. Totally non-blocking.
(This answer originally posted & debugged here: Python 3 non-blocking read with pySerial (Cannot get pySerial's "in_waiting" property to work))
pySerial documentation: http://pyserial.readthedocs.io/en/latest/pyserial_api.html
UPDATE:
27 Dec. 2018: added comment about in_waiting vs inWaiting(). Thanks to #FurkanTürkal for pointing that out in the comments below. See documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
27 Oct. 2018: Add sleep to let other threads run.
Documentation: https://docs.python.org/3/library/time.html#time.sleep
Thanks to #RufusV2 for bringing this point up in the comments.
Note on multi-threading:
Even though reading serial data, as shown above, does not require using multiple threads, reading keyboard input in a non-blocking manner does. Therefore, to accomplish non-blocking keyboard input reading, I've written this answer: How to read keyboard input?.
References:
Official pySerial serial.Serial() class API - https://pyserial.readthedocs.io/en/latest/pyserial_api.html

Put it in a separate thread, for example:
import threading
import serial
connected = False
port = 'COM4'
baud = 9600
serial_port = serial.Serial(port, baud, timeout=0)
def handle_data(data):
print(data)
def read_from_port(ser):
while not connected:
#serin = ser.read()
connected = True
while True:
print("test")
reading = ser.readline().decode()
handle_data(reading)
thread = threading.Thread(target=read_from_port, args=(serial_port,))
thread.start()
http://docs.python.org/3/library/threading

I would warn against using blocking IO in a thread. Remember Python has a GIL and at one time only one thread can execute. Now please note that pyserial module is a wrapper over an OS implementation of accessing the serial port. That means it calls code external to the Python. If that code blocks, then the interpreter also get blocked and nothing will execute in the Python program, even the main thread.
This can even happen when using non-blocking IO or timeout based polling if the underlying device driver does not implement timeout well.
A more robust approach is to use multiprocessing module with a queue. Run serial read code in a separate process. This will make sure main and other threads don't block and the program can exit in clean way.

Use a timer driven event to test and read the serial port.
Untested example:
import threading
class serialreading():
def __init__(self):
self.active = True
self.test()
def test(self):
n_in =comport.in_waiting()
if n_in> 0:
self.data = self.data + comport.read(size=n_in)
if len(self.data) > 0:
print(self.data)
self.data=""
if self.active:
threading.Timer(1, test).start() # start new timer of 1 second
def stop(self):
self.active = False

Using pySerial with Python 3.3

I've seen many code samples using the serial port and people say they are working codes too. The thing is, when I try the code it doesn't work.
import serial
ser = serial.Serial(
port=0,
baudrate=9600
# parity=serial.PARITY_ODD,
# stopbits=serial.STOPBITS_TWO,
# bytesize=serial.SEVENBITS
)
ser.open()
ser.isOpen()
print(ser.write(0xAA))
The error it gives me is : "SerialException: Port is already opened".
Is it me using python3.3 the problem or is there something additional I need to instal ? Is there any other way to use COM ports with Python3.3 ?

So the moral of the story is.. the port is opened when initialized. ser.open() fails because the serial port is already opened by the ser = serial.Serial(.....). And that is one thing.
The other problem up there is ser.write(0xAA) - I expected this to mean "send one byte 0xAA", what it actually did was send 170(0xAA) zeros. In function write, I saw the following :
data = bytes(data) where data is the argument you pass. it seems the function bytes() doesn't take strings as arguments so one cannot send strings directly with: serial.write(), but ser.write(bytearray(TheString,'ascii')) does the job.
Although I am considering adding:
if(type(data) == type('String')):
data = bytearray(data,'ascii')
in ser.write(), although that would make my code not work on other PCs.