Develop Reference

Python - serial does not read correct avlue

I am trying to create a communication between an STM32 and a laptop.
I am trying to receive data from the serial, sent thanks to an STM32. Actual code that I am sending is 0x08 0x09 0x0A 0x0B
I checked on the oscilloscope and I am indeed sending the correct values in the correct order.
What I receive is actually :
b'\n\x0b\x08\t'
I assume that Python is not reading an input that is greater than a 3 bit size, but can not figure out why
Please find my code below :
import serial
ser = serial.Serial('COM3', 115200, bytesize=8)
while 1 :
if(ser.inWaiting() != 0) :
print(ser.read(4))
If someone could help, it would be nice ! :)

check your uart rate, keep the python serial rate the same for stm32

What comes to my mind when looking at pySerial library is that while You initialize Your COM port:
You are not providing read timeout parameter.
You are awaiting 4 bytes of data from serial port.
The problem with such approach is that Your code will wait forever until it gets these 4 bytes and if You are sending the same array from STM32 for me it looks like You received 0x0A,0x0B from older packet and 0x08,0x09 from newer packet, so python code printed these 4 bytes and in fact received into buffer also 0x0A,0x0B of newer packet but it waits until it will receive 2 more bytes before it will be allowed to return with data to print.
Putting here a timeout on read and limiting read argument to single byte might solve Your problem.
Also for further development if You would like to create regular communication between microcontroller and computer I would suggest to move these received bytes into separate buffer and recognize single packets in separate thread parser. In Python it will be painful to create more complex serial communication as even with separate thread it will be quite slow.

Best approach to parse buffer string in python

I'm working on an embedded system that sends commands via Uart.
Uart works at 115200 baud
On PC side I want to read these commands, parse them and execute the related action.
I choose python as language to build a script.
This is a typical command received from the embedded system:
S;SEND;40;{"ID":"asg01","T":1,"P":{"T":180}};E
Each message starts with S and ends with E.
The command associated to the message is "SEND" and the payload length is 40.
My idea is read the bytes coming from the UART and:
check if the message starts with S
check if the message ends with E
if the above assumptions are true, split the message in order to find the command and the payload.
Which is the best way to parse the all bytes coming from an asynchronous uart?
My concern regards the lost of message due to wrong (or slow) parsing.
Thanks for the help!
BR,
Federico

In my day job, I wrote the software for an embedded system and a PC communicating with each other by a USB cable, using the UART protocol at 115,200 baud.
I see that you tagged your post with PySerial, so you already know about Python's most popular package for serial port communication. I will add that if you are using PyQt, there's a serial module included in that package as well.
115,200 baud is not fast for a modern desktop PC. I doubt that any parsing you do on the PC side will fail to keep up. I parse data streams and plot graphs of my data in real time using PyQt.
What I have noticed in my work with communication between an embedded system and a PC over a UART is that some data gets corrupted occasionally. A byte can be garbled, repeated, or dropped. Also, even if no bytes are added or dropped, you can occasionally perform a read while only part of a packet is in the buffer, and the read will terminate early. If you use a fixed read length of 40 bytes and trust that each read will always line up exactly with a data packet as you show above, you will frequently be wrong.
To solve these kinds of problems, I wrote a FIFO class in Python which consumes serial port data at the head of the FIFO, yields valid data packets at the tail, and discards invalid data. My FIFO holds 3 times as many bytes as my data packets, so if I am looking for packet boundaries using specific sequences, I have plenty of signposts.
A few more recommendations: work in Python 3 if you have the choice, it's cleaner. Use bytes and bytearray objects. Don't use str, because you will find yourself converting back and forth between Unicode and ASCII.

This format is almost parseable as a csv, but not quite, because the fourth field is JSON, and you may not be able to guarantee that the JSON doesn't contain any strings with embedded semicolons. So, I think you probably want to just use string (or, rather, bytes) manipulation functions:
def parsemsg(buf):
s, cmd, length, rest = buf.split(b';', 3)
j, _, e = rest.rpartition(b';')
if s != b'S' or e != b'E':
raise ValueError('must start with S and end with E')
return cmd.decode('utf-8'), int(length), json.loads(j)
Then:
>>> parsemsg(b'S,SEND,40,{"ID":"asg01","T":1,"P":{"T":180}},E')
('SEND', 40, {'ID': 'asg01', 'T': 1, 'P': {'T': 180}})
The actual semicolon-parsing part takes 602ns on my laptop, The decode and int raise that to 902ns. The json.loads, on the other hand, takes 10us. So, if you're worried about performance, the JSON part is really the only part that matters (trying third-party JSON libs I happen to have installed, the fastest one is still 8.1us, which isn't much better). You might as well keep everything else simple and robust.
Also, considering that you're reading this at 115000 baud, you can't get these messages any faster than about 6ms, so spending 11us parsing them is not even close to a problem in the first place.

sending multi-digit integers from python to arduino through pyserial

I want to send a number(say 90) from python to arduino through pyserial. But I guess arduino works with ASCII only. So how do I read the integer 90 in arduino ? please help...

In Python it is easy to convert a number as a string: You can just do
str(90)
https://docs.python.org/3/library/functions.html#func-str
Then in the Arduino code, you can use
parseInt()
https://www.arduino.cc/en/Serial/ParseInt
Or you can just send binary data over the serial line. If your number is always between 0 and 255, you can just send one byte. Otherwise you will have to send the number in several bytes.

Pyserial Sending More than One Byte

First time poster.
Before I start, I just want to say that I am a beginner programmer so bear with me, but I can still follow along quite well.
I have a wireless device called a Pololu Wixel, which can send and receive data wirelessly. I'm using two of them. One to send and one to receive. It's USB so it can plug straight into my Raspberry Pi or PC, so all I have to do is connect to the COM port through a terminal to read and write data to it. It comes with a testing terminal program that allows me to send 1-16 bytes of info. I've done this and I've sent and received 2 bytes (which is what I need) with no problem.
Now here's my problem: When I open up the Ubuntu terminal and use Pyserial to connect to the correct sending Wixel COM Port and write a value larger than 255, my receiving COM port, which is connected to another instance of Terminal also using Pyserial, doesn't read the right value, hence I think I'm not being able to read and write two bytes, but only one. After doing something reading online in the pyserial documentation, I believe, not know, that Pyserial can only read and write 5,6,7, or 8 bits at a time.
I hope my problem is obvious now. How the heck can I write 2 bytes worth of info to the COM port to my device and send it to the other device which needs to read those 2 bytes through, all using pyserial?
I hope this all makes sense, and I would greatly appreciate any help.
Thanks
UPDATE
Okay, I think I've got this going now. So I did:
import serial
s=serial.Serial(3) //device #1 at COM Port 4 (sending)
r=serial.Serial(4) //device #4 at COM Port 5 (receiving)
s.timeout=1
r.timeout=1
s.write('0x80')
r.readline()
//Output said: '0x80'
s.write('hh')
r.readline()
//Output said: 'hh'
Honestly, I think this solves my problem. Maybe there never was a problem to begin with. Maybe I can take my 16bit binary data from the program, example "1101101011010101", turn it into characters (I've seen something called char() before I think that's it)
then use s.write('WHATEVER')
then use r.readline() and convert back to binary

You'll likely need to pull your number apart into multiple bytes, and send the pieces in little endian or big endian order.
EG:
low_byte = number % 256
high_byte = number // 256
That should get you up to 65535. You can reconstruct the number on the other side with high_byte * 256 + low_byte.

Sending hex data over RS-232 between an Arduino and a Mac OS X system

I'm using Voice Recognition Module -- Arduino Compatible. It says there to use AccessPort, but since I'm on Mac OS X it won't work. I don't know of any other solution but to write my own in Python while I'm open to any other language that works.
I found some solutions over here about pySerial but as of now it doesn't seem to work.
import serial
import struct
def C(val):
return struct.pack('H', val)
ser = serial.Serial(port='/dev/tty.PL2303-00001004', baudrate=9600, timeout=1)
ser.nonblocking()
ser.write(C(0xaa11))
print ser.read().__repr__()
ser.close()
It opens, write two bytes and doesn't read anything. 0xaa11 is used to start recording. My device seems to be in waiting mode since the red LEDs flash rapidly. Even if I do 0xaa00, it doesn't do much either. I even tried to change the byte order to 0x00aa, 0x11aa and so on, but I never receive my 0xcc or 0xe0 error code.
In the documentation it says to send it as hex, so my guess is that accessport converts hex to a Short which use two bytes. 0xaa00 is a two bytes number. I even tried to change the baud rate to 38400 in case someone before me did set that to this speed.
I'd be happy to read anything from that serial port because as of now I can write, it does nothing and reads nothing...
As of now, I'll try to install AccessPort with crossover. If it still doesn't work, I'll have to find a Windows box. I could even write my own bridge with my Arduino board since the driver I'm using might be buggy.
In case it matters, I'm using a USB to Serial PL2303.
Edit
I plugged my output and input pin to test my USB-to-serial. I do read what I write so the driver is working fine.
I'll try to set it up directly with Arduino and test directly from there.
Sending 0xaa11 is different than sending 0xaa and then 0x11. The serial seems to reverse the byte order.
On the other hand, the documentation doesn't tell about bit endianess, so it might be the problem too.

I kind of solved my problem while it was pretty dumb after all.
Here is my final script:
import serial
import struct
from time import sleep
def C(val):
return struct.pack('!H', val)
ser = serial.Serial(port='/dev/tty.PL2303-00002006', baudrate=9600)
sleep(2)
ser.write(C(0xaa00))
sleep(1)
ser.write(C(0xaa36))
sleep(1)
ser.write(C(0xaa11))
sleep(1)
while True:
print ser.readline()
If someone ever want to use that module, it works. It will switch to a mode where it puts to output words instead of bytes so you can read it using readline...
I used in struct.pack "!H" to use network endianess it change 0xaa00 to 0xaa 0x00 like it should be.
But the real problem here wasn't in the script. Since the usb to serial did work I thought that it was strange that the module didn't respond anything... not even a byte for errors. Every command in this module must return something. That's why I looked back at my cable and checked the pins... gnd to gnd, power supply to power supply and then rxd to txd, txd to rxd... While everything seems ok... I should receive something... And then I really thought... what if it's not supposed to use a cross cable and txd should be plugged to txd and rxd to rxd... Apparently it does.
I switch the cable and now data is sent and received.
So anyone using that module should check that it's correctly plugged in. If it doesn't work, switch the data pins.