Develop Reference

pyserial not replenish to input buffer immediately after received data?

I want to use pyserial module to read serial data sent by Arduino, the data update rate is about 250Hz, one data consists of 29 bytes and the baud rate is 230400.
When my code is running, I found that the pyserial input buffer does not automatically replenish data from arduino until about 0.5 seconds has passed. Is there any way to replenish the input buffer immediately after data received from Arduino?
Many thanks!
Adam Shiau
import serial
import time
ser = serial.Serial()
old = 0
while True
if ser.in_waiting > 29:
print('buf: ', ser.readInputBuffer())
new = time.perf_counter_ns()
print(ser.readBinaryList(29))
print(''%.1f\n %((new - old)*1e-3))
old = new
# updated function definition_07/23
def readBinaryList(mum):
data_r = ser.read(mum)
data_r = [i for i in data_r]
return data_r
def readInputBuffer(self):
return ser.in_waiting
result1
result2

ser.readInputBuffer() and ser.readBinaryList() look weird. I cannot find those functions in the docs. If I was writing this code, I would just use ser.read(29)
import serial
import time
ser = serial.Serial()
old = 0
while True
if ser.in_waiting > 29:
new = time.perf_counter_ns()
print(ser.read(29))
print(''%.1f\n %((new - old)*1e-3))
old = new

Does this show the signal strength/quality from GSM modems? why does it print 8?

Im trying to read the signal strength/quality from gsm modems, so i used this AT+CSQ command.
from time import sleep
import serial
from curses import ascii
ser = serial.Serial()
ser.port = "COM10"
ser.baudrate = 9600
ser.open()
print(ser.write('AT+CSQ=?'.encode("ascii")))

ser.write returns the number of bytes written (on the port).
(https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.write)
You need to call ser.read afterward to read the answear.
Something like :
from time import sleep
import serial
ser = serial.Serial()
ser.port = "COM10"
ser.baudrate = 9600
ser.open()
try:
res_write = ser.write('AT+CSQ=?'.encode("ascii"))
sleep(0.01)
res_read = b""
while ser.inWaiting() > 0:
res_read += ser.read(1)
finally : # ensure you close the use of the port in case of crash
ser.close()
print(res_read.decode("ascii"))

Reading bytes from Arduino using Python

I am trying to read serial input from Arduino in Python. I am trying to do a simple task - my Arduino has one button writing to serial its state. This is the code for the Arduino:
const int ledPin = 7;
const int buttonPin = 2;
int buttonState= 0;
void setup() {
Serial.begin(9600);
pinMode(ledPin, OUTPUT);
pinMode(buttonPin, INPUT_PULLUP);
}
void loop() {
buttonState = digitalRead(buttonPin);
if(buttonState == LOW){ digitalWrite(ledPin, HIGH);} else digitalWrite(ledPin, LOW);
Serial.println(buttonState);
delay(50);
}
Using serial monitor in Arduino 1.8.13 software I can see it prints either 0 or 1.
I'm trying to read it with the following code in Python:
import serial
ser = serial.Serial('COM10', baudrate = 9600, timeout = 1)
t = 0
while t<10:
arduinoData = ser.readline().decode()
print(arduinoData)
t+=1
print(type(arduinoData))
ser.close()
I read the first 10 inputs and print the type of the arduinoDatawhich in this case is string, since I'm using .decode(). However, I can't use boolean operators such as arduinoData=='1' to do anything, even though it is decoded to string. I have tried using .decode('ascii'), .decode('UTF-8') to no avail.
How can I check the values or how can I scrap the .decode() and convert the bytes to simple integers to then use boolean operators?
I am using python 3.9 with an Arduino Uno.
UPDATE 1
I've managed to convert it from str to int this way in Python:
import serial
ser = serial.Serial('COM10', baudrate = 9600, timeout = 1)
t=0
while t<10:
arduinoData = ser.readline().decode()
print(arduinoData)
t+=1
if(int(arduinoData) == 1):
print("Input is 1")
ser.close()
This way I can see that indeed my serial input is equal to 1. However I wish to do something if it is not equal to 1. So I put the checking in the loop like so:
import serial
ser = serial.Serial('COM10', baudrate = 9600, timeout = 1)
t=0
while t<10:
arduinoData = ser.readline().decode()
print(arduinoData)
if(int(arduinoData) == 1):
print("Input is 1")
t+=1
ser.close()
And get the following error:
Traceback (most recent call last):
File "C:\Users\01132892\Desktop\Serial\SerialButtonPurple.py", line 12, in <module>
if(int(arduinoData) == 1):
ValueError: invalid literal for int() with base 10: ''
I am assuming this is a problem with converting it inside the loop, however I have no idea how to solve it.

Okay, I've managed to solve the problem.
Please note that to check whether it is working in the main loop, I am pressing key 'a' if the button is not pressed (signal read is 0) and releasing it if the signal is 1.
import serial
import pynput
ser = serial.Serial('COM10', baudrate = 9600, timeout = 1)
from pynput.keyboard import Key, Controller
keyboard = Controller()
while 1:
arduinoData = ser.readline().rstrip()
print(arduinoData)
if(arduinoData==b'0'):
keyboard.press('a')
else:
keyboard.release('a')
ser.close()
I would like to thank ocrdu for suggesting to use the rstrip() to get rid of EOL.

My Python isn't all that good, but maybe ser.readline() also returns the EOL? If so, arduinoData doesn't contain just the "1".
If I'm right, it should work if you strip off the EOL.
Alternatively: have the Arduino send single bytes with Serial.print(buttonState); and read single bytes in Python with ser.read(). There is no need for a delimiter when you send and read single bytes.

Trying to send serial data over bluetooth from arduino sensors to python

***Python code:***
import serial
import pandas as pd
import time
import re
import xlrd
from msvcrt import getch
import numpy as np
i = 0
x = 0
y = 0
df = pd.read_excel(r'C:\Users\lynchfamily\Desktop\mlglovesdata.xls')
# Read COM9
# Read from COM10 as well
# Readline() only works with a timeout (IMPORTANT)
serHC = serial.Serial('COM9', 115200,timeout=.250,parity=serial.PARITY_NONE,rtscts=1) # This is the JY
serRN = serial.Serial('COM10', 115200,timeout=.250,parity=serial.PARITY_NONE,rtscts=1) # This is the silvermate
def serialin():
# Sensor lists
sensor_names = list()
sensor_values = list()
global i
# Read a certain amount of bytes from serial and then continue
# Regular expressions for finding the proper data
while i < 6:
# print(i) for debugging
global serHC
global serRN
#searchObj = re.search(r'(A\d?\d*)?(\d*)?',serHC.read(4).decode(),re.I)
#searchObjRN = re.search(r'(A\d?\d*)?(\d*)?',serRN.read(4).decode(),re.I)
# Serial data stops while in loop
# The if statements keep the false values out of the program
#if searchObj.group(1):
sensor_names.append(serHC.read(2))
#if searchObj.group(2):
sensor_values.append(serHC.read(2))
#if searchObjRN.group(1):
sensor_names.append(serRN.read(2))
#if searchObjRN.group(2):
sensor_values.append(serRN.read(2))
i = i + 1
while 1:
# Get the key from the msvcrt module
key = getch().decode('ASCII')
# If key is pressed, do something
if key:
print(key)
# Zip them together
# Final 2D list
final_2d_list = zip(sensor_names,sensor_values)
print(list(sorted(final_2d_list)))
#vals = df.Dataframe([
#df.append(vals)
#print(sorted_array_1stdim[r])
#sensor_values = [0] * 10
# Thread for reading definition
break
# Fancy recursion
sensor_values.clear()
sensor_names.clear()
i = 0
serialin()
serialin()
Arduino Code:
// The device with green colored wires
void setup() {
Serial.begin(115200);
}
void loop() {
// It won't work with the I2C while loop for some reason. Perhaps it is getting stuck up on it
Serial.print("A4");
Serial.print(analogRead(0)); // Read the local analog signal
delay(5);
Serial.print("A5");
Serial.print(analogRead(1)); // Read the local analog signal
delay(5);
Serial.print("A6");
Serial.print(analogRead(2)); // Read the local analog signal
delay(5);
Serial.print("A7");
Serial.print(analogRead(3)); // Read the local analog signal
}
I'm trying to send analog data from sensors over through bluetooth silver mate from sparkfun, and HC-06 modules to python.
I have to read the analog data at a delay of 5 seconds between each, so that the readings aren't conflicted.
The data comes through serial ports COM9 and COM10. I know that serial in python can be blocking, that's why I attempted to read it first, and then put it in a list.
I also knows that once serial has been read through, it appears to be non-blocking. When I was using serHC.readline() and serRN.readline(), I was getting something like what I'd expect to see.
However, the data in the list were not updating according to the change in the sensors. I have to admit python is not my main programming language, so that is why I'm asking for help.
I thought maybe using multiple threads might work, but I wasn't able to get the serHC and serRN variables in the main thread.
Any help will be appreciated!!

As you have discovered it is not possible to read sequentially from serial ports: a blocking read over one port implies a loss of data simultaneous sent over the other port.
Use a thread based approach.
The following sketch should be enough to get started:
import serial
import time
import re
import threading
BYTES_TO_READ = 6
# read from serial port
def read_from_serial(board, port):
print("reading from {}: port {}".format(board, port))
payload = b''
ser = serial.Serial(port, 115200,timeout=.250, parity=serial.PARITY_NONE, rtscts=1)
bytes_count = 0
while bytes_count < BYTES_TO_READ:
read_bytes = ser.read(2)
# sum number of bytes returned (not 2), you have set the timeout on serial port
# see https://pythonhosted.org/pyserial/pyserial_api.html#serial.Serial.read
bytes_count = bytes_count + len(read_bytes)
payload = payload + read_bytes
# here you have the bytes, do your logic
# ...
print("READ from {}: [{}]".format(board, payload))
return
def main():
board = {
'JY': 'COM9',
'SILVER': 'COM10'
}
threads = []
for b in board:
t = threading.Thread(target=read_from_serial, args=(b, board[b],))
threads.append(t)
t.start()
# wait for all threads termination
for t in threads:
t.join()
main()
For learning about threading: https://pymotw.com/3/threading/
Periodic read from serials
Below a sketch for reading each TIME_PERIOD seconds.
A parte the infinite while loop around the read there is a "thread" loop with a nested try/catch block
for catching serials communication problems and retrying to connect after TIME_PERIOD.
Take it just as a starting example!
import serial
import time
import re
import threading
BYTES_TO_READ = 6
TIME_PERIOD = 5
def read_message(board, port, handle):
payload = b''
bytes_count = 0
while bytes_count < BYTES_TO_READ:
read_bytes = handle.read(2)
bytes_count = bytes_count + len(read_bytes)
payload = payload + read_bytes
# here you have the bytes, do your logic
# ...
print("READ from {}: [{}]".format(board, payload))
def serial_thread(board, port):
print("reading from {}: port {}".format(board, port))
while True:
try:
handle = serial.Serial(port, 115200,timeout=.250, parity=serial.PARITY_NONE, rtscts=1)
while True:
read_message(board, port, handle)
time.sleep(TIME_PERIOD)
except Exception as e:
print("ERROR: {}".format(e))
print("retrying in {} seconds".format(TIME_PERIOD))
handle.close()
time.sleep(TIME_PERIOD)
def main():
board = {
'JY': '/dev/ttyUSB0',
'SILVER': '/dev/ttyACM0'
}
threads = []
for b in board:
t = threading.Thread(target=serial_thread, args=(b, board[b],))
threads.append(t)
t.start()
# wait for all threads termination
for t in threads:
t.join()
main()

Weird strings when reading the GPS data from the receiver

I tried to read the GPS data from the receiver. I got strange strings as a result. Where is the problem?
import serial
port = "/dev/ttyUSB0" # Raspberry Pi 3
def parseGPS(data):
print(data)
#...
ser = serial.Serial(port, baudrate = 9600, timeout = 0.5)
while True:
data = ser.readline()
parseGPS(data)
result

The Problem was the wrong baud rate. Next example works problem-free:
import serial
port = "/dev/ttyUSB0" # Raspberry Pi 3
def parseGPS(data):
print(data)
# ...
ser = serial.Serial(port, baudrate=4800, timeout=0.5)
while True:
data = ser.readline()
parseGPS(data)