Develop Reference

Raspberry Pi, Python: How to send large data over usb serial?

I am trying to send a large amount of data from my Raspberry Pi 4 to my computer. I configured the Raspberry Pi as USB OTG Serial Gadget and the data is sent through the usb-c port to my computer.
Please take a look at the following code running on the Raspberry Pi. One MB of data is sent.
import serial
ser = serial.Serial( port='/dev/ttyGS0', baudrate=115200)
packet = bytearray()
for i in range(0, 1000000):
packet.append(0x2f)
ser.write(packet)
This is the code I am running first on my computer.
import serial
import time
ser = serial.Serial(port='COM30', baudrate=115200)
sum = 0
while 1:
bytesToRead = ser.inWaiting()
if bytesToRead > 0:
serial_line = ser.read(bytesToRead)
sum += bytesToRead
print(sum)
time.sleep(0.01)
I would expect that the received data has always the same length as the sent data. But in this example the computer receives a data length of around 990.000 Bytes in most cases. Even if I run the code without the sleep function on my computer, there are sometimes missing bytes.
How can I make sure that the data is sent and received without data loss?

First, if you have enabled logins on the serial port, you need to disable them first otherwise the port will be inaccessible.
Second, stop getty#ttyGS0.service and disable it.
And then everything will work fine.

Python hangs when using serial.write() on my Raspberry 2

I use pySerial for communication between RaspberryPi 2 and Arduino but after my first 100 write-calls it starts to become very slowly when writing.
My Code looks like this:
import serial
ser = serial.Serial("/dev/ttyACM0", 2000000, write_timeout=0)
while True:
byteData = getData()
sentBytes = ser.write(byteData)
if sentBytes == 4:
print("All Data was sent successfully!")
Everything is fine for the first second but then it hangs and I only send like 4 bytes each second. I also saw this post here but on my Raspbian machine a /dev/serial0 or /dev/ttyS0 doesn't exist. How I get this rushing like in the first second permanently?

You are using a very high baud rate, a buffer may be running full and cause the hick up after a short while.
Try a very conservative baud rate of 9600 and see if you have the same issue.
Also make sure your getData() actually always returns 4 bytes, otherwise your print statement might not get evaluated in every loop.

Python read from serial

I have a Card read connected to a ttyUSB0 device, I need to make a python script that when ran will wait (for instance 30s) for a card to pass and after it passes and receives a line with data closes the script and prints the line with data only. If the card does not pass within 30s closes the script.
Here is the script:
#!/usr/bin/python
import time
import serial
ser = serial.Serial(
port='/dev/ttyUSB0',
baudrate = 4800,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.SEVENBITS,
timeout=1
)
counter=0
while 1:
x=ser.readline()
print x
With this what happens is that it keeps printing the line for ever until I hit Ctrl+C
EDIT: Found how to wait for the read I want, now, what would be the best way to make the whole script timeout after 30s?
#!/usr/bin/python
import time
import serial
ser = serial.Serial(
port='/dev/ttyUSB0',
baudrate = 4800,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.SEVENBITS,
timeout=None
)
#while 1:
x=ser.read(size=16)
print x

You could certainly wait for 30 seconds to finish before testing if anything is inserted (as is suggested by Fejs' answer), but I assume you wouldn't want to wait 30 full seconds every time something is inserted.
If you wanted to continuously test for anything and terminate the script if something is found, you could do something like this:
for x in range(30):
if ser.read(size=16) != None:
x = ser.read(size=16)
#(DO WHATEVER WITH X HERE)
break
else:
time.sleep(1)
This will check for something readable continuously once every second, for 30 seconds. If something is found, it terminates the for loop.

In order to make script sleep for 30 seconds, just add following:
time.sleep(30)
before x = ser.read().

The easiest way I have found to poll a serial port is to utilize the time module. This is done by calculating the end point in time you wish to run to, and using that as the condition for a while loop.
#Set up serial connection
import time
time_end = time.time() + 30
while time.time() < time_end:
x = ser.read(16)
if x is not None:
break
As an aside, according to the pySerial api, passing a timeout value of None enters a blocking mode. In other words, the serial port sits and listens to the port until data is read.I would suggest passing a float instead (the value of which depends on your application).

Python: Serial timeout not working when using readline

Ok I don't get this. I've looked everywhere now, but I don't see why this is not working:
def main():
time = sys.argv[1]
ser = serial.Serial('/dev/ttyACM0',9600, timeout=1)
paramstr= "A 5 " + time + " 0 0 0"
ser.write(paramstr)
print 'sent'
print 'now listening...'
while True:
dbstr = ser.readline()
fo.write(str(dbstr));
fo.close()
ser.close()
print 'exiting.'
This is my def main in python. What I'm doing, is sending a string over serial from my Raspberry Pi to my Teensy (Arduino). The Teensy successfully starts a program and sends 1200 lines back over serial to the raspberry. This is working so far.
What does not work is the while loop. The data is written to the file, but the loop goes on forever, although the transmission (Teensy->RPi) has already stopped. For this case I implemented a timeout=1, but seems to be ignored. The program does not come out of the while loop.
Can somebody pleas help? Thanks in advance!

The timeout will not affect the while loop. It will only affect the maximum time that each call to read() or readline() will wait. If you want to stop looping when you are no longer receiving data, then stop looping when you are no longer receiving data. E.g. something like this:
while True:
dbstr = ser.readline()
fo.write(str(dbstr));
if not dbstr:
break

Reading serial data in realtime in Python

I am using a script in Python to collect data from a PIC microcontroller via serial port at 2Mbps.
The PIC works with perfect timing at 2Mbps, also the FTDI usb-serial port works great at 2Mbps (both verified with oscilloscope)
Im sending messages (size of about 15 chars) about 100-150x times a second and the number there increments (to check if i have messages being lost and so on)
On my laptop I have Xubuntu running as virtual machine, I can read the serial port via Putty and via my script (python 2.7 and pySerial)
The problem:
When opening the serial port via Putty I see all messages (the counter in the message increments 1 by 1). Perfect!
When opening the serial port via pySerial I see all messages but instead of receiving 100-150x per second i receive them at about 5 per second (still the message increments 1 by 1) but they are probably stored in some buffer as when I power off the PIC, i can go to the kitchen and come back and im still receiving messages.
Here is the code (I omitted most part of the code, but the loop is the same):
ser = serial.Serial('/dev/ttyUSB0', 2000000, timeout=2, xonxoff=False, rtscts=False, dsrdtr=False) #Tried with and without the last 3 parameters, and also at 1Mbps, same happens.
ser.flushInput()
ser.flushOutput()
While True:
data_raw = ser.readline()
print(data_raw)
Anyone knows why pySerial takes so much time to read from the serial port till the end of the line?
Any help?
I want to have this in real time.
Thank you

You can use inWaiting() to get the amount of bytes available at the input queue.
Then you can use read() to read the bytes, something like that:
While True:
bytesToRead = ser.inWaiting()
ser.read(bytesToRead)
Why not to use readline() at this case from Docs:
Read a line which is terminated with end-of-line (eol) character (\n by default) or until timeout.
You are waiting for the timeout at each reading since it waits for eol. the serial input Q remains the same it just a lot of time to get to the "end" of the buffer, To understand it better: you are writing to the input Q like a race car, and reading like an old car :)

A very good solution to this can be found here:
Here's a class that serves as a wrapper to a pyserial object. It
allows you to read lines without 100% CPU. It does not contain any
timeout logic. If a timeout occurs, self.s.read(i) returns an empty
string and you might want to throw an exception to indicate the
timeout.
It is also supposed to be fast according to the author:
The code below gives me 790 kB/sec while replacing the code with
pyserial's readline method gives me just 170kB/sec.
class ReadLine:
def __init__(self, s):
self.buf = bytearray()
self.s = s
def readline(self):
i = self.buf.find(b"\n")
if i >= 0:
r = self.buf[:i+1]
self.buf = self.buf[i+1:]
return r
while True:
i = max(1, min(2048, self.s.in_waiting))
data = self.s.read(i)
i = data.find(b"\n")
if i >= 0:
r = self.buf + data[:i+1]
self.buf[0:] = data[i+1:]
return r
else:
self.buf.extend(data)
ser = serial.Serial('COM7', 9600)
rl = ReadLine(ser)
while True:
print(rl.readline())

You need to set the timeout to "None" when you open the serial port:
ser = serial.Serial(**bco_port**, timeout=None, baudrate=115000, xonxoff=False, rtscts=False, dsrdtr=False)
This is a blocking command, so you are waiting until you receive data that has newline (\n or \r\n) at the end:
line = ser.readline()
Once you have the data, it will return ASAP.

From the manual:
Possible values for the parameter timeout:
…
x set timeout to x seconds
and
readlines(sizehint=None, eol='\n') Read a list of lines,
until timeout. sizehint is ignored and only present for API
compatibility with built-in File objects.
Note that this function only returns on a timeout.
So your readlines will return at most every 2 seconds. Use read() as Tim suggested.