I want to write a program that communicates with intelligent meters (electricity, heat, etc.) over Meter-Bus protocol. I have an M-Bus <-> RS232 converter and RS232 <-> USB.
When I test my Python script with heat meter, by sending it a command it responds me with a long but correct frame, then if I query it again soon after I get only partial frame respond. Waiting around 30s helps to get a whole frame correctly. This partial respond is always the same.
Heat meter respond frames
But with electricity meter it's always the same, although the whole, correct frame would be shorter than in case of heat meter, I always get only the same part of it (little over half of bytes and never the beginning). Electricity meter respond frames
There's also command that initializes slave - slave responds with '\xE5', even though I see Rx diode blinking I cant catch this single byte.
What I have tried:
I tried reading certainly more bytes than could be in my buffer like ser.read(500)
Tried using ser.inwaiting but it would return 0 until I used time.sleep(1) prior to it
Loading chunks of bytes from pySerial inWaiting returns incorrect number of bytes
Manipulating xonxoff parameter and timeout.
Everything with no effect.
import serial
import time
def ser_to_mbus():
ser = serial.Serial(
port='COM4',
baudrate=2400,
bytesize=serial.EIGHTBITS,
parity=serial.PARITY_EVEN,
stopbits=serial.STOPBITS_ONE,
timeout=2,
#write_timeout=1,
xonxoff=False,
rtscts=False,
dsrdtr=False,
#inter_byte_timeout=0.01,
)
# Should receive a hex response '\xE5' with command below.
#to_send = b'\x68\x0B\x0B\x68\x73\xFD\x52\x84\x11\x10\x00\xFF\xFF\xFF\xFF\x63\x16'
#to_send = b'\x10\x7B\x01\x7C\x16' # heat meter request frame
to_send = b'\x10\x7B\xFD\x78\x16' # electricity meter request frame
ser.write(to_send)
ser.close()
def mbus_to_ser():
ser = serial.Serial(
port='COM4',
baudrate=2400,
bytesize=serial.EIGHTBITS,
parity=serial.PARITY_EVEN,
stopbits=serial.STOPBITS_ONE,
timeout=1,
#write_timeout=1,
xonxoff=1,
rtscts=False,
dsrdtr=False,
#inter_byte_timeout=0.2,
)
time.sleep(1) # Otherwise ser.in_waiting is empty.
buffer_size = ser.in_waiting
frame = ser.read(buffer_size)
return frame
ser_to_mbus()
frame = mbus_to_ser()
print(frame)
I've been monitoring M-Bus frames with EMU and my serial port with Serial Port Monitor. Frames sent by EMU and received are always correct, the same with Serial Monitor. If I use my code to send and receive frames, Serial Monitor gets same results as I did in my terminal - it sees wrong, 'sliced', frames.
Devices respond only after they receive correct requests (I see an Rx diode blinking).
I don't know if there is a problem with threading, or with a buffer or maybe Python script is not fast enough to catch all bytes from a buffer?
Related
I am trying to send a large amount of data from my Raspberry Pi 4 to my computer. I configured the Raspberry Pi as USB OTG Serial Gadget and the data is sent through the usb-c port to my computer.
Please take a look at the following code running on the Raspberry Pi. One MB of data is sent.
import serial
ser = serial.Serial( port='/dev/ttyGS0', baudrate=115200)
packet = bytearray()
for i in range(0, 1000000):
packet.append(0x2f)
ser.write(packet)
This is the code I am running first on my computer.
import serial
import time
ser = serial.Serial(port='COM30', baudrate=115200)
sum = 0
while 1:
bytesToRead = ser.inWaiting()
if bytesToRead > 0:
serial_line = ser.read(bytesToRead)
sum += bytesToRead
print(sum)
time.sleep(0.01)
I would expect that the received data has always the same length as the sent data. But in this example the computer receives a data length of around 990.000 Bytes in most cases. Even if I run the code without the sleep function on my computer, there are sometimes missing bytes.
How can I make sure that the data is sent and received without data loss?
First, if you have enabled logins on the serial port, you need to disable them first otherwise the port will be inaccessible.
Second, stop getty#ttyGS0.service and disable it.
And then everything will work fine.
I'm on Windows, reading data via Serial from another device at 115200 baud rate. The data coming in is from a microcontroller which has a sensor connected to it sending integer sensor (gyroscope) readings ranging from 1 to 25.
I used PuTTY to connect to it and I can read those values perfectly and they update almost instantly when I move my gyroscope.
However, when I use the python code below, it takes almost 20-25 seconds for it to update after I move the gyroscope. Why is it taking so long? How do I fix it? I've tried all sorts of things like changing timeout, adding sleep delays, nothing works.
import serial
ser = serial.Serial(COM1, 115200, timeout=0)
ser.flushInput()
while True:
DATA = ser.readline()
VAL = DATA[0:len(DATA)-1].decode("utf-8")
print(VAL)
EDIT:
import serial
import time
ser = serial.Serial(COM1, 115200, timeout=0)
ser.flushInput()
while True:
DATA = ser.readline()
VAL = DATA[0:len(DATA)-1].decode("utf-8")
print(VAL)
bufClear = ser.read(ser.inWaiting())
time.sleep(0.5)
Now I can get it to update quickly, however it seems to cut out some info. Sometimes. For example if I move my gyroscope to a value that corresponds to 22. The print output would be like
22
22
2
2
2
22
300 items flow from a production band at a minute. Using an optical micrometer, with pyserial and qthread, I am trying to 200 receives per second. But I can not get it right as I expected. There is not a problem when I keep it fixed, but I can not get the correct result when the object is moving. I need to detect width of the item flowing fast with the serial port. How would you recommend a method for this?
Question:. When I decrease the timeout, it occasionally breaks in the data stream.
Try the following, decrease down to 9600:.
Open port at “38400,8,E,1”, non blocking HW handshaking:
>>> ser = serial.Serial('COM3', 38400, timeout=0,
... parity=serial.PARITY_EVEN, rtscts=1)
>>> s = ser.read(100) # read up to one hundred bytes
... # or as much is in the buffer
I am trying to read frames sent to a ZigBee module plugged in the USB. Every frame gets discarded by the Python xBee package because the delimiter is 0x00 when it should be 0x7E. Actually it seems that every byte is also zero.
XCTU receives the frames perfectly.
I work with OS X, PyCharm, Python 3.4 and borrowed this code from Internet:
# Open serial port
ser = serial.Serial(PORT, BAUD_RATE)
# Create API object
xbee = ZigBee(ser,escaped=True)
# Continuously read and print packets
while True:
try:
response = xbee.wait_read_frame()
sa = hex(response['source_addr_long'][4:])
rf = hex(response['rf_data'])
datalength=len(rf)
# if datalength is compatible with two floats
# then unpack the 4 byte chunks into floats
if datalength==16:
h=struct.unpack('f',response['rf_data'][0:4])[0]
t=struct.unpack('f',response['rf_data'][4:])[0]
print (sa,' ',rf,' t=',t,'h=',h)
# if it is not two floats show me what I received
else:
print (sa,' ',rf)
except KeyboardInterrupt:
ser.close()
break
ser.close()
The program executes the xbee.wait_read_frame() call and waits there forever because no frame arrives.
I have tracked the call to "base.py" from the xBee package:
while True:
if self._callback and not self._thread_continue:
raise ThreadQuitException
if self.serial.inWaiting() == 0:
time.sleep(.01)
continue
byte = self.serial.read()
if byte != APIFrame.START_BYTE:
continue
The call to serial.read() always returns a zero.
I can't see anything wrong in code fragments you have provided. If you are reading just zeroes (are you?) from that serial port - there most likely something is wrong with serial port settings (e.g. you are reading at 115200 while data is being transmitted at 9600). What is the BAUD_RATE that you are using?
It's also could be a worth to test if you can access device with just dumb terminal.
I am using a script in Python to collect data from a PIC microcontroller via serial port at 2Mbps.
The PIC works with perfect timing at 2Mbps, also the FTDI usb-serial port works great at 2Mbps (both verified with oscilloscope)
Im sending messages (size of about 15 chars) about 100-150x times a second and the number there increments (to check if i have messages being lost and so on)
On my laptop I have Xubuntu running as virtual machine, I can read the serial port via Putty and via my script (python 2.7 and pySerial)
The problem:
When opening the serial port via Putty I see all messages (the counter in the message increments 1 by 1). Perfect!
When opening the serial port via pySerial I see all messages but instead of receiving 100-150x per second i receive them at about 5 per second (still the message increments 1 by 1) but they are probably stored in some buffer as when I power off the PIC, i can go to the kitchen and come back and im still receiving messages.
Here is the code (I omitted most part of the code, but the loop is the same):
ser = serial.Serial('/dev/ttyUSB0', 2000000, timeout=2, xonxoff=False, rtscts=False, dsrdtr=False) #Tried with and without the last 3 parameters, and also at 1Mbps, same happens.
ser.flushInput()
ser.flushOutput()
While True:
data_raw = ser.readline()
print(data_raw)
Anyone knows why pySerial takes so much time to read from the serial port till the end of the line?
Any help?
I want to have this in real time.
Thank you
You can use inWaiting() to get the amount of bytes available at the input queue.
Then you can use read() to read the bytes, something like that:
While True:
bytesToRead = ser.inWaiting()
ser.read(bytesToRead)
Why not to use readline() at this case from Docs:
Read a line which is terminated with end-of-line (eol) character (\n by default) or until timeout.
You are waiting for the timeout at each reading since it waits for eol. the serial input Q remains the same it just a lot of time to get to the "end" of the buffer, To understand it better: you are writing to the input Q like a race car, and reading like an old car :)
A very good solution to this can be found here:
Here's a class that serves as a wrapper to a pyserial object. It
allows you to read lines without 100% CPU. It does not contain any
timeout logic. If a timeout occurs, self.s.read(i) returns an empty
string and you might want to throw an exception to indicate the
timeout.
It is also supposed to be fast according to the author:
The code below gives me 790 kB/sec while replacing the code with
pyserial's readline method gives me just 170kB/sec.
class ReadLine:
def __init__(self, s):
self.buf = bytearray()
self.s = s
def readline(self):
i = self.buf.find(b"\n")
if i >= 0:
r = self.buf[:i+1]
self.buf = self.buf[i+1:]
return r
while True:
i = max(1, min(2048, self.s.in_waiting))
data = self.s.read(i)
i = data.find(b"\n")
if i >= 0:
r = self.buf + data[:i+1]
self.buf[0:] = data[i+1:]
return r
else:
self.buf.extend(data)
ser = serial.Serial('COM7', 9600)
rl = ReadLine(ser)
while True:
print(rl.readline())
You need to set the timeout to "None" when you open the serial port:
ser = serial.Serial(**bco_port**, timeout=None, baudrate=115000, xonxoff=False, rtscts=False, dsrdtr=False)
This is a blocking command, so you are waiting until you receive data that has newline (\n or \r\n) at the end:
line = ser.readline()
Once you have the data, it will return ASAP.
From the manual:
Possible values for the parameter timeout:
…
x set timeout to x seconds
and
readlines(sizehint=None, eol='\n') Read a list of lines,
until timeout. sizehint is ignored and only present for API
compatibility with built-in File objects.
Note that this function only returns on a timeout.
So your readlines will return at most every 2 seconds. Use read() as Tim suggested.