Develop Reference

How to wait for a large number packet erminator pyserial

I am debugging packets sent from a device with pyserial. The packets have a specific terminator pattern to differentiate between packets of data. The pattern is a large number (0x3423fe67). I am able to receive bytes and concatenate them as they come in with the code below.
ser = serial.Serial(
ports="/dev/ttyUSB0",
baudrate=115200,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS
)
y = b''
while True:
bytes = ser.inWaiting()
if bytes > 0:
x = ser.read(bytes)
y += binascii.hexify(x)
print(y)
I can see the pattern I'm looking for but how do I grab the packet and terminator once I see the terminator. I tried indexing the bytes object but that returns strange values. Everything received before the terminator is part of the packet.

Here is code that accumulates packets in a list. The code can be used as a guide.
The serialData is data that would be received over the serial port as an example (i.e. ser.read() which would replace the for piece in x: loop). Note that an arbitrary number of bytes can be received, and the test variable n is only used to test the algorithm at arbitrary incoming received character lengths.
The term variable needed hexlify to match the hexlify on the incoming data.
import binascii
import re
term = binascii.hexlify(b'\x34\x23\xfe\x67')
serialData = b'packet0\x34\x23\xfe\x67packet1\x34\x23\xfe\x67'
n = 5
x = [serialData[i:i+n] for i in range(0, len(serialData), n)]
y = b''
packets = []
for piece in x:
y += binascii.hexlify(piece)
for i in re.finditer(term, y):
packets.append(y[:i.start()])
y = y[i.end():]
print('packets = ', packets)

Python bytes convert

I have a confused in processing byte.
My code background is about rs485 communication with current meter.
hex_string is command sent to meter.
I want to bytes convert to string deal recv,
trying to use recv.decode("utf-8") this method,but is invalid.
This is variable of recv content:
b'\x05\x03\x03\x00\x15\x88K' (bytes) ->
0503020015884b (hex)
I want to deal with this paragraph: 0503020015884b (hex) .
Remove 050302 and 884b, keep the middle 0015 and convert 0015 to decimal.
How should i do ?
#!/usr/bin/env python3.7
import serial,time
import RPi.GPIO as GPIO
GPIO.setmode(GPIO.BOARD)
GPIO.setup(40,GPIO.OUT)
GPIO.output(40,GPIO.LOW)
ser = serial.Serial(port='/dev/ttyS0',baurdate=9600,parity='N',bytesize=8,stopbits=2,timeout=1)
hex_string = '0503010000018472'
print ('\nType-hex_string',type(hex_string))
ser.reset_input_buffer()
ser.reset_output_buffer()
GPIO.output(40,GPIO.HIGH)
ser.write(bytes.fromehex(hex_string))
time.sleep(0.01)
GPIO.output(40,GPIO.LOW)
recv = ser.read(11)
print ('\nOrigin:',recv)
print ('\nType-Origin:,'type(recv))
print ('\nHex:',recv.hex())
print ('\nType-recv.hex():',type(recv))

You can use struct.unpack():
from struct import unpack
src = b'\x05\x03\x03\x00\x15\x88K'
res = unpack('>H', src[3: 5])) # remove '>' if byte order is little endian
Also you can use struct.unpack_from():
res = unpack_from('>H', src, 3)
You can use int.from_bytes():
res = int.from_bytes(src[3: 5], byteorder='big')

Once you have the bytes array as recv you can convert the required parts to integer by:
int(recv.hex()[6:10],16)
where 6 and 10 is the range where you have the required value.

Weird Python output when printing

I'm very new to python and I'm trying to read pressure data from a Honeywell differential pressure sensor using the LibMPSSE library. I'm reading it from a Adafruit FT232H chip and I'm using python 2.7.6 on ubuntu Linux.
#!/usr/bin/env python
from mpsse import *
SIZE = 2 # 2 bytes MSB first
WCMD = "\x50" # Write start address command
RCMD = "\x51" # Read command
FOUT = "eeprom.txt" # Output file
try:
eeprom = MPSSE(I2C)
# print "%s initialized at %dHz (I2C)" % (eeprom.GetDescription(), eeprom.GetClock())
eeprom.Start()
eeprom.Write(WCMD)
# Send Start condition to i2c-slave (Pressure Sensor)
if eeprom.GetAck() == ACK:
# ACK received,resend START condition and set R/W bit to 1 for read
eeprom.Start()
eeprom.Write(RCMD)
if eeprom.GetAck() == ACK:
# ACK recieved, continue supply the clock to slave
data = eeprom.Read(SIZE)
eeprom.SendNacks()
eeprom.Read(1)
else:
raise Exception("Received read command NACK2!")
else:
raise Exception("Received write command NACK1!")
eeprom.Stop()
print(data)
eeprom.Close()
except Exception, e:
print "MPSSE failure:", e
According to the library, Read returns a string of size bytes and whenever I want to print the data, the only output I see is �����2��T_`ʋ�Q}�*/�eE�
. I've tried encoding with utf-8 and still no luck.

Python may print "weird" output when bytes/characters in the string (data in your case) contain non-printable characters. To "view" the content of the individual bytes (as integers or hex), do the following:
print(','.join(['{:d}'.format(x) for x in map(ord, data)])) # decimal
or
print(','.join(['{:02X}'.format(x) for x in map(ord, data)]))
Since the length of your data buffer is set by SIZE=2, to extract each byte from this buffer as an integer you can do the following:
hi, lo = map(ord, data)
# or:
hi = ord(data[0])
lo = ord(data[1])
To read more about ord and what it does - see https://docs.python.org/2/library/functions.html#ord

how to send an hexadecimal from python to the UART?

I need to understand how to send a message in hexadecimal format from UART for example:
msg='99' +'70b4c55ad8cdb7806a7b043069c4e0d8'
'99'is to distinct the start of message and the rest is the data.
As
import serial
import time
#serial port
ser = serial.Serial(
port='COM4',\
baudrate=230400,\
parity=serial.PARITY_NONE,\
stopbits=serial.STOPBITS_ONE,\
bytesize=serial.EIGHTBITS,\
timeout=0)
msg=b'\x99\x70\xb4\xc5\x5a\xd8\xcd\xb7\x80\x6a\x7b\x04\x30\x69\xc4\xe0\xd8'
ser.write(msg)
time.sleep(0.4)
while True:
print(ser.read(30))
ser.close() # close ports
But i don't unerstand th error:
msg= b'\x99\x70\xb4\xc5\x5a\xd8\xcd\xb7\x80\x6a\x7b\x04\x30\x69\xc4\xe0\xd8'
^
IndentationError: unexpected indent
I need then to compare by using c langage, to compare the first received byte 99 with 0x99.
int lenght = dev_uart_ptr->uart_read((void*)buffer,34 );
if (lenght<34)
{
if buffer [0]='0x99'
}
Thanks in advance.

It seems like you are loading to buf but using buffer in your c code.
You are reading 34 bytes to the buffer, then checking if the length is less than 34. Shouldn't work well. And judging by your example, it should be 17.
Encode will not convert msg to hex string. It will treat every character as a char, i.e. '0' is 48, '1' is 49 etc
Create bytes object using
msg = b'\x99\x70\xb4\xc5\x5a\xd8\xcd\xb7\x80\x6a\x7b\x04\x30\x69\xc4\xe0\xd8'
ser.write(msg)

leading zero / zero padding doesn't work properly

I am getting data from arduino via Serial (pySerial). It is then split in to 3 value by a delimiter(:). I am using slice ([0:5]) to remove 2 extra characters (I think it the new line).
So far so good. But then rjust or zfill or even format() simply does not work properly. If I use zfill(5) for example and the value output by arduino is 8.00 there is no change and I get 3 characters. If I use zfill(7) then it works and I get 7 total characters.
What is going on?
Arduino is outputting this every ~1 seconds: 22.00:36.00:58.00
import time
import serial
connected = False
port = '/dev/ttyACM0'
baud = 9600
ser = serial.Serial(port, baud)
# loop until arduino is ready
while not connected:
serin = ser.read()
connected = True
# read aruino output
while ser.readline():
readings = ser.readline().split(':', 2)
lum = readings[2][0:5].rjust(5, '0')
#lum = readings[2][0:5].zfill(5) same result as with rjust
print(lum)
time.sleep(1)
I am using Arduino Uno connected to Raspberry Pi running Rasberian OS with IDLE as python editor.

readline() contain newline. Strip newline.
readings = ser.readline().rstrip().split(':', 2)
>>> '0.00\n'.rjust(5, '0')
'0.00\n'
>>> '0.00\n'.rstrip().rjust(5, '0')
'00.00'