I am getting data from arduino via Serial (pySerial). It is then split in to 3 value by a delimiter(:). I am using slice ([0:5]) to remove 2 extra characters (I think it the new line).
So far so good. But then rjust or zfill or even format() simply does not work properly. If I use zfill(5) for example and the value output by arduino is 8.00 there is no change and I get 3 characters. If I use zfill(7) then it works and I get 7 total characters.
What is going on?
Arduino is outputting this every ~1 seconds: 22.00:36.00:58.00
import time
import serial
connected = False
port = '/dev/ttyACM0'
baud = 9600
ser = serial.Serial(port, baud)
# loop until arduino is ready
while not connected:
serin = ser.read()
connected = True
# read aruino output
while ser.readline():
readings = ser.readline().split(':', 2)
lum = readings[2][0:5].rjust(5, '0')
#lum = readings[2][0:5].zfill(5) same result as with rjust
print(lum)
time.sleep(1)
I am using Arduino Uno connected to Raspberry Pi running Rasberian OS with IDLE as python editor.
readline() contain newline. Strip newline.
readings = ser.readline().rstrip().split(':', 2)
>>> '0.00\n'.rjust(5, '0')
'0.00\n'
>>> '0.00\n'.rstrip().rjust(5, '0')
'00.00'
Related
I am debugging packets sent from a device with pyserial. The packets have a specific terminator pattern to differentiate between packets of data. The pattern is a large number (0x3423fe67). I am able to receive bytes and concatenate them as they come in with the code below.
ser = serial.Serial(
ports="/dev/ttyUSB0",
baudrate=115200,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS
)
y = b''
while True:
bytes = ser.inWaiting()
if bytes > 0:
x = ser.read(bytes)
y += binascii.hexify(x)
print(y)
I can see the pattern I'm looking for but how do I grab the packet and terminator once I see the terminator. I tried indexing the bytes object but that returns strange values. Everything received before the terminator is part of the packet.
Here is code that accumulates packets in a list. The code can be used as a guide.
The serialData is data that would be received over the serial port as an example (i.e. ser.read() which would replace the for piece in x: loop). Note that an arbitrary number of bytes can be received, and the test variable n is only used to test the algorithm at arbitrary incoming received character lengths.
The term variable needed hexlify to match the hexlify on the incoming data.
import binascii
import re
term = binascii.hexlify(b'\x34\x23\xfe\x67')
serialData = b'packet0\x34\x23\xfe\x67packet1\x34\x23\xfe\x67'
n = 5
x = [serialData[i:i+n] for i in range(0, len(serialData), n)]
y = b''
packets = []
for piece in x:
y += binascii.hexlify(piece)
for i in re.finditer(term, y):
packets.append(y[:i.start()])
y = y[i.end():]
print('packets = ', packets)
I'm new to using Python3 for data acquisition. I'm trying to find a way to parse binary data from a serial port on Linux.
import serial
ser = serial.Serial(
port='/dev/ttyS0',
baudrate = 9600,
parity=serial.PARITY_NONE,
stopbits=serial.STOPBITS_ONE,
bytesize=serial.EIGHTBITS,
timeout=1)
counter = 0
while 1:
x = ser.read(31)
print (x)
This gives me a string which I'm not sure about the format of:
x='\x00\x00\x91\x00\x02\x88BM\x00\x1c\x00\x00\x00\x01\x00\x01\x00\x00\x00\x01\x00\x01\x00\xe1\x00K\x00\x1a\x00\x02\x00\x00'
using
x.encode('hex')
gives a string of hex values
x='000091000288**424d**001c00000001000100000001000100e1004b001a00020000'
where 0x42 is the end of message and 0x4d is start of message.
I can convert it into a base 10 list using
y = map(ord,x)
print(y)
Then I have a way to re-order the message using the indexes but surely there is a neater way? How do I create a list which starts at 0x4d to parse with?
If you are using python3, this is likely already bytes:
x='\x00\x00\x91\x00\x02\x88BM\x00\x1c\x00\x00\x00\x01\x00\x01\x00\x00\x00\x01\x00\x01\x00\xe1\x00K\x00\x1a\x00\x02\x00\x00'
It likely looks this way because Python printed it for you, and all of the non-ascii characters are shown in hex. Your start of message is in 0x42, 0x4d which is BM in ascii and can be seen in the data above between 0x88 and 0x00 as \x88BM\x00.
I would suggest just iterating over the byte array in x to do your parsing. The encoding and mapping should not be needed.
for b in x:
if b == 0x4d:
found_byte1 = True
... # etc
I need to understand how to send a message in hexadecimal format from UART for example:
msg='99' +'70b4c55ad8cdb7806a7b043069c4e0d8'
'99'is to distinct the start of message and the rest is the data.
As
import serial
import time
#serial port
ser = serial.Serial(
port='COM4',\
baudrate=230400,\
parity=serial.PARITY_NONE,\
stopbits=serial.STOPBITS_ONE,\
bytesize=serial.EIGHTBITS,\
timeout=0)
msg=b'\x99\x70\xb4\xc5\x5a\xd8\xcd\xb7\x80\x6a\x7b\x04\x30\x69\xc4\xe0\xd8'
ser.write(msg)
time.sleep(0.4)
while True:
print(ser.read(30))
ser.close() # close ports
But i don't unerstand th error:
msg= b'\x99\x70\xb4\xc5\x5a\xd8\xcd\xb7\x80\x6a\x7b\x04\x30\x69\xc4\xe0\xd8'
^
IndentationError: unexpected indent
I need then to compare by using c langage, to compare the first received byte 99 with 0x99.
int lenght = dev_uart_ptr->uart_read((void*)buffer,34 );
if (lenght<34)
{
if buffer [0]='0x99'
}
Thanks in advance.
It seems like you are loading to buf but using buffer in your c code.
You are reading 34 bytes to the buffer, then checking if the length is less than 34. Shouldn't work well. And judging by your example, it should be 17.
Encode will not convert msg to hex string. It will treat every character as a char, i.e. '0' is 48, '1' is 49 etc
Create bytes object using
msg = b'\x99\x70\xb4\xc5\x5a\xd8\xcd\xb7\x80\x6a\x7b\x04\x30\x69\xc4\xe0\xd8'
ser.write(msg)
I am a beginner in python, and I am trying to read the data from several sensors (humidity, temperature, pressure sensors...) that I connect with a usb hub to my computer. My main goal is to record every five minutes the different values of those sensors and then store it to analyse it.
I have got all the data sheets and manuals of my sensors (which are from Hygrosens Instruments), I know how they work and what kind of data they are sending. But I do not know how to read them. Below is what I tried, using pyserial.
import serial #import the serial library
from time import sleep #import the sleep command from the time library
import binascii
output_file = open('hygro.txt', 'w') #create a file and allow you to write in it only. The name of this file is hygro.txt
ser = serial.Serial("/dev/tty.usbserial-A400DUTI", 9600) #load into a variable 'ser' the information about the usb you are listening. /dev/tty.usbserial.... is the port after plugging in the hygrometer, 9600 is for bauds, it can be diminished
count = 0
while 1:
read_byte = ser.read(size=1)
So now I want to find the end of the line of the data as the measurement informations that I need are in a line that begins with 'V', and if the data sheet of my sensor, it said that a line ends by , so I want to read one byte at a time and look for '<', then 'c', then 'r', then '>'. So I wanted to do this:
while 1:
read_byte = ser.read(size=8) #read a byte
read_byte_hexa =binascii.hexlify(read_byte) #convert the byte into hexadecimal
trad_hexa = int(read_byte_hexa , 16) #convert the hexadecimal into an int in purpose to compare it with another int
trad_firstcrchar = int('3c' , 16) #convert the hexadecimal of the '<' into a int to compare it with the first byte
if (trad_hexa == trad_firstcrchar ): #compare the first byte with the '<'
read_byte = ser.read(size=1) #read the next byte (I am not sure if that really works)
read_byte_hexa =binascii.hexlify(read_byte)# from now I am doing the same thing as before
trad_hexa = int(read_byte_hexa , 16)
trad_scdcrchar = int('63' , 16)
print(trad_hexa, end='/')# this just show me if it gets in the condition
print(trad_scdcrchar)
if (trad_hexa == trad_scdcrchar ):
read_byte = ser.read(size=1) #read the next byte
read_byte_hexa =binascii.hexlify(read_byte)
trad_hexa = int(read_byte_hexa , 16)
trad_thirdcrchar = int('72' , 16)
print(trad_hexa, end='///')
print(trad_thirdcrchar)
if (trad_hexa == trad_thirdcrchar ):
read_byte = ser.read(size=1) #read the next byte
read_byte_hexa =binascii.hexlify(read_byte)
trad_hexa = int(read_byte_hexa , 16)
trad_fourthcrchar = int('3e' , 16)
print(trad_hexa, end='////')
print(trad_fourthcrchar)
if (trad_hexa == trad_fourthcrchar ):
print ('end of the line')
But I am not sure that it works, I mean I think it does not have the time to read the second one, the second byte I am reading, it's not exactly the second one. So that's why I want to use a buffer, but I don't really get how I can do that. I am going to look for it, but if someone knows an easier way to do what I want, I am ready to try it!
Thank you
You seem to be under the impression that the end-of-line character for that sensor's communication protocol is 4 different characters: <, c, r and >. However, what is being referred to is the carriage return, often denoted by <cr> and in many programming languages just by \r (even though it looks like 2 characters, it represents just one character).
You could simplify your code greatly by reading in the data from the sensors line by line, as the protocol is structured. Here's something to help you get started:
import time
def parse_info_line(line):
# implement to your own liking
logical_channel, physical_probe, hardware_id, crc = [line[index:index+2] for index in (1, 3, 5, 19)]
serialno = line[7:19]
return physical_probe
def parse_value_line(line):
channel, crc = [line[ind:ind+2] for ind in (1,7)]
encoded_temp = line[3:7]
return twos_comp(int(encoded_temp, 16), 16)/100.
def twos_comp(val, bits):
"""compute the 2's compliment of int value `val`"""
if (val & (1 << (bits - 1))) != 0: # if sign bit is set e.g., 8bit: 128-255
val = val - (1 << bits) # compute negative value
return val # return positive value as is
def listen_on_serial(ser):
ser.readline() # do nothing with the first line: you have no idea when you start listening to the data broadcast from the sensor
while True:
line = ser.readline()
try:
first_char = line[0]
except IndexError: # got no data from sensor
break
else:
if first_char == '#': # begins a new sensor record
in_record = True
elif first_char == '$':
in_record = False
elif first_char == 'I':
parse_info_line(line)
elif first_char == 'V':
print(parse_value_line(line))
else:
print("Unexpected character at the start of the line:\n{}".format(line))
time.sleep(2)
The twos_comp function was written by travc and you are encouraged to upvote his answer when you have enough reputation and if you intend to use his code (and even if you won't, it's still a good answer, I upvoted it just now). The listen_on_serial could be improved as well (many Python programmers will recognize the switch-structure and implement it with a dictionary rather than if... elif... elif...), but this is only intended to get you started.
As a test, the following code extract simulates the sensor sending some data (which is line-delimited, using the carriage return as the end-of-line marker), which I copied from the pdf you linked to (FAQ_terminalfenster_E.pdf).
>>> import serial
>>> import io
>>>
>>> ser = serial.serial_for_url('loop://', timeout=1)
>>> serio = io.TextIOWrapper(io.BufferedRWPair(ser, ser), newline='\r', line_buffering=True)
>>> serio.write(u'A1A0\r' # simulation of starting to listen halfway between 2 records
... '$\r' # marks the end of the previous record
... '#\r' # marks the start of a new sensor record
... 'I0101010000000000001B\r' # info about a sensor's probe
... 'V0109470D\r' # data matching that probe
... 'I0202010000000000002B\r' # other probe, same sensor
... 'V021BB55C\r') # data corresponding with 2nd probe
73L
>>>
>>> listen_on_serial(serio)
23.75
70.93
>>>
Note that it is recommended by the pyserial docs to be using TextIOWrapper when the end-of-line character is not \n (the linefeed character), as was also answered here.
Using a raspberry pi I am having issues reading data that is being transmitted serially. My code was working when I tested it on a different machine but isn't working now.
The baud rate is 9600 w/ no parity, 8 data bits, 1 stop bit and I want the program to handle a variable length of characters (Sometimes 100K+). The reading portion of the code is as follows:
ser = serial.Serial('/dev/ttyAMA0', 9600, parity = serial.PARITY_NONE, timeout=1)
While True:
data = ser.read(1)
bytesToRead = ser.inWaiting()
if bytesToRead:
data = data + ser.read(bytesToRead)
encodedData = data.encode('hex')
With this code, Shouldn't I be able to read all the characters as Hex as long as the baud/parity/etc match up with the transmitting system?
While True:
data = ser.read(1)
This waits for a single character to be read (times out after 1s due to the timeout specified in the Serial constructor) and stores it in data
bytesToRead = ser.inWaiting()
if bytesToRead:
data = data + ser.read(bytesToRead)
encodedData = data.encode('hex')
Now instantly check for any other characters in buffer - this will usually be zero. Due to the fact that you're running at 9600 baud, Python will usually see the characters come in one at a time. So your if bytesToRead statement will mostly be false as each incoming character is consumed by the above ser.read(1).
If you just want to process each character individually, you can do:
While True:
data = ser.read(1)
if data:
encodedData = data.encode('hex')
Or if you want to keep adding it to a buffer, use something like:
data = ''
While True:
bytesToRead = ser.inWaiting()
if bytesToRead:
data += ser.read(bytesToRead)
encodedData = data.encode('hex')
if encodedData.startswith('1234deadb33f`):
data = data[6:] # strip 6 chars from start of data