When reading the serial port in python, all works fine until after hours where receiving data stops. This happens both on a laptop running Linux and on a Raspberry PI 2. Sending data still works at that point.
The source is still sending data (now and again). Restarting the python script helps.
The source sends small packages (4 - 12 bytes) and interval times may vary from minutes to over 12 hours.
My code, just only the receiving part, extracted from a bigger part, looks like this:
#!/usr/bin/python3
import tkinter
import os
import serial
class Transceiver():
def __init__(self, main):
self.port = serial.Serial(
port = "/dev/ttyUSB0",
baudrate = 38400,
bytesize = serial.EIGHTBITS,
parity = serial.PARITY_NONE,
stopbits = serial.STOPBITS_ONE,
timeout = 0
)
self.ReceivedData = bytearray()
self.Time = 0
def Receiver(self):
while (self.port.in_waiting > 0):
# Receiving data
self.Time = 0 # Reset time elapsed to 0
self.ReceivedData += self.port.read(1) # Add data to bytearray
if (self.Time == 20): # Some time elapsed, packet considered complete
print(''.join('{:02x} '.format(x) for x in self.ReceivedData))
self.port.reset_input_buffer()
self.ReceivedData = bytearray()
if (self.Time < 2000): # Just avoid running to 0 again
self.Time += 1 # Increment time elapsed
root.after(5, self.Receiver)
root = tkinter.Tk()
transceiver = Transceiver(root)
transceiver.Receiver()
root.mainloop()
It is just this receiving part that stops working after hours, I am trying to find the root cause.
So the above script is just an extract from a bigger script.
It controls the lights in the house. I want to run it on a raspberry PI, instead of a PC. The R'PI has a little 7" touch screen.
The bigger scripts has some decoding of the received data and updates a very simple GUI, just a bunch of small rectangles with some short name as text in it, indicating which lamps are on and off, and they act as buttons too, to toggle the lamps on and off. All works well, apart from the receiving part, though, in the end it may be possible that some other part disturbs the receiving part.
I have just started running just the above extraction of the script, to see if it keeps working without the GUI updates. I suspect the function Receiver fails to be restarted after some time. I need to have the above script running for a day to see of it keeps receiving though. I will update with the findings.
Update: The above script keeps working. The above is the extraction of a bigger program where only this part stops working after some hours. Receiving stops, but transmitting keeps working, hence I did not include that part.
By now I found the problem. In the receiver function, a decode function is called where above is the print line. Without calling the decode function, there is no problem. The decode function has some conditions, depending on the packet length and some values. Code for one of these conditions contained an error. I always started to run the script again in early evening, and then about 6 hours later a timer (at a fixed time late in the evening) would send the offending packet (a different format) that then caused the problem. The packet was not at fault, but my decoding of that packet.
Strange is that the script kept working, only the reception part stopped. I thought it stopped silently, but I printed quite a lot in this stage, hence I didn't see the error message telling me the problem of the use of an uninitialised variable. It's not in the above code. Hope the above code is useful for other people.
Related
I have 2 x Pimoroni BME280 and they both produce the same initial reading of 21.95*C 698.09hPa 76.34% humidity.
Using this simple code
import time
from smbus2 import SMBus
from bme280 import BME280
bus = SMBus(1)
bme280 = BME280(i2c_dev=bus)
while True:
temperature = bme280.get_temperature()
pressure = bme280.get_pressure()
humidity = bme280.get_humidity()
print('{:05.2f}*C {:05.2f}hPa {:05.2f}%'.format(temperature, pressure, humidity))
time.sleep(1)
and I always get as the first line of output as...
21.95*C 698.09hPa 76.34%
followed by the correct data for example...
18.70*C 993.54hPa 55.88%
18.70*C 993.53hPa 56.12%
18.71*C 993.54hPa 56.06%
18.71*C 993.54hPa 55.95%
Does anybody know why this is?
Currently I have the same thing on both of my BME280 so presume it's some sort of initialization thing on the first reading which must be discarded. If I run my program the only solution I can see it to ask twice what thr readings are and discard the first reading..
Thanks for reading and helping...
I would presume that this is caused due to an initial transient value being outputted by the sensor as a result of initialising your sensor.
It would be interesting to see how an Arduino would handle the initialisation process
vis-à-vis said transient value with your sensor.
As you said, if your continuous readings are 'correct', I would try to perhaps delay the output process or omit the first reading in some way or another.
I'm working on getting a weight from a usb scale using python and PyUSB. Here's my code
import sys
import usb.core
import usb.util
from reports import \
ReportFactory, STATUSES, ZERO_WEIGHT, STABLE_WEIGHT, DATA_REPORT
device = usb.core.find(idVendor=0x67b, idProduct=0x2303)
if device is None:
raise ValueError('Device Not Found')
if device.is_kernel_driver_active(0) is True:
device.detach_kernel_driver(0)
usb.util.claim_interface(device, 0)
device.set_configuration()
collected = 0
attempts = 50
while collected < attempts:
ret = device.read(0x2,0x0040)
sret=''.join([chr(x) for x in ret])
print "Return Raw: ",ret
print "Return : ", sret
print ReportFactory.build(ret)
# release the device
usb.util.release_interface(device, 0)
device.attach_kernel_driver(0)
The first time I run this program I get Resource busy error
The second run I get Operation timed out on the 5th dump.
Subsequent runs after that result in immediate timeouts.
There are several things going on here
Why do I get a resource busy error on first run? I would think the kernel_driver if statement I have should take care of it.
Why do I get data that looks nothing like what a USB scale should produce?
Why the timeout after 4 dumps of data? Everytime!
The device information is thus
Any help would be much appreciated!
Ok, anyone that is struggling to get anything from the GP100 Brecknell scale using PyUSB, switch to PySerial. Since this is one of the cheapest USB scales on the market, I'm guessing this information will help many.
I emailed the manufacturer for documentation and they've sent me some valuable serial protocol information.
If you set the protocol of the scale to 7010 you can use python code that looks like this
import time
import serial
ser = serial.Serial(
port='/dev/ttyUSB0',
baudrate=2400,
bytesize=8,
parity='N',
stopbits=2
)
while 1:
x = ser.read(100)
print(x)
This will stream 8 bytes of data.
The first byte is status information that can be interpreted thusly
Bits 2-4
Bits 5-8
Bytes 2 & 3 can be safely ignored.
Bytes 4-8 have the weight information, represented by characters 0-9 (e.g. 00023 = 2.3 lbs). This scale is only accurate +- 0.6 lbs and does not work in ounces, perhaps other models using the same serial controller utilize the oz capabilities. The PG100 does not.
I am attempting to read multiple Reflective IR Sensors continuously until each one is blocked. I am using an Arduino running Standard Firmata and Pythons Pyfirmata library.
Every time I try a standard read without printing the result in a while loop I am getting a 15 to 30 second delay (I cannot figure out why):
Example1 - works but with random unexplained delay:
Sensor1 = board.get_pin('a,0,i') #analogue, pin 0, input mode
while Sensor1.read() != 0: #Sensor defaults to 0.6 V but when blocked 0.0v
Sensor1.read()
Example2 - works but I now have a bunch of sensor read outputs:
Sensor1 = board.get_pin('a,0,i') #analogue, pin 0, input mode
while Sensor1.read() != 0: #Sensor defaults to 0.6 V but when blocked 0.0v
Sensor1.read()
print(Sensor1.read()) # this prints a bunch of read outputs
For some reason when I add the print(Sensor1.read()) I will get an immediate response when the sensor is blocked. But if I remove this portion of code to eliminate the garbage output I get an unexplained time delay in between when the sensor is blocked and when it is recognized by the code and moves on. What I would like to do is constantly read the sensor without printing that read and get an immediate response of breaking the while loop once the sensor is blocked and produces the 0.0v. I believe that I also have the option of suppressing the print outputs for these while loops but I want to know if there is an alternative? Thank you so much for reviewing this question and thanks a million for any help!
The read() might not always be exact 0. Maybe use a threshold value, for example:
while Sensor1.read() > 10:
Another solution would be to use interrupts and set flags, so you don't have to do the polling (and could put the device to sleep).
I know i'ts an old topic, but I ran into this problem with pyfirmata a few days ago, and today I solved it simply by printing a new line with the end= parameter, it creates a new line, but the following prints stays on the same line.
Sensor1 = board.get_pin('a,0,i')
while Sensor1.read() != 0:
Sensor1.read()
print('', end='')
I use pySerial for communication between RaspberryPi 2 and Arduino but after my first 100 write-calls it starts to become very slowly when writing.
My Code looks like this:
import serial
ser = serial.Serial("/dev/ttyACM0", 2000000, write_timeout=0)
while True:
byteData = getData()
sentBytes = ser.write(byteData)
if sentBytes == 4:
print("All Data was sent successfully!")
Everything is fine for the first second but then it hangs and I only send like 4 bytes each second. I also saw this post here but on my Raspbian machine a /dev/serial0 or /dev/ttyS0 doesn't exist. How I get this rushing like in the first second permanently?
You are using a very high baud rate, a buffer may be running full and cause the hick up after a short while.
Try a very conservative baud rate of 9600 and see if you have the same issue.
Also make sure your getData() actually always returns 4 bytes, otherwise your print statement might not get evaluated in every loop.
Hi i'm quite a newbie to Python. I' writting a simple LAN game (not simple for me) using a pygame module.
Here's the problem - I have two computers (one old intel Atom netbook, the other intel i5 NTB). I want to achieve at least 5 FPS (the netbook is slowering the NTB, but not so much, now i have around 1,5 FPS), but calling recv() function twice a main loop takes total around 0,5 seconds on each machine. The wifi signal is strong and the router is 300Mbit/s and it sends a short roughly 500-character string. As you can see for measuring time i use time.clock().
Here's the part of the "server" code, which i usually run on the i5 NTB:
while 1:
start = time.clock()
messagelen = c.recv(4) #length of the following message (fixed 4 character)
if " " in messagelen:
messagelen = messagelen.replace(" ","")
message = cPickle.loads(c.recv(int(messagelen))) #list of the arrows, other player position and changes in the game map
arrowsmod = message[0]
modtankposan = message[1]
removelistmod = message[2]
for i in removelistmod:
try:
randopos.remove(i)
except ValueError:
randopossv.remove(i)
print time.clock()-start
tosendlist=[]
if len(arrows) == 0: #if there are no arrows it appends only an empty list
tosendlist.append([])
else:
tosendlist.append(arrows)
tosendlist.append([zeltankpos, 360-angle])
if len(removelist) == 0: #if there are no changes of the map it appends only an empty list
tosendlist.append([])
else:
tosendlist.append(removelist)
removelist=[]
tosend=cPickle.dumps(tosendlist)
tosendlen = str(len(tosend))
while len(tosendlen)<4:
tosendlen+=" "
c.sendall(tosendlen) #sends the length to client
c.sendall(tosend) #sends the actual message(dumped list of lists) to client
...something else which takes <0,05 sec on the NTB
Here's the part of the "client" game code (just inverted the beginning - sending/receiving parts):
while 1:
tosendlist=[]
if len(arrows) == 0: #if there are no arrows it appends only an empty list
tosendlist.append([])
else:
tosendlist.append(arrows)
tosendlist.append([zeltankpos, 360-angle])
if len(removelist) == 0: #if there are no changes of the map it appends only an empty list
tosendlist.append([])
else:
tosendlist.append(removelist)
removelist=[]
tosend=cPickle.dumps(tosendlist)
tosendlen = str(len(tosend))
while len(tosendlen)<4:
tosendlen+=" "
s.sendall(tosendlen) #sends the length to server
s.sendall(tosend) #sends the actual message(dumped list of lists) to server
start = time.clock()
messagelen = s.recv(4) #length of the following message (fixed 4 character)
if " " in messagelen:
messagelen = messagelen.replace(" ","")
message = cPickle.loads(s.recv(int(messagelen))) #list of the arrows, other player position and changes in the game map
arrowsmod = message[0]
modtankposan = message[1]
removelistmod = message[2]
for i in removelistmod:
try:
randopos.remove(i)
except ValueError:
randopossv.remove(i)
print time.clock()-start
... rest which takes on the old netbook <0,17 sec
When I run let's say a single player version of the game on one machine (without the socket module) on the i5 NTB it has 50 FPS in the up left corner of the map and 25 FPS in the down right corner (the 1000x1000 pixel map contains 5x5 pixel squares, i think it's slower because of the bigger coordinates, but i can't believe that so much. BTW recv while ran as a LAN game in the down right corner of the map takes approx. the same time)
on the Atom netbook it has 4-8 FPS.
So could you please tell me, why it's so slow? The computers are not synchronized, one is faster, the other slower, but it can't be that they are waiting for each other, it would be max 0,17 secs delay, right? And plus the long recv calling would be only on the faster computer?
Also I don't exactly know how the send/recv function work. It's weird the sendall takes literally no time while receiving takes 0,5 secs. Maybe sendall
is trying to send in the background while the rest of the program continues forward.
As mentioned by Armin Rigo, recv will return after packets are received by the socket, but packets don't necessarily need to be transmitted immediately after calling send. While send returns immediately, OS caches the data internally and might wait some time for more data being written to the the socket before actually transmitting it; this is called Nagle's algorithm and avoids sending lots of small packets over the network. You can disable it and push packets quicker to the wire; try enabling TCP_NODELAY options on the sending socket (or both if your communication is bidirectional), by calling this:
sock.setsockopt(socket.IPPROTO_TCP, socket.TCP_NODELAY, 1)
This could potentially reduce amount of time recv is sleeping due to no data.
As the Wikipedia states:
This algorithm interacts badly with TCP delayed acknowledgments, a
feature introduced into TCP at roughly the same time in the early
1980s, but by a different group. With both algorithms enabled,
applications that do two successive writes to a TCP connection,
followed by a read that will not be fulfilled until after the data
from the second write has reached the destination, experience a
constant delay of up to 500 milliseconds, the "ACK delay". For this
reason, TCP implementations usually provide applications with an
interface to disable the Nagle algorithm. This is typically called the
TCP_NODELAY option.
There is a mention of 0.5s which you're seeing in your benchmark, so this might be a reason.
Yes, send() or sendall() will occur in the background (unless the connexion is saturated right now, i.e. there is already too much data waiting to be sent). By contrast, recv() will immediately get the data only if it arrived already, but if none did, it waits. Then it returns possibly a fraction of it. (I am assuming that c is a TCP socket, not a UDP one.) Note that you should not assume that recv(N) returns N bytes; you should write a function like this:
def recvall(c, n):
data = []
while n > 0:
s = c.recv(n)
if not s: raise EOFError
data.append(s)
n -= len(s)
return ''.join(data)
Anyway, to the point. The issue is not the speed of recv(). If I understood correctly, there are four operations:
the server renders (1/25th sec)
the server sends something on the socket, received by the client;
the client renters (1/4th sec);
the client send something back on the socket.
This takes almost (0.3 + 2 * network_delay) seconds. Nothing occurs in parallel. If you want more frames-per-second, you need to parallelize some of these four operations. For example, let's assume reasonably that the operation 3 is by far the slowest. Here's how we can make 3 run in parallel with the three other operations. You should change the client so that it receives data, process it, and immediately sends an answer to the server; and only then it proceeds to render it. This should be enough in this case, as it takes 1/4th seconds to do this rendering, which should be enough time for the answer to reach the server, the server to render, and the next packet to be sent again.
I ended up here when having same issue with it appearing that socket recv in python to be super slow. The fix for me (after days) was to do something along the lines:
recv_buffer = 2048 # ? guess & check
...
rx_buffer_temp = self._socket.recv(recv_buffer)
rx_buffer_temp_length = len(rx_buffer_temp)
recv_buffer = max(recv_buffer, rx_buffer_temp_length) # keep to the max needed/found
The gist of it is set to the amount of bytes trying to receive closest to the actual expected.