I'm using PySerial library for a project and it works fine. Though, now that my requirements have changed and i need to implement a non blocking serial write option. I went through PySerial official documentation as well as several examples however, couldn't find a suitable option. Now my question is : is non-blocking write possible with PySerial ? If yes, how ?
Thanks in advance.
From the pySerial Documentation:
-timeout = None: wait forever
-timeout = 0: non-blocking mode (return immediately on read)
-timeout = x: set timeout to x seconds (float allowed)
Writes are blocking by default, unless writeTimeout is set. For possible values refer to the list for timeout above
So I guess when you open your serial port you can set:
ser = serial.Serial(port='/dev/ttyXXX', writeTimeout = 0)
It turns out. To drain serial output physically, we have serial.drainOutput() function. It works but if you're looking for a real time operation it might not help as python is not a very good language if you're expecting a real time performance.
Hope it helps.
Try threading the send function.
import _thread as thread
def send_message(message, serial_interface)
serial_interface.write(message)
print("Done sending message!")
serial_interface = serial.Serial('COM1')
message = "hi"
thread.start_new_thread(send_message, (message, serial_interface))
Related
I'm struggling with pySerial. To be brief ... The code below works great when executed in the Python Shell ...
>>> import serial
>>> s=serial.Serial("COM5", 9600)
>>> while(1):
s.write("#")
s.readline()
Produces the output below in the shell:
1L
'56.73\r\n'
1L
'56.73\r\n'
When the same code is written in a script say "readSerial.py" the script will either not transmit the hashtag that triggers the serial device to transmit the data, or will not receive the replied data.
I'm using pySerial 3, but have noticed the same behavior with 2.7. Using Python 2.7.10 64 bit on Win10. But also noticed this behavior on Raspberry Pi with /dev/ttyACM0. I would really like to have this solved. I'm not that experienced in Python so this might be an oversight.
Hardware is checked and double checked.
Thanks,
KK
Thanks, but I really know how to print data from Python. The problem is really with pySerial. Here is the complete code, please don't discus errors in commented out code. These are of no concern here.
#from numpy import array
#import matplotlib.animation as animation
import time
import serial as s
#data = array([])
Arduino = s.Serial("COM5", 9600)
i = 0
while (1):
try:
Arduino.write("#")
time.sleep(.1)
inString = Arduino.readline()
data = float(inString)
print i, ":", data
i += 1
time.sleep(1)
except KeyboardInterrupt:
break
Arduino.close()
But like said this doesn't work. As far as I can tell the readline() function does not return. And ... there 's really no point in making it return by setting the tx timeout. To add to the mistery; When the code is debugged (i.e. stepped trough) it does work.
Thanks in advance,
KK
From the FAQ :
Example works in serial.tools.miniterm but not in script.
The RTS and DTR lines are switched when the port is opened. This may
cause some processing or reset on the connected device. In such a
cases an immediately following call to write() may not be received by
the device.
A delay after opening the port, before the first write(), is
recommended in this situation. E.g. a time.sleep(1)
I am reading string from serial in a loop and realize that the processor is at 100% (RaspberryPI) while waiting for the next serial.read().
I found recommendation to add a few sleeps here and there, but doing this might cause missing serial data. In theorie I am getting a string from serial every 5 seconds, but could be a bit more or less and not in my control.
Is there a way to solve this in python better and with less processor use?
#!/usr/bin/env python
import serial
ser = serial.Serial("/dev/ttyUSB0", 57600, timeout=0)
def sr():
while True:
for line in ser.read():
try:
response = ser.readlines(None)
response = str(response)
print response
except:
print datetime.datetime.now(), " No data from serial connection."
if __name__ == '__main__':
sr
ser.close()
from what i remember (been a while since i used pyserial) i am sure that serial uses buffers, so as long as your message doesn't fill the buffer you shouldn't lose any data.
assuming i'm looking at the docs for the right module the following page:
[Pyserial docs][1]http://pyserial.sourceforge.net/pyserial_api.html
make mention about buffers both on the input and output.
so you should have no problems with putting sleeps into your program as the buffers will collect the data until you read it. (assuming your messages are not big enough to cause an overflow)
James
I am reading serial data like this:
connected = False
port = 'COM4'
baud = 9600
ser = serial.Serial(port, baud, timeout=0)
while not connected:
#serin = ser.read()
connected = True
while True:
print("test")
reading = ser.readline().decode()
The problem is that it prevents anything else from executing including bottle py web framework. Adding sleep() won't help.
Changing "while True"" to "while ser.readline():" doesn't print "test", which is strange since it worked in Python 2.7. Any ideas what could be wrong?
Ideally I should be able to read serial data only when it's available. Data is being sent every 1,000 ms.
Using a separate thread is totally unnecessary. Just follow the example below for your infinite while loop instead.
I use this technique in my eRCaGuy_PyTerm serial terminal program here (search the code for inWaiting() or in_waiting).
Notes:
To check your python3 version, run this:
python3 --version
My output when I first wrote and tested this answer was Python 3.2.3.
To check your pyserial library (serial module) version, run this--I first learned this here:
python3 -c 'import serial; \
print("serial.__version__ = {}".format(serial.__version__))'
This simply imports the serial module and prints its serial.__version__ attribute.
My output as of Oct. 2022 is: serial.__version__ = 3.5.
If your pyserial version is 3.0 or later, use property in_waiting in the code below. If your pyserial version is < 3.0, use function inWaiting() in the code below. See the official pyserial documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
Non-blocking, single-threaded serial read example
import serial
import time # Optional (required if using time.sleep() below)
ser = serial.Serial(port='COM4', baudrate=9600)
while (True):
# Check if incoming bytes are waiting to be read from the serial input
# buffer.
# NB: for PySerial v3.0 or later, use property `in_waiting` instead of
# function `inWaiting()` below!
if (ser.inWaiting() > 0):
# read the bytes and convert from binary array to ASCII
data_str = ser.read(ser.inWaiting()).decode('ascii')
# print the incoming string without putting a new-line
# ('\n') automatically after every print()
print(data_str, end='')
# Put the rest of your code you want here
# Optional, but recommended: sleep 10 ms (0.01 sec) once per loop to let
# other threads on your PC run during this time.
time.sleep(0.01)
This way you only read and print if something is there. You said, "Ideally I should be able to read serial data only when it's available." This is exactly what the code above does. If nothing is available to read, it skips on to the rest of your code in the while loop. Totally non-blocking.
(This answer originally posted & debugged here: Python 3 non-blocking read with pySerial (Cannot get pySerial's "in_waiting" property to work))
pySerial documentation: http://pyserial.readthedocs.io/en/latest/pyserial_api.html
UPDATE:
27 Dec. 2018: added comment about in_waiting vs inWaiting(). Thanks to #FurkanTürkal for pointing that out in the comments below. See documentation here: https://pyserial.readthedocs.io/en/latest/pyserial_api.html#serial.Serial.in_waiting.
27 Oct. 2018: Add sleep to let other threads run.
Documentation: https://docs.python.org/3/library/time.html#time.sleep
Thanks to #RufusV2 for bringing this point up in the comments.
Note on multi-threading:
Even though reading serial data, as shown above, does not require using multiple threads, reading keyboard input in a non-blocking manner does. Therefore, to accomplish non-blocking keyboard input reading, I've written this answer: How to read keyboard input?.
References:
Official pySerial serial.Serial() class API - https://pyserial.readthedocs.io/en/latest/pyserial_api.html
Put it in a separate thread, for example:
import threading
import serial
connected = False
port = 'COM4'
baud = 9600
serial_port = serial.Serial(port, baud, timeout=0)
def handle_data(data):
print(data)
def read_from_port(ser):
while not connected:
#serin = ser.read()
connected = True
while True:
print("test")
reading = ser.readline().decode()
handle_data(reading)
thread = threading.Thread(target=read_from_port, args=(serial_port,))
thread.start()
http://docs.python.org/3/library/threading
I would warn against using blocking IO in a thread. Remember Python has a GIL and at one time only one thread can execute. Now please note that pyserial module is a wrapper over an OS implementation of accessing the serial port. That means it calls code external to the Python. If that code blocks, then the interpreter also get blocked and nothing will execute in the Python program, even the main thread.
This can even happen when using non-blocking IO or timeout based polling if the underlying device driver does not implement timeout well.
A more robust approach is to use multiprocessing module with a queue. Run serial read code in a separate process. This will make sure main and other threads don't block and the program can exit in clean way.
Use a timer driven event to test and read the serial port.
Untested example:
import threading
class serialreading():
def __init__(self):
self.active = True
self.test()
def test(self):
n_in =comport.in_waiting()
if n_in> 0:
self.data = self.data + comport.read(size=n_in)
if len(self.data) > 0:
print(self.data)
self.data=""
if self.active:
threading.Timer(1, test).start() # start new timer of 1 second
def stop(self):
self.active = False
I am using PySerial (a Python API for serial communication) to send AT commands to a Nokia phone via bluetooth.
import serial
com = serial.Serial()
com.port = 19
com.timeout = 0 #also tried a timeout value greater than 0.
try:
com.open()
# at this point I turn off the phone.
com.write("AT\r\n")
print com.readlines()
except SerialException, e:
print e
Just after I open() the com, I turn off the phone. Then, I write("AT\r\n"). At this point, the function blocks and the runtime hangs.
Do you have any solution?
With a timeout set to 0, you desactivate the timeout parameter, the read()/readlines() becomes a blocking call. The caller will be blocked until the device answers.
Try to set a non-zero timeout value to your serial connection com = serial.Serial(timeout=0.5).
If it still hangs, the problem should be in the bluetooth stack.
Actually, what you are looking for is the writeTimeout and not the timeout argument. Late answer, I know, but I still needed it and I guess I'm not the only one.
I'm trying to write a simple Python script for my mobile phone to periodically load a web page using urrlib2. In fact I don't really care about the server response, I'd only like to pass some values in the URL to the PHP. The problem is that Python for S60 uses the old 2.5.4 Python core, which seems to have a memory leak in the urrlib2 module. As I read there's seems to be such problems in every type of network communications as well. This bug have been reported here a couple of years ago, while some workarounds were posted as well. I've tried everything I could find on that page, and with the help of Google, but my phone still runs out of memory after ~70 page loads. Strangely the Garbege Collector does not seem to make any difference either, except making my script much slower. It is said that, that the newer (3.1) core solves this issue, but unfortunately I can't wait a year (or more) for the S60 port to come.
here's how my script looks after adding every little trick I've found:
import urrlib2, httplib, gc
while(true):
url = "http://something.com/foo.php?parameter=" + value
f = urllib2.urlopen(url)
f.read(1)
f.fp._sock.recv=None # hacky avoidance
f.close()
del f
gc.collect()
Any suggestions, how to make it work forever without getting the "cannot allocate memory" error?
Thanks for advance,
cheers, b_m
update:
I've managed to connect 92 times before it ran out of memory, but It's still not good enough.
update2:
Tried the socket method as suggested earlier, this is the second best (wrong) solution so far:
class UpdateSocketThread(threading.Thread):
def run(self):
global data
while 1:
url = "/foo.php?parameter=%d"%data
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('something.com', 80))
s.send('GET '+url+' HTTP/1.0\r\n\r\n')
s.close()
sleep(1)
I tried the little tricks, from above too. The thread closes after ~50 uploads (the phone has 50MB of memory left, obviously the Python shell has not.)
UPDATE:
I think I'm getting closer to the solution! I tried sending multiple data without closing and reopening the socket. This may be the key since this method will only leave one open file descriptor. The problem is:
import socket
s=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.connect(("something.com", 80))
socket.send("test") #returns 4 (sent bytes, which is cool)
socket.send("test") #4
socket.send("test") #4
socket.send("GET /foo.php?parameter=bar HTTP/1.0\r\n\r\n") #returns the number of sent bytes, ok
socket.send("GET /foo.php?parameter=bar HTTP/1.0\r\n\r\n") #returns 0 on the phone, error on Windows7*
socket.send("GET /foo.php?parameter=bar HTTP/1.0\r\n\r\n") #returns 0 on the phone, error on Windows7*
socket.send("test") #returns 0, strange...
*: error message: 10053, software caused connection abort
Why can't I send multiple messages??
Using the test code suggested by your link, I tested my Python installation and confirmed that it indeed leaks. But, if, as #Russell suggested, I put each urlopen in its own process, the OS should clean up the memory leaks. In my tests, memory, unreachable objects and open files all remain more or less constant. I split the code into two files:
connection.py
import cPickle, urllib2
def connectFunction(queryString):
conn = urllib2.urlopen('http://something.com/foo.php?parameter='+str(queryString))
data = conn.read()
outfile = ('sometempfile'. 'wb')
cPickle.dump(data, outfile)
outfile.close()
if __name__ == '__main__':
connectFunction(sys.argv[1])
###launcher.py
import subprocess, cPickle
#code from your link to check the number of unreachable objects
def print_unreachable_len():
# check memory on memory leaks
import gc
gc.set_debug(gc.DEBUG_SAVEALL)
gc.collect()
unreachableL = []
for it in gc.garbage:
unreachableL.append(it)
return len(str(unreachableL))
#my code
if __name__ == '__main__':
print 'Before running a single process:', print_unreachable_len()
return_value_list = []
for i, value in enumerate(values): #where values is a list or a generator containing (or yielding) the parameters to pass to the URL
subprocess.call(['python', 'connection.py', str(value)])
print 'after running', i, 'processes:', print_unreachable_len()
infile = open('sometempfile', 'rb')
return_value_list.append(cPickle.load(infile))
infile.close()
Obviously, this is sequential, so you will only execute a single connection at a time, which may or may not be an issue for you. If it is, you will have to find a non-blocking way of communicating with the processes you're launching, but I'll leave that as an exercise for you.
EDIT: On re-reading your question, it seems you don't care about the server response. In that case, you can get rid of all the pickling related code. And obviously, you won't have the print_unreachable_len() related bits in your final code either.
There exist a reference cycle in urllib2 created in urllib2.py:1216. The issue is on going and exists since 2009.
https://bugs.python.org/issue1208304
I think this is probably your problem. To summarize that thread, there's a memory leak in Pys60's DNS lookup, and you can work around it by moving DNS lookup outside the inner loop.
This seems like a (very!) hacky workaround, but a bit of googling found this comment on the problem:
Apparently adding f.read(1) will stop the leaking!
import urllib2
f = urllib2.urlopen('http://www.google.com')
f.read(1)
f.close()
EDIT: oh, I see you already have f.read(1)... I'm all out of ideas then :/
Consider using the low-level socket API (related howto) instead of urllib2.
HOST = 'daring.cwi.nl' # The remote host
PORT = 50007 # The same port as used by the server
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
s.send('GET /path/to/file/index.html HTTP/1.0\n\n')
# you'll need to figure out how much data to read and read that exactly
# or wait for read() to return data of zero length (I think!)
DATA_SZ = 1024
data = s.recv(DATA_SZ)
s.close()
print 'Received', repr(data)
How to execute and read a HTTP request via low-level sockets is a bit beyond the scope of the question (and perhaps may make a good question on its own on stackoverflow — I searched but didn't see it), but I hope this points you in the direction of a solution that may resolve your problem!
edit An answer in here about using makefile may be helpful: HTTP basic authentication using sockets in python
This does not leak for me with Python 2.6.1 on a Mac. Which version are you using?
BTW, your program doesn't work due to a few typos. Here is one that does work:
import urllib2, httplib, gc
value = "foo"
count = 0
while(True):
url = "http://192.168.1.1/?parameter=" + value
f = urllib2.urlopen(url)
f.read(1)
f.fp._sock.recv=None # hacky avoidance
f.close()
del f
print "count=",count
count += 1
Depending on platform and python version, python might not release memory back to OS. See this stackoverflow thread. That said, python should not endlessly consume memory. Judging from the code you use, it appears to be bug in python runtime unless, urllib/sockets use globals which I don't believe it does - blame it on Python on S60!
Have you considered other sources of memory leakage? Endless log file open, ever increasing array or smth like that? If it truly is a bug in sockets interface, then your only option is to use the subprocess approach.