I have a PIR-sensor and a controller hooked up to my USB, and I can read in the Terminal the controllers output if there is any motion using serial.readline().
But what I really want to do is just check if there is any activity on the port (meaning there is motion detected).
Somthing like;
if(serial.readline() == true)
//do something
or maybe:
if(serial.readline() != null)
//do something
Can someone help me out with this, or at least point me in the right direction?
Given your use case, an option would be to perform non-blocking read by creating the connection with a time-out of 0 (see http://pyserial.sourceforge.net/pyserial_api.html)
my_serial = serial.Serial(..., timeout = 0)
That way, you will have non-blocking read.
That being said, by doing so, you might end up by regularly pooling the serial port for incoming data. Depending the application this might be acceptable or not.
Depending your OS, with non-blocking serial connection, you might probably use select or poll to check for data availability.
Related
I'll have a loop that keep refreshing a HTML file.When data arrives at the serial port that I'll choose through serial library. The data is coming from an Arduino board, and when the data arrives I'll read it and refresh some variables. I searched a little and see that Interrupts are usually used to that, but I'm thinking how to implement such interruption.
Interrupts are usually used for very time critical things that can happen any time and which have therefor be executed with minimum delay.
I don't think what you want to do is a good use case for interrupts.
Why don't you try something like this: (pseudocode)
repeat forever:
read serial port
if data received:
update variables (call a function that does that)
You can apply pretty much any example on processing serial commands out there. Just search the web.
It's always the same, read the serial input buffer frequently and check the received data. When the data is what you expect, do something. Then continue listening for further data.
The Arduino core has a built-in pseudo-interrupt function, serialEvent, that kinda does what you want. It only checks in between each loop():
https://www.arduino.cc/en/Tutorial/SerialEvent
But #Piglet's answer is probably the best solution—check if serial data is waiting for you, and if so, respond to it.
if (Serial.available()) {
// do something
}
I'm building a live radio streamer, and I was wondering how I should handle multiple connections. Now from my experience select will block the audio from being streamed. It only plays 3 seconds then stops playing. I will provide an example of what I mean.
import socket, select
headers = """
HTTP/1.0 200 OK\n
Content-Type: audio/mpeg\n
Connection: keep-alive\n
\n\n
"""
file="/path/to/file.mp3"
bufsize=4096 # actually have no idea what this should be but python-shout uses this amount
sock = socket.socket()
cons = list()
buf = 0
nbuf = 0
def runMe():
cons.append(sock)
file = open(file)
nbuf = file.read(bufsize) # current buffer
while True:
buf = nbuf
nbuf = file.read(bufsize)
if len(buf) == 0:
break
rl, wl, xl = select.select(cons, [], [], 0.2)
for s in rl:
if s == sock:
con, addr = s.accept()
con.setblocking(0)
cons.append(con)
con.send(header)
else:
data = s.recv(1024)
if not data:
s.close()
cons.remove(s)
else:
s.send(buf)
That is an example of how i'd use select. But, the song will not play all the way. But if I send outside the select loop it'll play but it'll die on a 2nd connection. Should I use threading?
That is an example of how i'd use select. But, the song will not play
all the way. But if I send outside the select loop it'll play but
it'll die on a 2nd connection. Should I use threading?
You can do it either way, but if your select-implementation isn't working properly it's because your code is incorrect, not because a select-based implementation isn't capable of doing the job -- and I don't think a multithreaded solution will be easier to get right than a select-based solution.
Regardless of which implementation you choose, one issue you're going to have to think about is timing/throughput. Do you want your program to send out the audio data at approximately the same rate it is meant to be played back, or do you want to send out audio data as fast as the client is willing to read it, and leave it up to the client to read the data at the appropriate speed? Keep in mind that each TCP stream's send-rate will be different, depending on how fast the client chooses to recv() the information, as well as on how well the network path between your server and the client performs.
The next problem to deal with after that is the problem of a slow client -- what do you want your program to do when one of the TCP connections is very slow, e.g. due to network congestion? Right now your code just blindly calls send() on all sockets without checking the return value, which (given that the sockets are non-blocking) means that if a given socket's output-buffer is full, then some (probably many) bytes of the file will simply get dropped -- maybe that is okay for your purpose, I don't know. Will the clients be able to make use of an mp3 data stream that has arbitrary sections missing? I imagine that the person running that client will hear glitches, at best.
Implementation issues aside, if it was me I'd prefer the single-threaded/select() approach, simply because it will be easier to test and validate. Either approach is going to take some doing to get right, but with a single thread, your program's behavior is much more deterministic -- either it works right or it doesn't, and running a given test will generally give the same result each time (assuming consistent network conditions). In a multithreaded program, OTOH, the scheduling of the threads is non-deterministic, which makes it very easy to end up with a program that works correctly 99.99% of the time and then seriously malfunctions, but only once in a blue moon -- a situation that can be very difficult to debug, as you end up spending hours or days just reproducing the fault, let alone diagnosing and fixing it.
I am busy developing a Python system that uses web-sockets to send/received data from a serial port.
For this to work I need to react to data from the serial port as it is received. Problem is to detect incoming data the serial port needs to queried continuously looking for incoming data. Most likely a continuous loop. From previous experiences(Slow disk access + heavy traffic) using Flask this sounds like it could cause the web-sockets to be blocked. Will this be the case or is there a work around?
I have looked at how NodeJS interact with serial ports and it seems much nicer. It raises an event when there is incoming data instead of querying it all the time. Is this an option in Python?
Extra Details:
For now it will only be run on Linux.(Raspbian)
Flask was my first selection but I am open to other Python Frameworks.
pyserial for serial connection.(Is the only option I know of)
Python provides the select module in the stdlib which can do what you want. It DOES depend on what operating system you are using though. So since you haven't provided that information I can't be that helpful. However a simple example under Linux would be:
import select
epoll = select.epoll()
# Do stuff to create serial connection and websocket connection
epoll.register(websocket_file_descriptor, select.EPOLLIN)
epoll.register(serial_file_descriptor, select.EPOLLIN)
while True:
events = epoll.poll(1)
# Do stuff with the event,
for fileno, event in events:
if fileno == serial_file_descriptor:
data = os.read(serial_file_descriptor)
os.write(websocket_file_descriptor, data)
elif fileno == websocket_file_descriptor:
data = os.read(websocket_file_descriptor)
# Do something with the incoming data
That's a basic, incomplete, example. But it should give you an idea of the general process of using a system like epoll.
Simply start a subprocess that listens to the serial socket and raises an event when it has a message. Have a separate sub-process for each web port that does the same.
I would like to send a message to a subset of clients connected to a server which uses Python epoll. Is there a practical way of getting list of fileno's of connected clients?
Normally, the way you handle this is to keep track of the clients yourself. Every time you accept a new client, before adding it to the epoll, also add it to your collection. Every time you drop a client, remove it from your collection.
The typical minimal thing to store is a dict mapping sockets to whatever else you need to keep track of. You can use the socket itself, a weakref to it, or its fileno as the key. The data you need often includes things like a read buffer and a write buffer, the peer address (as received on accept), a name or user ID or auth cookie, etc.
And I'm not sure how you're doing anything useful with a polling server that doesn't have at least a read buffer attached to each client socket. What do you do when you get a partial message? (For that matter, if you're planning to send a message to these clients, and you don't have a write buffer, how are you going to do that?)
But to answer your specific question:
There is no way to get a list of fd's from a Python epoll object, because there is no way to do this with the underlying API.
Conceivably, you could write something that steps through everything from 0 to your max open fd and tries to do an epoll_ctl with it and distinguish based on the error. This is probably a very bad idea, but it may not be impossible. You'd probably have to ctypes down to the native function, and you may need to play around with different possibilities to find out what has the right effect. For example, maybe if you do epoll_ctl(my_epoll.fileno(), EPOLL_CTL_MOD, fd, NULL), you'll get ENOENT for a unregistered fd, EBADF for a nonexistent fd, but EINVAL (because of the NULL event) for a valid fd. Or maybe it'll segfault. I can't guarantee there's any combination of parameters that will distinguish, but with some trial and error, you might find one.
(By the way, there's nothing that says that an epoll has to be a list of connected clients; e.g., it may be the connected clients plus the listener socket plus a "quit" pipe.)
I am reading data from a microcontroller via serial, at a baudrate of 921600. I'm reading a large amount of ASCII csv data, and since it comes in so fast, the buffer get's filled and all the rest of the data gets lost before I can read it. I know I could manually edit the pyserial source code for serialwin32 to increase the buffer size, but I was wondering if there is another way around it?
I can only estimate the amount of data I will receive, but it is somewhere around 200kB of data.
Have you considered reading from the serial interface in a separate thread that is running prior to sending the command to uC to send the data?
This would remove some of the delay after the write command and starting the read. There are other SO users who have had success with this method, granted they weren't having buffer overruns.
If this isn't clear let me know and I can throw something together to show this.
EDIT
Thinking about it a bit more, if you're trying to read from the buffer and write it out to the file system even the standalone thread might not save you. To minimize the processing time you might consider reading say 100 bytes at a time serial.Read(size=100) and pushing that data into a Queue to process it all after the transfer has completed
Pseudo Code Example
def thread_main_loop(myserialobj, data_queue):
data_queue.put_no_wait(myserialobj.Read(size=100))
def process_queue_when_done(data_queue):
while(1):
if len(data_queue) > 0:
poped_data = data_queue.get_no_wait()
# Process the data as needed
else:
break;
There's a "Receive Buffer" slider that's accessible from the com port's Properties Page in Device Manager. It is found by following the Advanced button on the "Port Settings" tab.
More info:
http://support.microsoft.com/kb/131016 under heading Receive Buffer
http://tldp.org/HOWTO/Serial-HOWTO-4.html under heading Interrupts
Try knocking it down a notch or two.
You do not need to manually change pyserial code.
If you run your code on Windows platform, you simply need to add a line in your code
ser.set_buffer_size(rx_size = 12800, tx_size = 12800)
Where 12800 is an arbitrary number I chose. You can make receiving(rx) and transmitting(tx) buffer as big as 2147483647a
See also:
https://docs.python.org/3/library/ctypes.html
https://msdn.microsoft.com/en-us/library/system.io.ports.serialport.readbuffersize(v=vs.110).aspx
You might be able to setup the serial port from the DLL
// Setup serial
mySerialPort.BaudRate = 9600;
mySerialPort.PortName = comPort;
mySerialPort.Parity = Parity.None;
mySerialPort.StopBits = StopBits.One;
mySerialPort.DataBits = 8;
mySerialPort.Handshake = Handshake.None;
mySerialPort.RtsEnable = true;
mySerialPort.ReadBufferSize = 32768;
Property Value
Type: System.Int32
The buffer size, in bytes. The default value is 4096; the maximum value is that of a positive int, or 2147483647
And then open and use it in Python
I am somewhat surprised that nobody has yet mentioned the correct solution to such problems (when available), which is effective flow control through either software (XON/XOFF) or hardware flow control between the microcontroller and its sink. The issue is well described by this web article.
It may be that the source device doesn't honour such protocols, in which case you are stuck with a series of solutions that delegate the problem upwards to where more resources are available (move it from the UART buffer to the driver and upwards towards your application code). If you are losing data, it would certainly seem sensible to try and implement a lower data rate if that's a possibility.
For me the problem was it was overloading the buffer when receiving data from the Arduino.
All I had to do was mySerialPort.flushInput() and it worked.
I don't know why mySerialPort.flush() didn't work. flush() must only flush the outgoing data?
All I know is mySerialPort.flushInput() solved my problems.