Develop Reference

Reading synchronised data from 2 serial USB ports concurrently in Python. Threading? Multiprocessing?

I'm building an autonomous rover controlled by a Raspberry Pi using RTK GPS navigation. I am using two separate U-Blox RTK GPS modules on the rover operating at 4Hz to get very accurate position and heading calculations and they are both connected to the Pi through USB ports. I'm using the pyserial library to read the NMAE sentences as they are output through USB. The problem I have encountered is that the serial data sent by each module is precisely synchronised and so while calling Serial.readline() for one port, the data sent to the other port is missed.
The basics of the code currently looks something like this:
`class GPS():
def __init__(self,port,baud):
self.gpsSerial = serial.Serial(port,baud)
# init some variables
return
def read(self):
NMEASentance = str(self.gpsSerial.readline())
if NMEASentance not what_i_wanted_or_blank:
return 0
# do some processing, update vars
return 1
frontGps = GPS(port1,baud)
rearGps = GPS(port2,baud)
while True:
if frontGps.read():
# grab the new data
if rearGps.read():
# grab the new data
What ends up happening is that while one serial port is being read and the data processed, the other is transmitting simultaneously and is missed. I have tried doing both reads one after the other and leaving the processing for later, but the problem persists.
How do i read both ports simultaneously? Will threading each serial read work (they will both still be running in the same process, and so won't be executed concurrently, right?)
If the solution is multiprocessing then could someone please post some skeleton code for me to follow because this problem is really doing my head in.
Thanks for any help!

Would love to get some code on this:
Set up two processes to read from the serial ports and add them to a shared queue
I have a GPS serial loop running and need to add a second serial link for a magnetometer and needed a way to keep the two streams separate
Currently, working with a Neo6M GPS and noticed that the #GPRMC message is like a heartbeat that is transmitted once a second on each UTC second tick.
The #GPRMC is the GPS standard sentence that transmits UTC time/date
(and location, speed and heading) so it makes sense that it would be transmitted at UTC second = 00.00
After reading about the problem it occurred to me that all the GPS receivers are probably doing the same. So if you have any number of them powered up, they will all be synchronized.
If my theory is correct:
Armies of GPS-enabled pi soldier robots could be made to walk in step even though they are continents apart.

Remotely sending inputs to a running Python script

I'll try to be as clear as possible with what I'm trying to aim for.
I have a running Python script on my Raspberry Pi and I'd like multiple users to send inputs to the script remotely (through SSH or anything else that might work better).
So for example if I have this script running:
Name = input ("Please type in your name. \n")
type (Name)
print ("Hello there" , Name)
time.sleep(3) # Pause for 3 seconds.
I want users to send names to this script remotely from devices that are connected to the same network as the Raspberry Pi.
If possible, I also want to implement the following functionalities:
Sending the output (aka the printed text) back to the specific device the input came from.
A queuing system: If multiple users send names at the same time, the script will take the names in order, one by one.
I know it's a lot to ask for, but I'd really appreciate if someone could help me get started with this by pointing me in the right direction. I've searched around quite a bit for the past few days but I haven't really come across anything that fits my needs.
Edit: I'm running this on PYTHON 3

Your comment that you would like to communicate (via network) to the script directly, opens up a world of possibilities. You have to modify your Python script a little though, because it won't communicate via stdin/stdout any longer.
I'm still not entirely sure how you want things to work but it does sound to me that a solution based around RPC can possibly work for you. May I suggest you have a look at Pyro4? Basically what that does for you is enable you to do normal Python method calls, but over the network, to code running on another computer.
So you can set up a server on your Pi (that needs to run continuously) which accepts remote calls from other computers, and can then call into your python code on the pi. It can process calls in parallel or in sequence. You didn't say if you need any form of security, but some basic security features are provided (no built-in encryption or communication over TLS yet, sorry).
A simple example is here and lots more are on github so you can have a look to see if this fits your requirements?
Another solution that doesn't require third party libraries is perhaps to write a WSGI http server that calls your script, run this on the pi, and access it via HTTP from your other computers.

Serial communication over USB converter in Python - how to aproach this?

A python program needs to accept a string every second from a serial port. I plan on using a RS-232 to USB converter. The application is going to work under Ubuntu 10.04.
How do I approach this? Do I use pySerial or libusb?
There needs to be done some processing in the meantime, so synchronous communication is not viable. Do I use some kind of interrupts or do I need to open separate threads? Or do I use blocking reads, believing that 1s is enough for my computations (it is plenty ... for now)?
I know, RTFM, but heading in the correct direction from the start will save me a lot of time! Thanks for bearing with me.

If your RS232-USB converter has a driver in Ubuntu that makes it look like a COM port then you will want to use pySerial (the interface is the same as any other COM port). If there is no driver for your device then you might have to use libusb and find the protocol for your specific device. Most major RS232-USB converters these days have usbserial based drivers committed and maintained in the Linux kernel. Simply check with your vendor for this.
There are many ways to do parallel processing but typically I write my applications in two ways:
Have a read thread that does nothing but read and fill a local thread safe buffer so data is ready for other threads when needed.
Have a read thread that reads data, determines where it goes and delivers it via messaging/event processing to the component that needs it.
The decisions here will depend on what your goals are, and how much processing is needed outside the reading.

Serial Communication one to one

If this is a stupid question, please don't mind me. But I spent some time trying to find the answer but I couldn't get anything solid. Maybe this is a hardware question, but I figured I'd try here first.
Does Serial Communication only work one to one? The reason this came up is because I had an arduino board listening for communication on its serial port. I had a python script feed bytes to the port as well. However, whenever I opened up the arduino's serial monitor, the connection with the python script failed. The serial monitor also connects to the serial port for communication for its little text input field.
So what's the deal? Does serial communication only work between a single client and a single server? Is there a way to get multiple clients writing to the server? I appreciate your suggestions.

Multiple clients (e.g. Arduinos) communicating with one server (e.g. a desktop computer) is commonly done with the serial variant:
RS-485
This is a simple method widely used in industrial settings where you want to have many devices connected to one computer via one serial port. This type of arrangement is also called multi-drop, because one cable strings around a building with Tees that tap in and drop lines to each device.
The hardware for this is widely available. You can buy USB serial adapters that provide the hardware interface for a computer. Programmatically the port looks just like an RS232 port. For the Arduino you would just add a transceiver chip. A sea of serial transceivers exists, e.g.
Example computer USB adapter with 485 interface
Sample RS485 transceiver chip from Element14
All the devices hang on the same bus listening at the same time. A simple communication protocol used is just add a device address before every command. For example:
001SETLIGHT1 <- tells Arduino "001" to turn on the light
013SETLIGHT0 <- tells "013" to turn off the light
Any device hanging on the cable ignores commands that do not start with their address. When a device responds, it prepends its address.
001SETLIGHT1DONE <- response from device "001" that the command has been received and executed
The address in the response lets the receiving party know which device was talking.

Well, your question can be quite wide, so I'm going to layer my answer:
On the hardware side, the same pair of wires can work be shared with many devices. It is mostly a question of electronics (maintaining the signal in the good voltage range), and not having all devices writing to the serial port at the same time (or you'll get wreckage).
On the software side, on the host, yes you can share the same serial connection to a device with multiple processes. But that's not straight forward. I'll assume you're using an unix (macos or linux):
in unix, everything is a file, your serial connection is one too: /dev/ttyACM0 on linux, for example.
When you have a process opening that file, it will block it (using ioctl, iirc) so no other process can mess with that file too.
Then, you can input and output to that file using the process that opened it, that's all.
But hopefully, it is still possible to share the connection between processes. One of them would simply be to use the tee command, that will be able to get input from one process, and give it back output, and copy the output to another process. You can also do it from within python, by duplicating the file descriptor.
To easily output stuff that can be redirected the unix way (using pipes), you can use socat: http://www.dest-unreach.org/socat/
here's an usage example:
socat -,raw,echo=0,escape=0x0f /dev/ttyACM0,raw,echo=0,crnl
you may want to tweak it for your needs.

Edit:
I forgot about RS-485, which 'jdr5ca' was smart enough to recommend. My explanation below is restricted to RS-232, the more "garden variety" serial port. As 'jdr5ca' points out, RS-485 is a much better alternative for the described problem.
Original:
To expand on zmo's answer a bit, it is possible to share serial at the hardware level, and it has been done before, but it is rarely done in practice.
Likewise, at the software driver level, it is again theoretically possible to share, but you run into similar problems as the hardware level, i.e. how to "share" the link to prevent collisions, etc.
A "typical" setup would be two serial (hardware) devices attached to each other 1:1. Each would run a single software process that would manage sending/receiving data on the link.
If it is desired to share the serial link amongst multiple processes (on either side), the software process that manages the link would also need to manage passing the received data to each reading process (keeping track of which data each process had read) and also arbitrate which sending process gets access to the link during "writes".
If there are multiple read/write processes on each end of the link, the handshaking/coordination of all this gets deep as some sort of meta-signaling arrangement may be needed to coordinate the comms between the process on each end.
Either a real mess or a fun challenge, depending on your needs and how you view such things.

How to make screensaver accept other credentials?

Under Windows XP I have seen some commercial software that protects the computer with a USB device. That is, the screensaver activates after a certain period of time as usual. But to deactivate, you need not only a passphrase, but also a USB device plugged in. The device contains certificates and has to be verified before deactivating the screensaver.
I am looking for some way to implement such feature with Python. I have searched in the Ubuntu Software Center and got BlueProximity being the software most close to my purpose but still different. This software monitors a certain Bluetooth device and its presence is used to simulate user activities periodically to avoid screensaver's activation.
Surely I can do a similar program, periodically check a certain USB disk's presence and validate its containing certificate, and if all-OK, poke the screensaver as some user activity, otherwise lock the screen.
However this is not immediate. Suppose someone have stolen my passphrase to unlock the screensaver, but not the USB disk, then he can unlock the screen. And within at least one minute or so the screen should be locked again. Even if my program has a rather short checking period--like 0.1 second, intervals summing from the 0.1s and the slowly fading time cost(usually nearly 1s) exists between one and another lock-up.
So is there any better solution, such as some APIs that my program can tell screensaver refuse to unlock any way?

You might want to take a look to PAM (Pluggable Authentication Modules). The solution will be more generic, robust and it could be applied to any program that relies on PAM for authentication.