I am looking for a simple means of triggering my data acquisition software using an external TTL pulse. I need to sample data from multiple sources synchronously with a 5 Hz reference clock. The acquisition does not need real-time priority but I want to ensure that my software is triggered as soon as possible and exactly once per external clock cycle. I would prefer to do this by somehow getting an interrupt from the external trigger without needing to use a fast polling loop. As far as I can tell, you can't just use a parallel port pin for an interrupt in a modern OS like Linux. Any ideas?
I am also thinking of generating broadcast packets on my network to notify other machines on the network that a trigger event has occurred. Due to network latency however there may not be enough time available in the 200ms period between triggers to do the acquisition.
Rather than use the parallel port, have you considered using a serial device? Since you have a TTL signal, you'll possibly need a level converter to convert TTL to RS232 +/- 12V levels. Once you're using a serial device, you can use the standard serial ioctl() calls to detect a change in control signal status.
Specifically, you could use the TIOCMIWAIT ioctl on the connected serial device to wait for a change on say the DCD line, which you would connect to your clock source.
Your user space application would be blocked waiting in the TIOCMIWAIT ioctl system call until there is a change of status on your clock line, at which point your app would become runnable and return from the ioctl. You might have to take care to ensure that you handle the case where you get a change of status interrupt on both rising and falling edges of your serial control signals. On some UART hardware (eg TL16C554A UART) it's possible that you'll only get an interrupt for a signal transitioning in a single direction. For the TL16C554A for example, the TIOCMIWAIT would only fall through on the rising edge of any Ring Indicate signal change.
Using the serial ioctls in this manner also has the advantage that you could use a USB-Serial dongle that supports TIOCMIWAIT if required (eg PL2303), and still retain user level software compatibility, albeit at the expense of increased latency due to USB.
If you require lower latency than can be achieved through user space, you'd be best to write a kernel driver module which can handle the timing and sampling, but I wouldn't suggest this route unless absolutely needed. It's easier to develop user space code.
Here's some incomplete sample C code snippets for using the TIOCMIWAIT ioctl.
int serial_fd = open(cmdline.device_name, O_RDWR | O_NONBLOCK | O_NOCTTY);
static const unsigned int ri_flag = TIOCM_RNG;
/* Set up serial port here using tcsetattr. Set CRTSCTS | CLOCAL to ensure status interrupts
* are generated.
*/
while (1) {
/* Wait for positive RI transition. TIOCMIWAIT takes a mask
* as argument, only returning when the appropriate signal has changed.
*/
if (ioctl(serial_fd, TIOCMIWAIT, ri_flag)) {
fprintf(stderr, "ioctl() failed waiting for RI edge [%s]\n", strerror(errno));
break;
}
/* Do sensor sampling here. You could use TIOCMGET to first verify that
* the clock line is in the expected state, eg high, before continuing.
*/
}
Polling is a fine method for such a slow data rate. Poll at 1 ms. That should be fine. Trying to use a hardware interrupt is going to cause much pain.
Google for "Interrupt Linux GPIO" if you want to do it the hard way. :)
https://developer.ridgerun.com/wiki/index.php/How_to_use_GPIO_signals
Consider connecting the external pulse source to the 'CD' ping of a real (! not a USB to RS232 converter) serial port. Then you can use the "PPS api" to get an as exact timestamp from which the pin "went high" as possible.
You might need a TTL signal shifter; not all serial ports work correctly with TTL signal levels.
The PPS api is normally used for time keeping. E.g. connect the PPS pin of a GPS module to your pc and let NTP sync with that. This gives you microsecond accuracy.
This PPS api is supposed to be more accurate than any other userspace solution (e.g. the TIOCMIWAIT ioctl) as it is completely handled in the kernel, immediately when the interrupt (as triggered by the CD signal change) comes in. With the ioctl solution you have at least a context switch. I did some testing on a raspberry pi and a userspace solution gives at least 6us jitter.
The PPS api gives you a timestamp from when the pulse was detected.
Why not use a USB to RS232 converter: I read somewhere (in relation to timekeeping) that USB devices are polled once every +/- 1ms. This polling frequency also depends on how busy the system is with other USB devices and I think the internal clock of the USB device may also influence things. I've never measured this though.
Relevant URLS:
https://www.rfc-editor.org/rfc/rfc2783 RFC describing the PPS API
http://linuxpps.org/ Linux implementation the PPS API (it is in the standard kernel)
http://comments.gmane.org/gmane.comp.hardware.gps.gpsd.user/3747 a thread which mentions why not to use USB for timekeeping
Regarding the network functionality: use UDP broadcasts, do not use TCP.
I ended up using the serial port CTS line for the trigger using the TIOCMIWAIT ioctl per Austin Phillips answer. Since RS232 requires +/-12V levels I was able to get the necessary power for this level shifter from the other serial control lines.
The Python code to implement this solution can be found in question: Python monitor serial port (RS-232) handshake signals
Related
I'm developing a program in Python that uses UDP to receive data from an FPGA (a data collector device). The speed is very high, about 54 MB/s at the highest setting, that's why we use a dedicated gigabit ethernet connection. My problem is: a lot of packages get lost. This is not a momentary problem, the packets come in for a long time, then there's a few seconds long pause, then everything seems fine again. The pause depends on the speed (faster communication, more lost).
I've tried setting buffers higher, but something seems to be missing. I've set self.sock_data.setsockopt(socket.SOL_SOCKET,socket.SO_RCVBUF,2**28) to increase buffer size along with the matching kernel option: sysctl -w net.core.rmem_max=268435456.
Packages have an internal counter, so I know which one got lost (also, I use this to fix their order). An example: 11s of data lost, around 357168 packages. (I've checked, and it's not a multiple of an internal buffer size in either of my program or the FPGA's firmware). I'm watching the socket on a separate thread, and immediately put them into a Queue to save everything.
What else should I set or check?
I'm having trouble carrying out what I think should be a pretty straightforward task on a NIDAQ usb6002: I have a low frequency sine wave that I'm measuring at an analog input channel, and when it crosses zero I would like to light an LED for 1 second. I'm trying to use the nidaqmx Python API, but haven't been able to clear up some of my basic questions with the documentation. https://nidaqmx-python.readthedocs.io/en/latest/
If anyone can offer any thoughts about the code or the basic logic of my setup, that would be hugely helpful.
Here's what I have tried so far. I start with some imports and the definition of my channels:
import matplotlib.pyplot as plt
from math import *
import nidaqmx
from nidaqmx import *
from nidaqmx.constants import *
import time
V_PIN = "Dev1/ai6"
LED_PIN = "Dev1/ao0"
I understand how tasks and things work generally- I can read and plot a signal of a given sampling rate and number of samples using task.ai_channels methods without any trouble. But here's my best guess at how to carry out "detect zero and trigger output":
writeLED = nidaqmx.Task('LED')
writeLED.ao_channels.add_ao_voltage_chan(LED_PIN)
writeLED.timing.cfg_samp_clk_timing(1)
writeLED.triggers.start_trigger.cfg_anlg_edge_start_trig(V_PIN,trigger_level = 0)
writeLED.write([5], auto_start=True)
This gives me the error below at the cfg_anlg_edge line
DaqError: Requested value is not a supported value for this property. The property value may be invalid because it conflicts with another property.
Property: DAQmx_StartTrig_Type
Requested Value: DAQmx_Val_AnlgEdge
Possible Values: DAQmx_Val_DigEdge, DAQmx_Val_None
I don't know why an analog input channel wouldn't be supported here. Page 245 of this document makes it sound like it should be: https://media.readthedocs.org/pdf/nidaqmx-python/latest/nidaqmx-python.pdf
I'm sure there are other problems with the code, too. For example, it seems like the sample clock manipulations are quite a bit more complicated than what I've written above, but I haven't been able to find anything that explains how it would work in this situation.
Thanks in advance for any help!
With NI, it's "RTFMs"
When programming NI devices, you usually need two manuals.
NI-DAQmx Help (for the programming part)
the device specification (for the device part)
You need both because the NI-DAQmx API supports every DAQ device NI makes, but not every device has the same capabilities. "Capabilities" includes more than how many channels of each kind, but also the timing and triggering subsystems as well as internal signal routing. A DAQmx application that runs with one device is not guaranteed to run with another because the application might use the API in a way the second device cannot support.
Finally, on the documentation front, any given NI DAQ device typically belongs to family of related devices and these families also have a manual called User Guide. These User Guides act as a bridge between the API and device spec, helping you understand how the device responds to commands. For the 6002, the family is "Low-Cost DAQ USB Device".
Analog trigger for analog output on NI 6002
Your determination is correct that
writeLED.triggers.start_trigger.cfg_anlg_edge_start_trig(V_PIN,trigger_level = 0)
is possible, just not for the USB 6002. This line is asking the analog output subsystem to use an analog edge trigger, but the analog output subsystem for the 6002 only has these trigger capabilities:
software
PFI 0
PFI 1
For this device, you're only option is the software trigger because the PFI lines are digital triggers and their trigger level is specified to be between 0.8 V and 2.3 V.
Change your Python program to detect a zero-crossing from the analog input stream and, when it does, make it call stop() and then start() on the AO task.
The reason for the stop-start sequence is retriggering: you want to light the LED for each zero crossing, but a task cannot be restarted unless it has either been stopped (by the API or by completing its task) or configured for retriggering. Because the 6002 is in the low-cost family, this hardware feature isn't available, so you must use the API to stop the AO task or wait for the AO generation to complete before restarting the pulse for the LED
6002 AO Specification
Software triggering is not real-time, you will have non-deterministic delay before the led turns on. This depends on your program, interfaces, usb latencies, pc performances...
Otherwise, you can use a comparator (like lm393) to trigger a digital input (PFI0 or PFI1).
Though it's just an LED, it is probably not critical if the delay varies within milliseconds.
If this is a stupid question, please don't mind me. But I spent some time trying to find the answer but I couldn't get anything solid. Maybe this is a hardware question, but I figured I'd try here first.
Does Serial Communication only work one to one? The reason this came up is because I had an arduino board listening for communication on its serial port. I had a python script feed bytes to the port as well. However, whenever I opened up the arduino's serial monitor, the connection with the python script failed. The serial monitor also connects to the serial port for communication for its little text input field.
So what's the deal? Does serial communication only work between a single client and a single server? Is there a way to get multiple clients writing to the server? I appreciate your suggestions.
Multiple clients (e.g. Arduinos) communicating with one server (e.g. a desktop computer) is commonly done with the serial variant:
RS-485
This is a simple method widely used in industrial settings where you want to have many devices connected to one computer via one serial port. This type of arrangement is also called multi-drop, because one cable strings around a building with Tees that tap in and drop lines to each device.
The hardware for this is widely available. You can buy USB serial adapters that provide the hardware interface for a computer. Programmatically the port looks just like an RS232 port. For the Arduino you would just add a transceiver chip. A sea of serial transceivers exists, e.g.
Example computer USB adapter with 485 interface
Sample RS485 transceiver chip from Element14
All the devices hang on the same bus listening at the same time. A simple communication protocol used is just add a device address before every command. For example:
001SETLIGHT1 <- tells Arduino "001" to turn on the light
013SETLIGHT0 <- tells "013" to turn off the light
Any device hanging on the cable ignores commands that do not start with their address. When a device responds, it prepends its address.
001SETLIGHT1DONE <- response from device "001" that the command has been received and executed
The address in the response lets the receiving party know which device was talking.
Well, your question can be quite wide, so I'm going to layer my answer:
On the hardware side, the same pair of wires can work be shared with many devices. It is mostly a question of electronics (maintaining the signal in the good voltage range), and not having all devices writing to the serial port at the same time (or you'll get wreckage).
On the software side, on the host, yes you can share the same serial connection to a device with multiple processes. But that's not straight forward. I'll assume you're using an unix (macos or linux):
in unix, everything is a file, your serial connection is one too: /dev/ttyACM0 on linux, for example.
When you have a process opening that file, it will block it (using ioctl, iirc) so no other process can mess with that file too.
Then, you can input and output to that file using the process that opened it, that's all.
But hopefully, it is still possible to share the connection between processes. One of them would simply be to use the tee command, that will be able to get input from one process, and give it back output, and copy the output to another process. You can also do it from within python, by duplicating the file descriptor.
To easily output stuff that can be redirected the unix way (using pipes), you can use socat: http://www.dest-unreach.org/socat/
here's an usage example:
socat -,raw,echo=0,escape=0x0f /dev/ttyACM0,raw,echo=0,crnl
you may want to tweak it for your needs.
Edit:
I forgot about RS-485, which 'jdr5ca' was smart enough to recommend. My explanation below is restricted to RS-232, the more "garden variety" serial port. As 'jdr5ca' points out, RS-485 is a much better alternative for the described problem.
Original:
To expand on zmo's answer a bit, it is possible to share serial at the hardware level, and it has been done before, but it is rarely done in practice.
Likewise, at the software driver level, it is again theoretically possible to share, but you run into similar problems as the hardware level, i.e. how to "share" the link to prevent collisions, etc.
A "typical" setup would be two serial (hardware) devices attached to each other 1:1. Each would run a single software process that would manage sending/receiving data on the link.
If it is desired to share the serial link amongst multiple processes (on either side), the software process that manages the link would also need to manage passing the received data to each reading process (keeping track of which data each process had read) and also arbitrate which sending process gets access to the link during "writes".
If there are multiple read/write processes on each end of the link, the handshaking/coordination of all this gets deep as some sort of meta-signaling arrangement may be needed to coordinate the comms between the process on each end.
Either a real mess or a fun challenge, depending on your needs and how you view such things.
I am reading data from a microcontroller via serial, at a baudrate of 921600. I'm reading a large amount of ASCII csv data, and since it comes in so fast, the buffer get's filled and all the rest of the data gets lost before I can read it. I know I could manually edit the pyserial source code for serialwin32 to increase the buffer size, but I was wondering if there is another way around it?
I can only estimate the amount of data I will receive, but it is somewhere around 200kB of data.
Have you considered reading from the serial interface in a separate thread that is running prior to sending the command to uC to send the data?
This would remove some of the delay after the write command and starting the read. There are other SO users who have had success with this method, granted they weren't having buffer overruns.
If this isn't clear let me know and I can throw something together to show this.
EDIT
Thinking about it a bit more, if you're trying to read from the buffer and write it out to the file system even the standalone thread might not save you. To minimize the processing time you might consider reading say 100 bytes at a time serial.Read(size=100) and pushing that data into a Queue to process it all after the transfer has completed
Pseudo Code Example
def thread_main_loop(myserialobj, data_queue):
data_queue.put_no_wait(myserialobj.Read(size=100))
def process_queue_when_done(data_queue):
while(1):
if len(data_queue) > 0:
poped_data = data_queue.get_no_wait()
# Process the data as needed
else:
break;
There's a "Receive Buffer" slider that's accessible from the com port's Properties Page in Device Manager. It is found by following the Advanced button on the "Port Settings" tab.
More info:
http://support.microsoft.com/kb/131016 under heading Receive Buffer
http://tldp.org/HOWTO/Serial-HOWTO-4.html under heading Interrupts
Try knocking it down a notch or two.
You do not need to manually change pyserial code.
If you run your code on Windows platform, you simply need to add a line in your code
ser.set_buffer_size(rx_size = 12800, tx_size = 12800)
Where 12800 is an arbitrary number I chose. You can make receiving(rx) and transmitting(tx) buffer as big as 2147483647a
See also:
https://docs.python.org/3/library/ctypes.html
https://msdn.microsoft.com/en-us/library/system.io.ports.serialport.readbuffersize(v=vs.110).aspx
You might be able to setup the serial port from the DLL
// Setup serial
mySerialPort.BaudRate = 9600;
mySerialPort.PortName = comPort;
mySerialPort.Parity = Parity.None;
mySerialPort.StopBits = StopBits.One;
mySerialPort.DataBits = 8;
mySerialPort.Handshake = Handshake.None;
mySerialPort.RtsEnable = true;
mySerialPort.ReadBufferSize = 32768;
Property Value
Type: System.Int32
The buffer size, in bytes. The default value is 4096; the maximum value is that of a positive int, or 2147483647
And then open and use it in Python
I am somewhat surprised that nobody has yet mentioned the correct solution to such problems (when available), which is effective flow control through either software (XON/XOFF) or hardware flow control between the microcontroller and its sink. The issue is well described by this web article.
It may be that the source device doesn't honour such protocols, in which case you are stuck with a series of solutions that delegate the problem upwards to where more resources are available (move it from the UART buffer to the driver and upwards towards your application code). If you are losing data, it would certainly seem sensible to try and implement a lower data rate if that's a possibility.
For me the problem was it was overloading the buffer when receiving data from the Arduino.
All I had to do was mySerialPort.flushInput() and it worked.
I don't know why mySerialPort.flush() didn't work. flush() must only flush the outgoing data?
All I know is mySerialPort.flushInput() solved my problems.
I have a GSM modem that disconnect after a while, maybe because of low signal. I am just wondering is there an AT command that can detect the disconnection and re-establish a reconnection.
Is there a way in code (preferably python) I can detect the disconnection and re-establish a reconnection?
Gath
Depending on what type of connection, circuit switched (CS) or packet switched (PS), the monitoring will be a little bit different. To detect a disconnect you can enable UR (unsolicited result) code AT+CPSB=1 to monitor PDP context activity (aka packet switched connections). For circuit switched calls you can monitor with the +CIEV: UR code enabled with AT+CMER=3,0,0,2.
To re-establish the connection you have to set up the connection again. For CS you will either have to know the phone number dialed, or you can use the special form of ATD, ATDL [1] which will dial the last dialed number. You can use ATDL for PS as well if the call was started with ATD (i.e. "ATD*99*....") which is quite common, but I do not think there is any way if started with AT+CGDATA for instance.
However, none of the above related to ATD matters, because it is not what you want. For CS you might set up a call from your python script, but then so what? After receiving CONNECT all the data traffic would be coming on the serial connection that your python script are using. And for PS the connection will not even finish successfully unless the phone receives PPP traffic from the PC as part of connection establishment. Do you intend your python script to supply that?
What you really want is to trigger your PC to try to connect again, whether this is standard operating system dial up networking or some special application launching it. So monitor the modem with a python script and then take appropriate action on the PC side to re-establish the connection.
[1]
Side note to ATDL: notice that if you want to repeat the last voice call you should still terminate with a semicolon, i.e. ATDL;, otherwise you would start a data call.
Here is how I do it with Telit devices:
I use AT+CGREG=1 to subscribe to unsolicited messages. Extract from documentation:
+CGREG - GPRS Network Registration Status
AT+CGREG=[<n>]
Set command controls the presentation of an unsolicited result code
+CGREG: (see format below).
Parameter:
<n> - result code presentation mode
0 - disable network registration unsolicited result code
1 - enable network registration unsolicited result code; if there is a change in the terminal GPRS network registration status, it is issued the unsolicited result code:
+CGREG: <stat>
And I wait on the modem's serial line for +CGREG messages. When something comes I check to see if stat is 1 (connected to the home network) or 5 (connected in roaming).
NOTE: A different response +CGREG comes when issuing the AT+CGREG? which is not hard to isolate.
You can try to check the signal strength on a regular basis with AT+CSQ. If the signal goes under a given threshold consider that you are disconnected and force a new connection.
You can try the very nice pyserial http://pyserial.sourceforge.net/ Python library to send the AT commands to the modem.
I hope it helps