Develop Reference

PySerial : can't write until I disconnect STLink USB cable

I'm working on some software, which involves communication between PC and embedded device over UART. I'm using STLink's USB/Serial adapter.
Most of the time communication works fine, but after some time (can be hours/minutes or even days) I got write timeout which I can't solve from software level - the only solution that works is to disconnect USB from Nucleo's st-link and then connect it again. I tried to reset or reprogramm embedded device, rerun serial script and nothing seems to work. I tried to implement exception handling which looks like this now (but doesnt work):
self.ser.reset_output_buffer()
try:
self.ser.write(bytes(response))
except Exception as e:
print(e)
print("write timeout occured")
while self.ser.inWaiting()>0:
self.ser.read(1)
self.ser.reset_output_buffer()
self.ser.flushOutput()
self.ser.flushInput()
self.ser.close()
self.ser.open()
serial port configuration is:
def createSerialConnection():
if os.name == "nt":
return serial.Serial('COM4', 38400, timeout=1, rtscts=0,
xonxoff=0,dsrdtr=0, write_timeout=1, parity=serial.PARITY_EVEN)
so... the two devices (PC and embedded) are exchanging 5 messages every second, package size is constant (196 bytes) and before last error I've got 500k packages exchanged without any issue (I've got both higher and lower numbers before). Communication protocol is pretty simple, master(embedded) sends data and waits for response. If response isn't send back in 5seconds it retransmits last package and that's it.
I'd kill for some information on where should I look for the issue. How can I avoid it? Is it pyserial issue (then what should I use, what would be stable enough)? Maybe stlink's (from Nucleo 144) uart/usb is broken?
Thanks for any help!

Streaming data to clients

I have a program that sniffs network data and stores it in a database using pcapy (based on this). I need to make the data available in realtime over a network connection.
Right now when i run the program it will start a second thread for the sniffer and a Twisted server on the main thread, however i have no idea how to get clients to 'tap into' the sniffer that's running in the background.
The end result should be that a client enters an url and the connection will be kept open until the client disconnects (even when there's nothing to send), whenever the server has network activity the sniffer will sniff it and send it to the clients.
I'm a beginner with Python so i'm quite overwhelmed so if anyone could point me in the right direction it would be greatly appreciated.

Without more information (a simple code sample that doesn't work as you expect, perhaps) it's tough to give a thorough answer.
However, here are two pointers which may help you:
Twisted Pair, a (unfortunately very rudimentary and poorly documented) low-level/raw sockets networking library within Twisted itself, which may be able to implement the packet capture directly in a Twisted-friendly way, or
The recently-released Crochet, which will allow you to manage the background Twisted thread and its interactions with your pcapy-based capture code.

pyusb: function works well once, the second time fails

I have a raspberry pi (I'm using linux) and a device that measures some values.
I have a python server and when a client sends a "I want the values" command, I ask the device some values. This is the function for my device:
def usbDevice:
dev = usb.core.find(idVendor=0xXXXX, idProduct=0xXXXX)
if dev is None:
return 'Not found'
dev.set_configuration()
dev.write(1, '\x05', 0) # ask for the values
time.sleep(2)
ret = dev.read(0x81,2048 , 0,1200) # endpoint, siz, interf, timeout ->meas. vals
print ret
return ret
When I start the communication it always goes well (both ways, I send the command and I get the
proper response ...).However, the second time I try it (no matter if I wait too long or not) I'm getting no response from the device. I know I can send the request (I used a print "step XX" to see where it is failing), but I get no answer... I am usually getting a timeout exception, but sometimes getting a "resource busy" exception.
Should I close the usb communication after using it the first time? How? Despite, I'm always compiling with sudo permissions, I created a .rules file, so I think there's not the problem.
Apart from this, I would really like to know if something's wrong with this way to establish the USB communication as I used something similar in the past. ( and I'm not good with python...)

Can you provide additional information such as the linux distribution and kernel number e.g.
uname -a
There are known USB issues. The good news is that a recent fix may have resolved them. Basically, use rpi-update and it should update the kernel (you want at least #389).

Trouble with a Python server

I have a few test clients that are encountering the same issue each time. The clients can connect, and they can send their first message, but after that the server stops responding to that client. I suspect that the problem is related to s.accept(), but I'm not sure exactly what is wrong or how to work around it.
def startServer():
host = ''
port = 13572
backlog = 5
size = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((host,port))
s.listen(backlog)
print "Close the command prompt to stop Gamelink"
while 1:
try:
client, address = s.accept()
data = client.recv(size)
if data:
processData(data)
client.send("OK")
else:
print "Disconnecting from client at client's request"
client.close()
except socket.error, (value, message):
if s:
print "Disconnecting from client, socket issue"
s.close()
print "Error opening socket: " + message
break
except:
print "Gamelink encountered a problem"
break
print "End of loop"
client.close()
s.close()
The server is intended to be accessed across a local network, and it needs to be light weight and very quick to respond, so if another implementation (such as thread based) would be better for meeting those requirements please let me know. The intended application is to be used as a remote gaming keyboard, thus the need for low resource use and high speed.

Writing a server using socket directly will be hard. As Keith says, you need to multiplex the connections somehow, like with select or poll or threads or fork. You might think you need only one connection, but what will you do when something hiccups and the connection is lost? Will your server be able to respond to reconnection attempts from the client if it hasn't yet realized the connection is lost?
If your networking needs are basic, you might be able to let something else handle all the listening and accepting and forking stuff for you. You don't specify a platform, but examples of such programs are launchd on Mac OS and xinetd on Linux. The details differ between these tools, but basically you configure them, in some configuration file, to listen for a connection on some port. When they get it, they take care of setting up the connection, then they exec() your program with stdin and stdout aimed at the socket, so you can simply use all the basic IO you probably already know like print and sys.stdin.read().
The trouble with solutions like xinitd and launchd is that for each new connection, they must fork() and exec() a new instance of your program. These are relatively heavy operations so a large number of connections or a high rate of new connections might hit the limits of your server. But worse, since each connection is in a separate process, sharing data between them is hard. Also, most solutions you might find to communicate between processes involve a blocking API, and now you are back to the problem of multiplexing with select or threads or similar.
If that doesn't meet your needs, I think you are better off learning to use a higher-level networking framework which will handle all the problems you will inevitably encounter if you go down the path of socket. One such framework I'd suggest is Twisted. Beyond handling the mundane details of handling connections, and the more complex task of multiplexing IO between them, you will also have a huge library of tools that will make implementing your protocol much easier.

What are the functions / AT commands to reconnect a disconnected GSM modem?

I have a GSM modem that disconnect after a while, maybe because of low signal. I am just wondering is there an AT command that can detect the disconnection and re-establish a reconnection.
Is there a way in code (preferably python) I can detect the disconnection and re-establish a reconnection?
Gath

Depending on what type of connection, circuit switched (CS) or packet switched (PS), the monitoring will be a little bit different. To detect a disconnect you can enable UR (unsolicited result) code AT+CPSB=1 to monitor PDP context activity (aka packet switched connections). For circuit switched calls you can monitor with the +CIEV: UR code enabled with AT+CMER=3,0,0,2.
To re-establish the connection you have to set up the connection again. For CS you will either have to know the phone number dialed, or you can use the special form of ATD, ATDL [1] which will dial the last dialed number. You can use ATDL for PS as well if the call was started with ATD (i.e. "ATD*99*....") which is quite common, but I do not think there is any way if started with AT+CGDATA for instance.
However, none of the above related to ATD matters, because it is not what you want. For CS you might set up a call from your python script, but then so what? After receiving CONNECT all the data traffic would be coming on the serial connection that your python script are using. And for PS the connection will not even finish successfully unless the phone receives PPP traffic from the PC as part of connection establishment. Do you intend your python script to supply that?
What you really want is to trigger your PC to try to connect again, whether this is standard operating system dial up networking or some special application launching it. So monitor the modem with a python script and then take appropriate action on the PC side to re-establish the connection.
[1]
Side note to ATDL: notice that if you want to repeat the last voice call you should still terminate with a semicolon, i.e. ATDL;, otherwise you would start a data call.

Here is how I do it with Telit devices:
I use AT+CGREG=1 to subscribe to unsolicited messages. Extract from documentation:
+CGREG - GPRS Network Registration Status
AT+CGREG=[<n>]
Set command controls the presentation of an unsolicited result code
+CGREG: (see format below).
Parameter:
<n> - result code presentation mode
0 - disable network registration unsolicited result code
1 - enable network registration unsolicited result code; if there is a change in the terminal GPRS network registration status, it is issued the unsolicited result code:
+CGREG: <stat>
And I wait on the modem's serial line for +CGREG messages. When something comes I check to see if stat is 1 (connected to the home network) or 5 (connected in roaming).
NOTE: A different response +CGREG comes when issuing the AT+CGREG? which is not hard to isolate.

You can try to check the signal strength on a regular basis with AT+CSQ. If the signal goes under a given threshold consider that you are disconnected and force a new connection.
You can try the very nice pyserial http://pyserial.sourceforge.net/ Python library to send the AT commands to the modem.
I hope it helps