Develop Reference

Interrupt a pySerial readline in another thread

I am using pySerial to communicate to a microcontroller over USB, Most of the communication is initiated by the desktop python script which sends a command packet and waits for reply.
But there is also a alert packet that may be sent by the microcontroller without a command from the python script. In this case I need to monitor the read stream for any alerts.
For handling alerts, I dedicate a seperate process to call readline() and loop around it, like so:
def serialMonitor(self):
while not self.stopMonitor:
self.lock.acquire()
message = self.stream.readline()
self.lock.release()
self.callback(message)
inside a class. The function is then started in a seperate process by
self.monitor = multiprocessing.Process(target = SerialManager.serialMonitor, args = [self])
Whenever a command packet is send, the command function needs to take back control of the stream, for which it must interrupt the readline() call which is in blocking. How do I interrupt the readline() call? Is there any way to terminate a process safely?

You can terminate a multiprocessing process with .terminate(). Is this safe? Probably it's alright for a readline case.
However, this is not how I would handle things here. As I read your scenario, there are two possibilities:
MCU initiates alert package
Computer sends data to MCU (and MCU perhaps responds)
I assume the MCU will not send an alert package whilst an exchange is going on initiated by the computer.
So I would just initiate the serial object with a small timeout, and leave it in a loop when I'm not using it. My overall flow would go like this:
ser = Serial(PORT, timeout=1)
response = None
command_to_send = None
running = True
while running: # event loop
while running and not command_to_send and not line:
try:
line = ser.readline()
except SerialTimeoutException:
pass
if not command_to_send:
process_mcu_alert(line)
else:
send_command(command_to_send)
command_to_send = None
response = ser.readline()
This is only a sketch, as it would need to be run in a thread or subprocess, since readline() is indeed blocking, so you need some thread-safe way of setting command_to_send and running (used to exit gracefully) and getting response, and you likely want to wrap all this state up in a class. The precise implementation of that depends upon what you are doing, but the principle is the same---have one loop which handles reading and writing to the serial port, have it timeout to respond relatively quickly (you can set a smaller timeout if you need to), and have it expose some interface you can handle.
Sadly to my knowledge python has no asyncio compatible serial library, otherwise that approach would seem neater.

Python Sockets - How to shut down the server?

I tried to make a simple chat system with the socket module in Python. everything works, except, that i need to kill the process everytime when i want to shutdown the server. And i don't want to do this everytime.
So my question is:
How can i make a function, that when i type shutdown in the server terminal, it shutdowns the whole server?
I already tried to do this:
def close(self):
server.close(self)
server.shutdown(self)
But it doesn't work. When i type close(), nothing happens. Nothing.
Heres the full code of the server.py:
https://pastebin.com/gA4QYmQe
Every help is useful. Thanks.

Well... there are many problems with the code (your "MY IP" and "SERVERIP" are probably not what you want, but this is beside the point.
Your close() function has a "self" parameter, which is pointless as this is not a class. You also need to move the close function to the beginning of your code if you want to call it from your try-except -structure. You need to call shutdown() first and then close(), and shutdown takes an argument. I modified your close() to do this and it works.
def close():
server.shutdown(socket.SHUT_RDWR)
server.close()
print ("closed")
When you open your socket, you should also set SO_REUSEADDR to make the address reusable (meaning you can start the server again if you shut it down, instead of waiting for a minute for TIME_WAIT status to finish with your server port):
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
But how exactly are you calling close() when you type "shutdown" somewhere? You are not doing this. Your program is in the socket loop and is not reading keyboard input.
I see no point whatsoever adding keyboard input to this program. First, it adds complexity as you are operating with two possibly blocking inputs (socket input and keyboard input) and you would need to manage this. It is possible but definitely complicated. Second, it is faster to press Control + C instead of typing "shutdown" and hitting enter.
You currently do not call close after a keyboard interrupt. I added this to the inner KeyboardInterrupt (the outer you can remove - it is not doing anything and is never reached) and it now shuts down your program neatly, closing all connections. Remember to move close() function from the bottom of your code to the front before the try: statement:
except KeyboardInterrupt:
print("[!] Keyboard Interrupted!")
close()
break
If you want a remote shutdown (server shuts down if you type "shutdown" to the socket), you can add this to your server loop:
if message == "shutdown":
close()
exit(0)
There are other problems as well. For example, if you start your server, connect to it and shut down the connection, your server exits as it does not return to listen().
This is also (in my opinion), somewhat bad programming, as you use "server" as a global variable. I would rather create a class and put all socket operations in it, but if style is not important, this should work.
Hannu

One other approach would be having a default admin-client that can control the server. Admin-client will be created when the server starts and from that client admin can shutdown and do any of the admin tasks on the server.

How to assign minimum time wait to connect socket

I will send data to server if it is connected. Otherwise it will continue main program. I am using python 2.7. It takes longer time to continue main program if server is not connected.
server=socket.socket()
host="192.168.0.1"
port=12321
try:
server.connect(host,port)
server.send('data')
except:
print"server vot connected"
if the server computer is not power on or the server program is not running. it takes longer time to print server not connected. I want to assign 3 ms to try to connect server if it could not connect it will exit and print server not connected. how to assign the time to wait and get rid of the problem to hang for 1-2 minutes?
any kind of help will be highly appriciated.

The timeout is a property of the socket:
server=socket.socket()
server.settimeout(0.003)
host="192.168.0.1"
port=12321
try:
server.connect(host,port)
server.send('data')
except:
print"server vot connected"

How to launch mock HTTP server in separate thread Python?

I am trying to mock an HTTP server in my python script, but it fails.
Here is what I am doing:
import bottle
from restclient import GET
from threading import Thread
#bottle.route("/go")
def index():
return "ok"
server = Thread(target = bottle.run)
server.setDaemon(True)
server.start()
print "Server started..."
response = GET("http://127.0.0.1:8080/go")
assert response.body == "ok"
print "Done..."
Basically I am trying to launch bottle.py http server with 1 test route in a separate thread & then mock responses from it.
But it just won't work. The server is not getting started in a separate thread, so I always get "errno 111 connection refused" when trying to request it.
So the question is: how can it be solved? Is there any other ways to mock http servers?

You're not leaving enough time for the webserver to start up.
When you do:
server.start()
print "Server started..."
response = GET("http://127.0.0.1:8080/go")
You try to access the server just after starting it,
Depending on which thread (the main one, or the server one) gets to run first (and for how long), you might end up in a situation when the server hasn't started yet when you try to access it, hence the Connection Refused error.
You could try doing the following :
server.start()
import time
time.sleep(...) # Something long enough
# Continue your stuff.
As you can see in time.sleep -- sleeps thread or process?, doing time.sleep will only sleep the currently running thread, so you could leave enough time to your server thread to start.
Now, all that is a bit hackish, so you might want to look into your server startup process to see if there is a way to assess whether it's up and running and wait on this condition before starting up.
Looking at the bottle source now, I can't figure out a solution to do that cleanly; you could always try to hit the server repeatdly until it finally responds, thus indicating the server is alive.

python xinetd client disconnection handling

This may or may not being a coding issue. It may also be an xinetd deamon issue, i do not know.
I have a python script which is triggered from a linux server running xinetd. Xinetd has been setup to only allow one instance as I only want one machine to be able to connect to the service, which is therefore also limited by IP.
Currently when the client connects to xinetd the service works correctly and the script begins sending its output to the client machine. However, when the client disconnects (i.e: due to reboot), the process is still alive on the server, and this blocks the ability for the client to connect once its finished rebooting or so on.
Q: How can i detect in python that the client has disconnected. Perhaps i can test if stdout is no longer being read from by the client (and then exit the script), or is there a much eaiser way in xinetd to have the child process be killed when the client disconnects ?
(I'm using python 2.4.3 on RHEL5 linux - solutions for 2.4 are needed, but 3.1 solutions would be useful to know also.)

Add a signal handler for SIGHUP. (x)inetd sends this upon the socket disconnecting.

Monitor the signals sent to your proccess. Maybe your script isn't responding to the SIGHUP sent by xinet, monitor the signal and let it die.

You don't seem to get a SIGHUP, but you do get a SIGPIPE, at least so long as you are attempting any IO on the connection. If the application spends long periods of time not doing any IO, then you could just start a thread reading stdin to ensure you get the SIGPIPE as soon as the disconnection occurs. This was good enough for my application but then I didn't use any pipes other than the ones xinetd gave me.
I've seen several places on the net where people talk about the SIGHUP getting sent on client disconnection, so I've written an inetd python script to test out a couple of servers (one inetd and another xinetd), so you could use that to check on the signals getting sent. It just logs what it finds to /var/log/test.log. Perhaps it will be useful.
#!/usr/bin/python
import os, signal, sys
skip = ["SIGKILL", "SIG_DFL", "SIGSTOP", "SIG_IGN", "SIGCLD", "SIGCHLD"]
name_map = {}
identifiers = [i for i in dir(signal) if i.startswith("SIG") and not i in skip]
for i in identifiers:
name_map[getattr(signal, i)] = i
def handler(num, frame):
signame = name_map[num]
os.system("echo handled %s >> /var/log/test.log" % signame)
if __name__ == "__main__":
for id, name in name_map.iteritems():
signal.signal(id, handler)
while True:
print sys.stdin.readline()
sys.stdout.flush()