Develop Reference

How to stop a server in python?

I am trying to live-stream using a server and the code after serve_forever() is not running. I am not able to stop live-streaming without using the keyboard interrupt. I need to use python code for stopping the live-stream after a certain time, closing the server and closing the raspberry pi camera. Any help would be appreciated.
try:
address = ('',8000)
server = StreamingServer(address, StreamingHandler)
server.serve_forever()
finally:
camera.stop_recording()

The normal method is to call server.shutdown(). Simply it cannot be called from the request handler. Long story short: call it from a different thread.

How to tell when client is done sending image

I am creating a file server in python using ftp sockets where multiple clients send images to a server. This is the method I am using to receive the image on the server side:
while True:
data = client.recv(512)
if not data:
break
file.write(data)
And this is the method I am using to send the image to the server:
while True:
data = file.read(512)
if not data:
break
server.send(data)
The problem I am running into is that on the server side the while loop is never exited which means the code is either stuck in the recv call or the if statement is never true. On the client side there are no problems, the loop is exited properly. I've heard that the client side will send something to the server to tell it to stop but the if statement doesn't seem to pick it up. How can I get the server to stop trying to receive without closing the connection?

https://docs.python.org/2/howto/sockets.html#disconnecting
On the client, close the socket to the server. On the server, check whether the recv returned 0 bytes.
Also from the documentation for using a socket:
When a recv returns 0 bytes, it means the other side has closed (or is
in the process of closing) the connection. You will not receive any
more data on this connection. Ever. You may be able to send data
successfully; I’ll talk more about this later.

Data will never be nothing unless the client closes the connection.
server.close()

Python Sockets Select is hanging - Doing other tasks while waiting for socket data?

I am rather a noob here, but trying to setup a script where I can poll a socket, and when no socket data has been sent, a loop continues to run and do other things. I have been playing with several examples I found using select(), but no matter how I organize the code, it seems to stop on or near the server.recv() line and wait for a response. I want to skip out of this if no data has been sent by a client, or if no client connection exists.
Note that this application does not require the server script to send any reply data, if it makes any difference.
The actual application is to run a loop and animate some LEDs (which needs root access to the I/O on a Raspberry Pi). I am going to send this script data from another separate script via sockets that will pass in control parameters for the animations. This way the external script does not require root access.
So far the sending and receiving of data works great, I just can't get loop to keep spinning in the absence of incoming data. It is my understanding that this is what select() was intended to allow, but the examples I've found don't seem to be working that way.
I have attempted adding server.setblocking(0) a few different places to no avail. (If I understand correctly a non-blocking instance should allow the code to skip over the recv() if no data has been sent, but I may be off on this).
I have based my code on an example here:
http://ilab.cs.byu.edu/python/select/echoserver.html
Here is the server side script followed by the client side script.
Server Code: sockselectserver.py
#!/usr/bin/env python
import select
import socket
import sys
server = socket.socket()
host = socket.gethostname()
port = 20568
size = 1024
server.bind((host,port))
server.listen(5)
input = [server,sys.stdin]
running = 1
while running:
inputready,outputready,exceptready = select.select(input,[],[])
for s in inputready:
if s == server:
# handle the server socket
client, address = server.accept()
input.append(client)
elif s == sys.stdin:
# handle standard input
junk = sys.stdin.readline()
running = 0
else:
# handle all other sockets
data = s.recv(size)
if data:
s.send(data)
else:
s.close()
input.remove(s)
print "looping"
server.close()
Client Code: skclient.py
#!/usr/bin/python # This is client.py file
import socket # Import socket module
s = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 20568 # Reserve a port for your service.
s.connect((host, port))
data = "123:120:230:51:210:120:55:12:35:24"
s.send(data)
print s.recv(1024)
s.close # Close the socket when done
What I would like to achieve by this example is to see "looping" repeated forever, then when the client script sends data, see that data print, then see the "looping" resume printing over and over. That would tell me it's doing what is intended I can take it from there.
Interesting enough, when I test this as is, whenever I run the client, I see "looping" printed 3 times on the screen, then no more. I don't fully understand what is happening inside the select, but I'd assume it would only print 1 time.
I tried moving the inputready.. select.select() around to different places but found it appears to need to be called each time, otherwise the server stops responding (for example if it is called once prior to the endless while: loop).
I'm hoping this can be made simple enough that it can be taught to other hacker types in a maker class, so I'm hopeful I don't need to get too crazy with multi-threading and more elaborate solutions. As a last resort I'm considering logging all my parameters to mySQL from the external script then using this script to query them back out of tables. I've got experience there and would probably work, but it seems this socket angle would be a more direct solution.
Any help very much appreciated.

Great news. This was an easy fix, wanted to post in case anyone else needed it. The suggestion from acw1668 above got me going.
Simply added a timeout of "0" to the select.select() like this:
inputready,outputready,exceptready = select.select(input,[],[],0)
This is in the python docs but somehow I missed it. Link here: https://docs.python.org/2/library/select.html
Per the docs:
The optional timeout argument specifies a time-out as a floating point number in seconds. When the timeout argument is omitted the function blocks until at least one file descriptor is ready. A time-out value of zero specifies a poll and never blocks.
I tested the same code as above, adding a delay of 5 seconds using time.sleep(5) right after the print "looping" line. With the delay, if no data or client is present the code just loops every 5 seconds and prints "looping" to the screen. If I kick off the client script during the 5 second delay, it pauses and the message is processed the next time the 5 second delay ends. Occasionally it doesn't respond the very next loop, but rather the loop following. I assume this is because the first time through the server.accept is running and the next time through the s.recv() is running which actually exchanges the data.

Can't resolve URL until program is restarted

I've got a python script that basically looks something like this:
#############################
# MAIN LOOP
while True:
client_socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
client_socket.connect((url, socketnum))
packet = somedata
client_socket.sendall(packet)
except Exception as e:
# an error occurred
logging.error("An error occurred: {}".format(e))
pass
finally:
logging.info("Closing socket...")
client_socket.close()
time.sleep(70)
What I find is that if this script is run before an internet connection is established on the computer (an embedded Linux system), naturally, when the socket tries to connect, I get "Errno -3 Temporary failure in name resolution". However, if the internet connection is then established, the program STILL cannot resolve the hostname - the only way to get it to work is to restart the python script.
Since this system is not one where I can guarantee the presence of an internet connection at all times, is there anyway to get python to realise that the internet connection now exists and that name resolution information is now available?
EDIT: Some further testing shows that this only happens if the python program is started before any successful internet connection is established on the machine after a boot up. If the python program is started AFTER an internet connection has previously been established on the machine (even if it's subsequently been disconnected), the program operates correctly and will successfully connect to the internet after internet connectivity is restored.
So:
Bootup->Python started->Internet connection established = program doesn't work
Bootup->Internet connection established->Internet disconnected->Python started = program works fine.

Try flushing DNS cache in every iteration.
import os
...
while True:
os.popen('nscd -I hosts',"r")
...
or try service nscd restart command instead.

Python Socket Client Disappears, Server Can Not Tell

I'm going crazy writing a little socket server in python. Everything was working fine, but I noticed that in the case where the client just disappears, the server can't tell. I simulate this by pulling the ethernet cable between the client and server, close the client, then plug the cable back in. The server never hears that the client disconnected and will wait forever, never allowing more clients to connect.
I figured I'd solve this by adding a timeout to the read loop so that it would try and read every 10 seconds. I thought maybe if it tried to read from the socket it would notice the client was missing. But then I realized there really is no way for the server to know that.
So I added a heartbeat. If the server goes 10 seconds without reading, it will send data to the client. However, even this is successful (meaning doesn't throw any kind of exception). So I am able to both read and write to a client that isn't there any more. Is there any way to know that the client is gone without implementing some kind of challenge/response protocol between the client and server? That would be a breaking change in this case and I'd like to avoid it.
Here is the core of my code for this:
def _loop(self):
command = ""
while True:
socket, address = self._listen_socket.accept()
self._socket = socket
self._socket.settimeout(10)
socket.sendall("Welcome\r\n\r\n")
while True:
try:
data = socket.recv(1)
except timeout: # Went 10 seconds without data
pass
except Exception as e: # Likely the client closed the connection
break
if data:
command = command + data
if data == "\n" or data == "\r":
if len(command.strip()) > 0:
self._parse_command(command.strip(), socket)
command = ""
if data == '\x08':
command = command[:-2]
else: # Timeout on read
try:
self._socket.sendall("event,heartbeat\r\n") # Send heartbeat
except:
self._socket.close()
break
The sendall for the heartbeat never throws an exception and the recv only throws a timeout (or another exception if the client properly closes the connection under normal circumstances).
Any ideas? Am I wrong that sending to a client that doesn't ACK should generate an exception eventually (I've tested for several minutes).

The behavior you are observing is the expected behavior for a TCP socket connection. In particular, in general the TCP stack has no way of knowing that an ethernet cable has been pulled or that the (now physically disconnected) remote client program has shut down; all it knows is that it has stopped receiving acknowledgement packets from the remote peer, and for all it knows the packets could just be getting dropped by an overloaded router somewhere and the issue will resolve itself momentarily. Given that, it does what TCP always does when its packets don't get acknowledged: it reduces its transmission rate and its number-of-packets-in-flight limit, and retransmits the unacknowledged packets in the hope that they will get through this time.
Assuming the server's socket has outgoing data pending, the TCP stack will eventually (i.e. after a few minutes) decide that no data has gone through for a long-enough time, and unilaterally close the connection. So if you're okay with a problem-detection time of a few minutes, the easiest way to avoid the zombie-connection problem is simply to be sure to periodically send a bit of heartbeat data over the TCP connection, as you described. When the TCP stack tries (and repeatedly fails) to get the outgoing data sent-and-acknowledged, that is what eventually will trigger it to close the connection.
If you want something quicker than that, you'll need to implement your own challenge/response system with timeouts (either over the TCP socket, or over a separate TCP socket, or over UDP), but note that in doing so you are likely to suffer from false positives yourself (e.g. you might end up severing a TCP connection that was not actually dead but only suffering from a temporary condition of lost packets due to congestion). Whether or not that's a worthwhile tradeoff depends on what sort of program you are writing. (Note also that UDP has its own issues, particularly if you want your system to work across firewalls, etc)