I was looking at the socket programming module of the python standard library and I noticed a fucntion socket.setblocking. The documentation mentioned that setting a socket to non blocking mode would mean that an error would be raised if the data was not sent out through the socket immediately or if data was not available upon trying to read from the socket.
I'm having trouble understanding usecases in which this function might be useful. I'm working on a Linux machine(just in case the answer to this would be OS dependent).
Thanks!
When you set the socket to blocking, the socket waits for the specified time on that socket. While it is waiting on the socket, your program cannot do anything. At the end of the wait time it raises an error. Sometimes you dont want blocking to occur.
A good use case for this might be when you are sending an receiving message on a single threaded program using multiple sockets. You don't want to block on a socket while waiting to send or receive messages rather you may want to check if there are messages to send or receive for each of the sockets hence you would want no blocking time or limited blocking time while you loop through the sockets. This will provide a more in depth discussion of python sockets.
Related
I'm starting to learn more about TCP protocols in Python and I've been having some trouble with blocking threads inside clients.
Ideally, my application would work like this: I have different clients with thread functions, each one of them containing an input function in order to receive a specific command to send to the server (for example 'X'). When the 'X' is tapped in ONE client, the server receives it and sends a message to all the other clients informing that the program will continue and releasing them from their input functions - almost like cancelling them.
The problem lies on the fact that the input functions are blocking the clients from leaving the loop. I've tried setting the input thread functions as daemon but it blocks until you tap something anyway - which is unfortunately the only workaround that I've found so far.
I would like to use socket and the select module for connection, without being attached to any particular OS (so no msvcrt that works on Windows or the select module to monitor the stdin, which is only available in UNIX based OS).
Any help would be greatly appreciated!
Im trying to make a tcp communication, where the server sends a message every x seconds through a socket, and should stop sending those messages on a certain condition where the client isnt sending any message for 5 seconds.
To be more detailed, the client also sends constant messages which are all ignored by the server on the same socket as above, and can stop sending them at any unknown time. The messages are, for simplicity, used as alive messages to inform the server that the communication is still relevant.
The problem is that if i want to send repeated messages from the server, i cannot allow it to "get busy" and start receiving messages instead, thus i cannot detect when a new messages arrives from the other side and act accordingly.
The problem is independent of the programming language, but to be more specific im using python, and cannot access the code of the client.
Is there any option of receiving and sending messages on a single socket simultaneously?
Thanks!
Option 1
Use two threads, one will write to the socket and the second will read from it.
This works since sockets are full-duplex (allow bi-directional simultaneous access).
Option 2
Use a single thread that manages all keep alives using select.epoll. This way one thread can handle multiple clients. Remember though, that if this isn't the only thread that uses the sockets, you might need to handle thread safety on your own
As discussed in another answer, threads are one common approach. The other approach is to use an event loop and nonblocking I/O. Recent versions of Python (I think starting at 3.4) include a package called asyncio that supports this.
You can call the create_connection method on an event_loop to create an asyncio connection. See this example for a simple server that reads and writes over TCP.
In many cases an event loop can permit higher performance than threads, but it has the disadvantage of requiring most or all of your code to be aware of the event model.
I am currently working on a server + client combo on python and I'm using TCP sockets. From networking classes I know, that TCP connection should be closed step by step, first one side sends the signal, that it wants to close the connection and waits for confirmation, then the other side does the same. After that, socket can be safely closed.
I've seen in python documentation function socket.shutdown(flag), but I don't see how it could be used in this standard method, theoretical of closing TCP socket. As far as I know, it just blocks either reading, writing or both.
What is the best, most correct way to close TCP socket in python? Are there standard functions for closing signals or do I need to implement them myself?
shutdown is useful when you have to signal the remote client that no more data is being sent. You can specify in the shutdown() parameter which half-channel you want to close.
Most commonly, you want to close the TX half-channel, by calling shutdown(1). In TCP level, it sends a FIN packet, and the remote end will receive 0 bytes if blocking on read(), but the remote end can still send data back, because the RX half-channel is still open.
Some application protocols use this to signal the end of the message. Some other protocols find the EOM based on data itself. For example, in an interactive protocol (where messages are exchanged many times) there may be no opportunity, or need, to close a half-channel.
In HTTP, shutdown(1) is one method that a client can use to signal that a HTTP request is complete. But the HTTP protocol itself embeds data that allows to detect where a request ends, so multiple-request HTTP connections are still possible.
I don't think that calling shutdown() before close() is always necessary, unless you need to explicitly close a half-channel. If you want to cease all communication, close() does that too. Calling shutdown() and forgetting to call close() is worse because the file descriptor resources are not freed.
From Wikipedia: "On SVR4 systems use of close() may discard data. The use of shutdown() or SO_LINGER may be required on these systems to guarantee delivery of all data." This means that, if you have outstanding data in the output buffer, a close() could discard this data immediately on a SVR4 system. Linux, BSD and BSD-based systems like Apple are not SVR4 and will try to send the output buffer in full after close(). I am not sure if any major commercial UNIX is still SVR4 these days.
Again using HTTP as an example, an HTTP client running on SVR4 would not lose data using close() because it will keep the connection open after request to get the response. An HTTP server under SVR would have to be more careful, calling shutdown(2) before close() after sending the whole response, because the response would be partly in the output buffer.
According to the python documentation which says:
Strictly speaking, you’re supposed to use shutdown on a socket before
you close it. The shutdown is an advisory to the socket at the other
end. Depending on the argument you pass it, it can mean “I’m not going
to send anymore, but I’ll still listen”, or “I’m not listening, good
riddance!”. Most socket libraries, however, are so used to programmers
neglecting to use this piece of etiquette that normally a close is the
same as shutdown(); close(). So in most situations, an explicit
shutdown is not needed.
I think the most correct way to close a TCP connection would be to use shutdown before closing a connection, because close is not atomic! This can make some bugs. Suppose you're using close function without shutdown and the data didn't send to the server correctly, at the same time python closes the connection and server can't reply to client, now the socket at the other end may hang indefinitely.
I have a c program that connects to a python server, sends a short string (less than about 100chars) and then closes socket. It does this at a periodic rate.
The python server accepts connection, spawns a thread, and in that thread calls:
data = sock.recv(4096)
data often turns out to be empty.
After reading through the python man pages, and some of the stack overflow posts (thanks guys!), I realize that the problem is the c program that opens, writes, closes, sockets so quickly, that by the time the python server accepts, spawns thread, the recv() returns no data, as documented.
The problem is, I don't know a workaround to this problem? I have very little control over the c-program. Is where a way to tell python to buffer the message for recv() even if the other side closes connection?
(caveat: I haven't verified my hunch yet on wireshark, but the logs in both programs strongly indicate the c-program closes before recv() is even called for most of the time.)
thanks.
I tried googling this, couldn't find an answer, searched here, couldn't find an answer. Has anyone looked into whether it's thread safe to write to a Serial() object (pyserial) from thread a and do blocking reads from thread b?
I know how to use thread synchronization primitives and thread-safe data structures, and in fact my current form of this program has a thread dedicated to reading/writing on the serial port and I use thread-safe data structures to coordinate activities in the app.
My app would benefit greatly if I could write to the serial port from the main thread (and never read from it), and read from the serial port using blocking reads in the second thread (and never write to it). If someone really wants me to go into why this would benefit the app I can add my reasons. In my mind there would be just one instance of Serial() and even while thread B sits in a blocking read on the Serial object, thread A would be safe to use write methods on the Serial object.
Anyone know whether the Serial class can be used this way?
EDIT: It occurs to me that the answer may be platform-dependent. If you have any experience with a platform like this, it'd be good to know which platform you were working on.
EDIT: There's only been one response but if anyone else has tried this, please leave a response with your experience.
I have done this with pyserial. Reading from one thread and writing from another should not cause problems in general, since there isn't really any kind of resource arbitration problem. Serial ports are full duplex, so reading and writing can happen completely independently and at the same time.
I've used pyserial in this way on Linux (and Windows), no problems !
I would recommend to modify Thread B from "blocking read" to "non blocking read/write". Thread B would become your serial port "Daemon".
Thread A could run at full speed for a friendly user interface or perform any real time operation.
Thread A would write a message to Thread B instead of trying to write directly to the serial port. If the size/frequency of the messages is low, a simple shared buffer for the message itself and a flag to indicate that a new message is present would work. If you need higher performance, you should use a stack. This is actually implemented simply using an array large enough to accumulate many message to be sent and two pointers. The write pointer is updated only by Thread A. The read pointer is updated only by Thread B.
Thread B would grab the message and sent it to the serial port. The serial port should use the timeout feature so that the read serial port function release the CPU, allowing you to poll the shared buffer and, if any new message is present, send it to the serial port. I would use a sleep at that point to limit the CPU time used by Thread B.. Then, you can make Thread B loop to the read serial port function. If the serial port timeout is not working right, like if the USB-RS232 cable get unplugged, the sleep function will make the difference between a good Python code versus the not so good one.