Develop Reference

How to use `select.select` effectively when client and server inside same file to send and receive from echo server?

Suppose we have an echo server and a file transfer app.
There is a sender (Client) to send files and a receiver (Server) to receive files. The echo server will echo whatever received from the Client and the Server.
However, Client and Server cannot communicate directly, i.e., all packets have to go through the echo server. For example, the Client sends a UDP packet to the Echo server, and the Echo server echo that packet to Server and Server send Acknowledgment to the Echo server, and the Echo server echo that ack packet to the Client.
The objective is to implement a reliable UDP for file transfer. And we have only one UDP socket.
This figure demonstrates what the setup is
Client, Server and Echo Server
I have tried to use multi-thread and select.select and both do not work perfectly
The issue with multi-thread is that since Client and Server cannot communicate internally, and we have only one socket, it is difficult to choose who should send or receive now.
The issue with select.select is that the return list always has writeable non-empty, which makes the Client continues to send a bunch of packets before the readable is ready.
Here is the implementation for both Client and Server inside one file (say transceiver.py) what I do not use select.select (instead using send bool variable) but it seems to work fine. But I do believe this is bad practice, so I wonder what can I do to improve my design.
def create_transceiver(ip, port):
address = (ip, port)
udp = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp.settimeout(1)
inout = [udp]
client = Client(udp, address)
server = Server(udp, address)
client_to_server = True
send = True
while True:
# infds, outfds, errfds = select.select(inout, inout, [])
if not send: # len(infds) != 0
if client_to_server:
server.start_recv()
client_to_server = False
else:
client.start_recv()
client_to_server = True
send = True
elif send: # len(outfds) != 0
if client_to_server:
if client.has_ack_all():
print(server.write_content())
break
client.start_send()
client_to_server = True
else:
server.start_send()
client_to_server = False
send = False
Here is the implementation of Echo Server:
import socket
ip = "10.10.1.100"
port = 8888
address = (ip, port)
udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp_socket.bind(address)
while True:
data, address = udp_socket.recvfrom(2048)
udp_socket.sendto(data, address)

And we have only one UDP socket.
As far as I understand this objective, you have a server process somewhere (you call it echo server) that is listening on a specific port. Also there is a client that wants to send data to some sort of server.
You provided code shows an implementation of what is called a state machine that (in your case) switches between receiving and sending.
However, Client and Server cannot communicate directly
The scenario you are describing makes your echo server a classic server that handles different types of clients. In your case this would be your "client" and your "server". I would like to call these just client-A and client-B. Most tutorials on the internet would call them Alice and Bob, I guess.
The objective is to implement a reliable UDP for file transfer.
So you want to transfer files between different clients using UDP as the base protocol.
UDP is not very well suited for this purpose. It does not garantee delivery of each packet transmitted. It is possible that packets arrive in different order than they were sent.
Usually you would use TCP for this kind of transmission. UDP is usually used for live streaming data like audio/video calls and stuff like that.
For more information on the differences between UDP and TCP you might check out the wikipedia pages for each:
https://en.wikipedia.org/wiki/User_Datagram_Protocol
https://en.wikipedia.org/wiki/Transmission_Control_Protocol
It is possible to use UDP for your transfers but you would have to implement all the safeties provided by TCP yourself.
I assume, your client and your server are actually different programs. Otherwise there would be a way they could communicate directly.
If that is the case, this tutorial might give you a starting point:
https://pythonprogramming.net/server-chatroom-sockets-tutorial-python-3/

Writing an external program to interface with wpa_supplicant

I need to interact directly with wpa_supplicant from Python. As I understand it one can connect to wpa_supplicant using Unix sockets and wpa_supplicant control interface (https://w1.fi/wpa_supplicant/devel/ctrl_iface_page.html).
I wrote a simple program that sends a PING command:
import socket
CTRL_SOCKETS = "/home/victor/Research/wpa_supplicant_python/supplicant_conf"
INTERFACE = "wlx84c9b281aa80"
SOCKETFILE = "{}/{}".format(CTRL_SOCKETS, INTERFACE)
s = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
s.connect(SOCKETFILE)
s.send(b'PING')
while 1:
data = s.recv(1024)
if data:
print(repr(data))
But when I run it, wpa_supplicant reports an error:
wlx84c9b281aa80: ctrl_iface sendto failed: 107 - Transport endpoint is not connected
Could someone please provide an example, how you would do a 'scan' and then print 'scan_results'.

Apparently, the type of socket that wpa_supplicant uses (UNIX datagram) does not provide any way for the server to reply. There are a few ways to get around that. wpa_supplicant in particular seems to support replies through a separate socket (found at a path appended at the end of each message).
Weirdly enough, this seems to be a relatively common practice in Linux: /dev/log seems to work in the same way.
Here's a program that does what you asked for:
import socket, os
from time import sleep
def sendAndReceive(outmsg, csock, ssock_filename):
'''Sends outmsg to wpa_supplicant and returns the reply'''
# the return socket object can be used to send the data
# as long as the address is provided
csock.sendto(str.encode(outmsg), ssock_filename)
(bytes, address) = csock.recvfrom(4096)
inmsg = bytes.decode('utf-8')
return inmsg
wpasock_file = '/var/run/wpa_supplicant/wlp3s0'
retsock_file = '/tmp/return_socket'
if os.path.exists(retsock_file):
os.remove(retsock_file)
retsock = socket.socket(socket.AF_UNIX, socket.SOCK_DGRAM)
retsock.bind(retsock_file)
replyToScan = sendAndReceive('SCAN', retsock, wpasock_file)
print(f'SCAN: {replyToScan}')
sleep(5)
replyToScanResults = sendAndReceive('SCAN_RESULTS', retsock, wpasock_file)
print(f'SCAN_RESULTS: {replyToScanResults}')
retsock.close()
os.remove(retsock_file)

Socket programming stuck waiting for a response from the server

For a class assignment I need to use the socket API to build a file transfer application. For this project there two connections with the client and server, one is called the control and is used to send error messages and the other is used to send data. My question is, on the client side how can I keep the control socket open and waiting for any possible error messages to be received from the server while not blocking the rest of the program from running?
Example code (removed some elements)
#Create the socket to bind to the server
clientSocket = socket(AF_INET,SOCK_STREAM)
clientSocket.connect((serverName,portNum))
clientSocket.send(sendCommand) # Send to the server in the control connection contains either the list or get command
(If command is valid server makes a data connection and waits for client to connect)
clientData = socket(AF_INET,SOCK_STREAM)
clientData.connect((serverName,dataport)) #Client connects
recCommand = clientData.recv(2000) #Receive the data from server if command is successful
badCommand = clientSocket.recv(2000) #But if there is an error then I need to skip the clientData.recv and catch the error message in bad Command

when there is an error, the data-socket should be closed by the server, so recv ends automatically.

Problems running python program with srvany.exe

I must preface this with a full disclaimer that i'm very early in my python development days
I've made a simple python program that waits for a socket connection to the local ip address over port 20000. When it gets a connection, it pops up a message alert using the win32api.
#tcpintercomserver.py
import socket
import sys
import win32api
ip = socket.gethostbyname(socket.gethostname())
#socket creation
sock=socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#Binding
server_address = (ip, 20000)
sock.bind(server_address)
print server_address
#Listen
sock.listen(1)
while True:
# Wait for a connection
connection, client_address = sock.accept()
win32api.MessageBox(0,'MessageText','Titletext', 0x00001000)
# Close Connection
connection.close()
I also have a mated client program that simply connects to the socket. The script takes an argument of the host you're trying to reach (DNS name or ip address)
#tcpintercomcli.py
import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = (sys.argv[1], 20000)
sock.connect(server_address)
This all runs fine as scripts. I then used CX_Freeze to turn them into executables. Both run just like they did when they were scripts.
Now i've taken the server script and connected it to a service with srvany.exe and use of the SC command in windows.
I set up the service using SC create "intercom" binPath= "C:\dist\srvany.exe"
Under the intercom service key in the registry, i've added the Parameter's key, and under there set Application to a string value c:\dist\tcpintercomserver.exe
I then perform a "net start intercom" and the service launches successfully, and the tcpintercomserver.exe is listed as a running process. However, when i run the tcpintercomcli.py or tcpintercomcli.exe, no alert comes up.
I'm baffled...is there something with the CX_Freeze process that may be messing this up?

Service process cannot show messagebox, they don't have access to UI, they usually run as SYSTEM user. if you are running from service, proper way of debugging and showing messages are using EventLog.
See:
http://msdn.microsoft.com/en-us/library/system.diagnostics.eventlog%28VS.71%29.aspx

If you are on Windows Vista or later, your script is running headlong into Session 0 Isolation -- where GUI elements from a Windows service are not shown on an interactive user's desktop.
You will probably see your message box if you switch to session 0...

Both server and the client in the same program in python

Can both a client and a server be run in the same program at the same time in Python. I want to connect a client to an external server and a server to receive messages from that external server at the same time. Whenever my server receives message from that external server my client should send messages to that external server accordingly.
Following is the way I tried to achieve that ( Just the connecting part)
import select
import socket
host = 'localhost'
portClient = 6000
portServer = 7000
backlog = 5
size = 1024
client = socket.socket()
server = socket.socket()
client.connect((host,portClient))
client.send('#JOIN')
server.bind((host,portServer))
server.listen(backlog)
running = 1
while running:
c,address = server.accept()
c.close()
client.close()
server.close()
When I run this code, no response from the external server comes.
When the while loop is omitted. I get an error saying that our server has actively refused to accept the external server.
Can I achieve this by using Python select module or Threading module? Or is there a better way?

TCP socket is a bi-directional stream of bytes. You can, and should, do all your communication with the server over the same single socket.