I'm writing a multithreaded distributed networking algorithm.
I've one thread that listens to new connections. Every time a new connection is established a separate thread is started for listening to messages from that connection.
My problem is that the socket I open works perfectly in both directions inside the connection listener. After I pass the socket object for that connection to the message listener I can read data from the socket, but sending data through it doesn't reach the remote host.
Here's the essential snip from my code:
def connection_listener(port, start_e, terminate_e):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.settimeout(1)
s.bind(('', port))
s.listen(1)
while (not start_e.isSet()):
try:
conn, addr = s.accept()
msg_in = conn.recv(1024).split(":")
if (msg_in[1]=="hello"):
# If addr sends us a 'id:hello', we reply with a 'my_id:welcome'
conn.send(str(my_id)+":welcome")
t = Thread(target=message_listener, args=(conn, addr[0], terminate_e, ))
t.start()
except:
pass # timeout
def message_listener(conn, address, terminate_e):
while (not terminate_e.isSet()):
try:
msg_in = conn.recv(1024)
# Here I can receive everything that I send from the other end of conn,
# but conn.send("any data") doesn't reach the remote host
What I'd like to do is send acknowledgement-like messages from the message listener thread using the conn. Is this possible somehow or am I thinking and doing it wrong?
I sorted this out myself, so I'll share my answer.
I made the protocol exchange fixed size messages by padding with zeroes up to the desired length. I used a length of 32 bytes, which might be quite tiny from the hardware's point of view. Nevertheless it seems to work as supposed.
Pragmatically my solution looks like:
def send_everyone(message):
for i in range(len(peers)):
chunk = (str(my_id)+":"+message).rjust(32, '0')
peers[i].send(chunk)
And on the receiving side we want only 32 bytes at a time:
def message_listener(conn, address, terminate_e):
while (not terminate_e.isSet()):
try:
msg_in = conn.recv(32)
...
Related
Im trying to send a messages from the server to the client
I tried deleting the .close and puting a while loop on print but it still doesn't won't to work
Client
import socket
s = socket.socket()
host = socket.gethostname()
port = 12345
s.connect((host, port))
while True:
print (s.recv(1024))
Server
import socket
s = socket.socket()
host = socket.gethostname()
port = 12345
s.bind((host, port))
s.listen(5)
while True:
c, addr = s.accept()
print ('Got connection from', addr)
x = str(input("ënter a message"))
data = x.encode()
c.send(data)
I expect the output to be 2 messages from the server but it is only sending 1 and then closing the connection
Switch your accept and while True: lines. Once you accept a connection, keep sending on the same connection.
Note that TCP is a streaming protocol. There is no concept of "messages", but just a bunch of bytes. If you send fast enough, such as:
c.send(b'abc')
c.send(b'def')
then recv(1024) could receive b'abcdef'. For more complex communication, you'll have to define a protocol and buffer recv until you are sure you have a complete message. A simple way in this case is read until you find a newline, or send a byte (or more) indicating the size of the total message before sending the actual message.
I have a client class, in which it should connect to the server (in this case, iKettle) and sends and receives data. The IP address should come from another main server (so another class), and this IP address can change.
Below is the client code:
#command codes
_ON = "0x4<LF>"
_OFF = "0x0<LF>"
#base command
_BASE_COMMAND = "set sys output "
_SLEEP_TIME = 0.5
#size of buffer when receiving data
_BUFFER_SIZE = 1024
ip_address = ""
port = 2000
def initialiseSocket(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
return s
def setOn(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send("set sys output 0x4<LF>")
time.sleep(self._SLEEP_TIME)
self.kettleResponse(ip_address)
def setOff(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send(self._BASE_COMMAND + self._OFF)
time.sleep(self._SLEEP_TIME)
self.kettleResponse(ip_address)
def kettleResponse(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
data = s.recv(self._BUFFER_SIZE)
print(data)
And this is an example of commands from another class:
kettle.setOn(KETTLEIP)
However, when running this code, it does not print anything.
Can anyone help please?
You're probably connecting and reconnecting too much. Sometimes that simplifies things, but in this case I believe it's making you lose your response.
Try just connecting once, and reusing the socket.
Also, keep in mind that TCP is a byte-oriented protocol, not a message-oriented protocol. IOW, if you send 10k, 10k, 10k, the other side of the TCP socket may receive 5k, 8k, 17k - or even more bizarre possibilities. The total number of bytes will be the same, and the data will arrive in the correct order, but the sizes of the chunks could be totally scrambled. For this reason, most folks use REST with http these days - it's simpler in some ways.
If you're married to TCP, perhaps try my bufsock module (or actually, it's Opensource with a Univ Calif Irvine copyright, but I wrote it while I was working for them, and obtained permission from them to release it). It's at http://stromberg.dnsalias.org/~strombrg/bufsock.html . Its methods behave closer to what most people expect out of TCP.
HTH.
My guess is that you should reuse the socket as #dstromberg indicates:
class Kettle:
# other stuff omitted...
def setOn(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send("set sys output 0x4<LF>")
time.sleep(self._SLEEP_TIME)
self.kettleResponse(s) # pass the socket on
def kettleResponse(self, s):
# don't open a new socket, just reuse s
data = s.recv(self._BUFFER_SIZE)
print(data)
I'm developing a reverse shell application in python, and right now I'm trying to implement an autodiscovery feature. It should work as follows:
The server broadcasts the IP/port it listens for connections on, and waits for a client. If no client tries to connect in a few seconds, it broadcasts again (and repeat till a connection).
The client tries to receive the broadcast of the server, and connects to the advertised IP/port.
The broadcast works fine, the client receives the IP/port and successfully connects, however after using the connected pair of ports I get (server side):
socket.error: [Errno 35] Resource temporarily unavailable
Server side test code:
sckt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sckt.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sckt.settimeout(2)
sckt.bind(('', 9999))
sckt.listen(5)
broadcastSocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
broadcastSocket.setsockopt(socket.SOL_SOCKET, socket.SO_BROADCAST, 1)
broadcastSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
while True:
broadcastSocket.sendto(socket.gethostbyname(socket.getfqdn()) + ' ' + str(9999), ('<broadcast>', 8888))
try:
sock, address = sckt.accept()
break
except socket.timeout:
pass
broadcastSocket.close()
sckt.settimeout(None)
sock.send('test')
# if I add time.sleep(1) here, it works, but I don't get why
# would sock be unavailable at first, but available a second later
print sock.recv(1) # this is where it fails
# note that it also fails with any recv buffer size, for instance 1024
Why on earth would I want to receive 1 byte of data, you might ask. I have an algorithm which prefixes messages with their lengths, and the receiver reads this prefix byte-by-byte till a delimiter, thats why.
Client side test code:
broadcastSocket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
broadcastSocket.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
broadcastSocket.settimeout(3)
broadcastSocket.bind(('', 8888))
while True:
try:
data = broadcastSocket.recv(1024)
break
except socket.timeout:
pass
broadcastSocket.close()
sckt = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sckt.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
sckt.connect((str(data.split()[0]), int(data.split()[1])))
print sckt.recv(1024)
sckt.send('lel')
If I omit the whole broadcast and autodiscovery part of the code and simply manually enter the IP/port of the server print sock.recv(1) doesn't fail.
Any clues on what the issue might be?
Change sckt.settimeout(None) to sock.settimout(None) in the server code.
You want to have the accepted socket in blocking mode and not the accepting one.
This ensures that the sckt.recv waits for an incoming message from the client.
P.S.
sock.setblocking(1) is exactly the same
I'm am trying to write a client program in Python that can send and receive from the same socket, but it is always giving me the same error which address is already in use. Here is the function I'm trying to write.
def Login():
username=raw_input()
password=raw_input()
message=raw_input()
array=[username,password,message]
TCP_IP = '127.0.0.1'
TCP_PORT = 5563
BUFFER_SIZE = 1024 # Normally 1024, but we want fast response
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((TCP_IP, TCP_PORT))
array_string=pickle.dumps(array)
sock.send(array_string)
sock.close()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((TCP_IP, TCP_PORT))
sock.listen(1)
conn, info = sock.accept()
while 1:
data = serverSocket.recv(1024)
if not data:break
conn.send(data)
conn.close()
There is a bunch of truly newbie errors here.
You can't ever connect a TCP socket to itself. There must be two different sockets.
If you really want to get the data you sent earlier at a listening socket, this listening socket must be created, bound and configured to listen before the client side connects (or, at least, in parallel to this connect attempt, in a few seconds, so the connect attempt will try - but this very likely won't work on localhost).
You can't wait on connect and on accept in the same thread if both are blocking. The simplest approach is to separate the client side and the server side to 2 different programs and run them manually in parallel. Then, after successful debugging, you will be able to do this in different threads of the same process, or using an event-driven engine.
While you may not be able to connect a socket to itself to send and receive data, you might be able to learn from the following example inspired by your code that attempts to do something similar.
import _thread
import pickle
import socket
import time
def main():
"""Run a server in a thread and start a client to talk to it."""
_thread.start_new_thread(run_server, ('', 5563))
run_client('localhost', 5563)
def run_server(host, port):
"""Handle all incoming connections by spawning worker threads."""
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((host, port))
server.listen(5)
while True:
_thread.start_new_thread(handle_connection, server.accept())
def handle_connection(client, address):
"""Answer an incoming question from the connected client."""
print('Incoming connection from', address)
client.settimeout(0.1)
data = recvall(client)
client.shutdown(socket.SHUT_RD)
question = pickle.loads(data)
answer = '''len(username) = {}
len(password) = {}
len(message) = {}'''.format(*map(len, question))
client.sendall(answer.encode())
client.shutdown(socket.SHUT_WR)
client.close()
print('Finished with', address)
def recvall(connection):
"""Receive all data from a socket and return as a bytes object."""
buffer = bytearray()
while True:
try:
data = connection.recv(1 << 12)
except socket.timeout:
pass
else:
if data:
buffer.extend(data)
else:
return bytes(buffer)
def run_client(host, port):
"""Collect information from question and display returned answer."""
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
time.sleep(0.1) # wait for server to start listening for clients
client.connect((host, port))
time.sleep(0.1) # wait for handler thread to display connection
username = input('Username: ')
password = input('Password: ')
message = input('Message: ')
question = pickle.dumps((username, password, message))
client.sendall(question)
client.shutdown(socket.SHUT_WR)
answer = recvall(client)
client.shutdown(socket.SHUT_RD)
client.close()
print(answer.decode())
time.sleep(0.1) # wait for handler to cleanly terminate execution
if __name__ == '__main__':
main()
I am writing python daemon, which waiting for request and return the some data. But, when request the script by client, the daemon stopped it's work.
This is the code
......
def run(self):
while True:
server_socket = socket.socket() # Create a socket object
host = socket.gethostname() # Get local machine name
port = 12345 # Reserve a port for your service.
server_socket .bind((host, port)) # Bind to the port
server_socket .listen(5)
client_socket, address = server_socket.accept()
# Establish connection with client.
print 'Got connection from', address
logging.info(address)
#data = client_socket.recv(2048)
data = client_socket.recv(1024)
logging.info(data)
......
I am not certain what you really want to be the server functionality, but based on your example, what you have is a server that can accept and process one client at a time. What I would suggest is not rebinding the server every time after every connection. It is not necessary.
Bind the socket once, then go into a loop waiting for clients:
def run(self):
server_socket = socket.socket()
host = socket.gethostname()
port = 12345
server_socket.bind((host, port))
server_socket.listen(5)
while True:
# this will block until a client tries to connect
client_socket, address = server_socket.accept()
print 'Got connection from', address
logging.info(address)
# this will block until data is received from client
data = client_socket.recv(1024)
logging.info(data)
What will happen here is the server will accept a client and then block while waiting for the client to send something. Then it will loop again waiting for a new client. You may even want to close that socket if it is a one-off connection and you are done handling it.
If you want to be able to handle multiple clients at the same time, then you would do the loop in your main thread, and then pass each new connection off to a worker thread to handle the communication. This will free up the main thread to continue looping and waiting for new clients to connect.
This can be achieved by either starting a brand new thread for each client and passing the socket, or, using a Queue shared between a fixed pool of running threads. Each thread would be waiting to pull an item from the queue, and then go into a communication loop on the socket item. When it is done, it goes back to waiting on the queue.
Pseudo-code might look something like this:
socket_queue = Queue()
...
def main_server_thread():
while True:
# this will block until a client tries to connect
client_socket, address = server_socket.accept()
print 'Got connection from', address
logging.info(address)
# this will finish right away and loop again
socket_queue.put(client_socket)
...
def client_worker_thread():
while True:
sock = socket_queue.get()
while sock is still connected:
data = sock.recv(1024)
logging.info(data)