I'm trying to combine a Node.js with Python to create a socket connection.
The problem is that I can send data, but I can't maintain the connection.
This is my server in Node.js
var net = require('net');
var HOST = '127.0.0.1';
var PORT = 1337;
net.createServer(function(sock) {
console.log('CONNECTED: ' + sock.remoteAddress +':'+ sock.remotePort);
sock.on('data', function(data) {
console.log('DATA ' + sock.remoteAddress + ': ' + data);
sock.write('You said "' + data + '"');
});
sock.on('close', function(data) {
console.log('CLOSED: ' + sock.remoteAddress +' '+ sock.remotePort);
});
}).listen(PORT, HOST);
console.log('Server listening on ' + HOST +':'+ PORT);
and this is my client side in Python
import socket
import sys
# Create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# Connect the socket to the port where the server is listening
server_address = ('localhost', 1337)
print >>sys.stderr, 'connecting to %s port %s' % server_address
sock.connect(server_address)
try:
# Send data
message = 'This is the message.'
print >>sys.stderr, 'sending "%s"' % message
sock.sendall(message)
finally:
print >>sys.stderr, 'closing socket'
This works great but the client disconnects right after it has sent the data.
Ultimately, I want to be able to give user-input to send data, and also receive data.
Any suggestions on how to do this would be appreciated.
I'll approach the user input scenario. As of now, your program simply runs its course and exists.
You want to be able to combine two naively blocking operations, running some sort of input loop (e.g. while True: data = input()) but handle incoming traffic as well.
The basic way to do this is to have 2 threads, one for user input and the other for socket connections in similar while True: data = socket.recv(buff) loop, but there's another catch here as you might block on a single connection -- you'll have to dedicate a thread per connection. In order to avoid this, you could use select, which maintains socket connections for you asynchronously.
If there's no user-input, than you can just use select -- that will be sufficient to handle multiple connections in a concise manner.
Either way, I suggest you take a look at some asynchronous event-driven frameworks that are select based such as asyncio and Twisted.
Related
The server only listens for a message from the first socket to connect, even though it is set to nonblocking, it doesn't skip over it when it doesn't receive data. I'm new to networking and this is my first project, if anyone know of any others good for beginners please let me know. Thanks! Here is the code.
import socket
CONNECTED_SENDERS = []
CONNECTED_LISTENERS = []
def Main():
HOST = socket.gethostname()
PORT = 4444
SERVER_SOCKET = socket.socket()
SERVER_SOCKET.bind((HOST, PORT))
SERVER_SOCKET.listen(1)
for i in range(2):
CONNECTION, ADDRESS = SERVER_SOCKET.accept()
CONNECTED_LISTENERS.append(CONNECTION)
for i in range(2):
CONNECTION, ADDRESS = SERVER_SOCKET.accept()
CONNECTED_SENDERS.append(CONNECTION)
for DEVICE in CONNECTED_LISTENERS:
DEVICE.send(b'SERVER: You have succesfully connected.')
DEVICE.send(b'SERVER: Please wait for permission to talk.')
x = 0
for DEVICE in CONNECTED_LISTENERS:
DEVICE.send(b'SERVER: What is your name?')
Name = CONNECTED_SENDERS[x].recv(1024)
CONNECTED_LISTENERS[x] = (CONNECTED_LISTENERS[x], Name)
x += 1
del x, Name
for DEVICE, _ in CONNECTED_LISTENERS:
DEVICE.send(b'SERVER: You may now talk.')
SERVER_SOCKET.setblocking(0)
LEAVE = False
while LEAVE == False:
try:
MESSAGE = CONNECTED_SENDERS[0].recv(1024)
NAME = CONNECTED_LISTENERS[0][1]
for DEVICE, _ in CONNECTED_LISTENERS:
DEVICE.send(NAME + b': ' + MESSAGE)
if MESSAGE == 'QUIT':
LEAVE = True
except:
try:
MESSAGE = CONNECTED_SENDERS[1].recv(1024)
NAME = CONNECTED_LISTENERS[1][1]
for DEVICE, _ in CONNECTED_LISTENERS:
DEVICE.send(NAME + b': ' + MESSAGE)
if MESSAGE == 'QUIT':
LEAVE = True
except:
pass
for CONNECTION in CONNECTED_LISTENERS:
CONNECTION.close()
for CONNECTION in CONNECTED_SENDERS:
CONNECTION.close()
if __name__ == "__main__":
Main()
There are a number of issues with your code, some small and some big. But the main problem is that you're marking the server socket nonblocking, not any of the sockets on which communication takes place.
In standard TCP socket programming, you set up a server which listens for incoming connections. When that server accepts a new client, this returns a new socket, and it's on this new socket that all communication with the remote client happens. In other words, the server socket is just for accepting new connections, and nothing else. You never write data through the server socket.
So it doesn't matter that SERVER_SOCKET is marked nonblocking, you must do something like this:
conn, addr = server.accept()
conn.setblocking(False)
conn is the new socket through which you talk to the client, and can be used in a nonblocking fashion.
Smaller issues:
I should also point out that you call SERVER_SOCKET.listen(1). That argument of 1 means that the server will only have a backlog of waiting connections from one client. So if a second client connects before the first connection is made, the second client will receive an error, ECONNREFUSED. Given what it looks like you're trying to do, I'd guess SERVER_SOCKET.listen(4) is appropriate.
Next, nonblocking communication is much harder than blocking protocols. I'd suggest you improve your networking skills before tackling them, but when you're ready, look at the select or selectors modules for help. They provide tools to wait for communication from any of a number of clients, rather than looping over them all and checking if data is available, as you've done here. This looping is very inefficient.
Finally, in Python, it's good practice to name variables with lower case, underscore-separated names. UPPER_CASE_NAMES are usually reserved for constants. So change SERVER_SOCKET to server_socket, CONNECTED_LISTENERS to connected_listeners, etc.
I currently have a Python client & server sending json object over socket as follows.
Client
# Create the socket & send the request
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Connecting to server at host: ' + (self.host) + ' port: ' + str(self.port)
s.connect((self.host, self.port))
print 'Sending signing request to the server'
s.sendall(request_data)
print 'Waiting for server response'
response_data = s.recv(10 * 1024)
print 'Got server response'
s.close()
Server
# Create a socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Starting the server at host: ' + (self.host) + ' port: ' + str(self.port)
s.bind((self.host, self.port))
s.listen(1)
while True:
# Create a new connection
print 'Listening for client requests...'
conn, addr = s.accept()
print 'Connected to: ' + str(addr)
# Get the data
request_data = conn.recv(10 * 1024)
print 'Got message: ' + str(request_data)
# Get the json object
try:
# Decode the data and do stuff
# ...
# ...
except Exception as e:
print e
finally:
# Close the connection
conn.close()
However, besides the json object, I also need to send a file (which is not a json object). Inside the Server's while loop, the socket cannot distinguish when the json object ends and file starting receiving.
My questions here is about the methodology. What would be the usual approach to send two distinct types of data through the socket? Can we use the same socket to receive two data types in a serial order? Would that require two while loops (one for json, another for file) inside the current while loop?
Or are there any other ways of doing so?
Thanks.
First things first, you cannot just do
response_data = s.recv(10 * 1024)
print 'Got server response'
or
# Get the data
request_data = conn.recv(10 * 1024)
print 'Got message: ' + str(request_data)
and then say you've got the data. Transmissions over TCP do not preserve their borders.
Regarding methodology, you need a protocol built over TCP. HTTP would be a great choice if you don't need your server to connect to clients without a request. In this case great libraries and frameworks are available.
If you want to build your own protocol, consider using control characters in your data stream. Something like this is possible:
json = b"{foo: ['b', 'a', 'r']}\n" # \n here stands for end-of-the-json symbol
sock.send_byte(TYPE_JSON_MESSAGE)
sock.sendall(json)
sock.send_byte(TYPE_FILE_MESSAGE)
sock.send_int(file_size) # so the server can determine where the file transmission ends
for chunk in chunked_file:
sock.sendall(chunk)
Here it's up to you to implement send_byte and send_int. It's not really difficult, if you use struct module.
On the server side:
message_type = sock.recv(1)
if message_type == TYPE_JSON_MESSAGE:
# ...
elif message_type == TYPE_FILE_MESSAGE:
file_size = sock.recv_int() # not implemented
# ...
I have an idea like how basic communication between client and server is established. So serialize data streams can be passed between client and server. But I want to know, how socket objects can be passed between two clients: I want to know is it possible to pass socket objects between two clients and both share the same socket instance. Please suggest.
Client class:
import socket
import sys
# create a TCP/IP socket
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
#connect the socket to the port where server is listening
server_address = ('localhost',2000)
print >>sys.stderr, 'connecting to %s port %s' % server_address
sock.connect(server_address)
#after connection is established, data can be through socket with sendall() and recv()
try:
#send data
message = 'This is Message. It will be repeated'
print >>sys.stderr, 'sending "%s"' % message
sock.sendall(message)
#llok for the response
amount_received = 0
amount_expected = len(message)
while amount_received < amount_expected:
data = sock.recv(16)
amount_received += len(data)
print amount_received
print >>sys.stderr, 'received "%s"' % data
finally:
print >>sys.stderr, 'closing socket'
sock.close()
Server class created to receive message from the client and revert with some message.
Server class:
import socket
import sys
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server_address = ("localhost",2000)
print >>sys.stderr, 'starting up on %s port %s' %server_address
sock.bind(server_address)
sock.listen(1)
while True:
print >>sys.stderr, 'waiting for connection'
connection, client_address = sock.accept()
try:
print >>sys.stderr, 'connection from', cleint_address
while True:
data = connection.recv(16)
print >>sys.stderr, 'received "%s"' % data
if data:
print >>sys.stderr, 'sending data back to the client'
connection.sendall(data)
else:
print >>sys.stderr, 'no more data from', client_address
break
finally:
connection.close()
After server started the client connects with server and displays suitable messages. Now instead of sending messages between client and server, I want to send socket object to another client which can be achieved using either TCP or UDP. In TCP, serialization of data is required. I want to know is there any way to wrap socket object and pass it over.
Socket objects can not be transported (or you know, teleported :D ) to another language or anything. At most, you can create a protocol by which an instance of the socket can be transferred to another language. But again, I don't see how it may help you.
You have a server socket listening on 2000 port. And another Java socket may connect to it using a client socket. So, what's the point of sending one of the socket to the another? the communication link is somehow twirled. Like, we can just eat ourself to regenerate us. But that would be impossible
Similarly, at most, you can send an instance of the server socket to the java socket. But on the same computer, the Java won't be able to recreate it, because the port is already being listened by another program.
Next, if two programs could listen on the same port, that would make stealing of data and forging quite easy. So, it is not possible for two programs to listen on the same port.
I think what you are looking for is that, two programs combinedly handle the I/O of the same socket. That is rational, at least.
For that, you should create some sort of bidirectional communication link between these two processes. Like another socket on a different port.
Like S is the Server (the sole owner of the socket S1) meanwhile A and B are the handlers.
S should be listening on two different ports. where only A and B are connected. Then any data that comes to S, at S' discretion would be A or B appropriate, then, A or B will reply to that request. And then S will respond appropriate.
Another approach would be S is the main server socket. and A and B are servers listening on different ports. Whenever data comes to S, S sends it to A or B depending on content.
Thirdly, and the most messy solution would be that, A is the server and it offloads some tasks to B via some sort of communication (server-client or threads or a subprocess) and they handle data appropriately.
The reason of calling it messy is that one has to handle two tasks and its harder to maintain its functionality.
But still, sharing a socket is like using the same page of a copy for two different tasks. Hope it helped
I attempted to find an answer for this, but most examples out there are for purely echo base Socket Servers.
Basically I have the following code:
import socket
import sys
from thread import *
HOST = '' # Symbolic name meaning all available interfaces
PORT = 8888 # Arbitrary non-privileged port
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print 'Socket created'
#Bind socket to local host and port
try:
s.bind((HOST, PORT))
except socket.error as msg:
print 'Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1]
sys.exit()
print 'Socket bind complete'
#Start listening on socket
s.listen(10)
print 'Socket now listening'
#Function for handling connections. This will be used to create threads
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Type something and hit enter\n') #send only takes string
#infinite loop so that function do not terminate and thread do not end.
while True:
#Receiving from client
data = conn.recv(1024)
if data == "hello":
reply = 'OK...Hello back to you'
else:
reply = '01:OK - ' + data
if not data:
break
conn.sendall(reply)
#came out of loop
conn.close()
#now keep talking with the client
while 1:
#wait to accept a connection - blocking call
conn, addr = s.accept()
print 'Connected with ' + addr[0] + ':' + str(addr[1])
#start new thread takes 1st argument as a function name to be run, second is the tuple of arguments to the function.
start_new_thread(clientthread ,(conn,))
s.close()
Everything works great until I try to use a conditional statement. I am very new to python and I am using this as a way to learn it a little better, but when the following line runs it skips the if each time.
#Receiving from client
data = conn.recv(1024)
if data == "hello":
reply = 'Why hello there!'
else:
reply = '01:OK - ' + data
if not data:
break
conn.sendall(reply)
From the telnet client I am connecting to it just echos everything I send it including the 'hello' I send it rather than the phrase.
I have a feeling that it is something simple, but I am not sure the format of the data variable.
Very close!
Telnet will send whatever EOL delimiter you gave it, along with the text. So if you type "data" and press enter, data is actually something like hello\r\n.
You can effectively ignore this whitespace by doing something like changing
data = conn.recv(1024)
to
data = conn.recv(1024).strip()
And you should be good to go.
EDIT:
As noted in the comments, the network may split up the message into multiple packets. To work around this, you can use use the socket.makefile() method and get a file-like object and then use readline() which will block until a complete line is available. For example, changing clientthread to:
def clientthread(conn):
#Sending message to connected client
conn.send('Welcome to the server. Type something and hit enter\n') #send only takes string
sfh = conn.makefile("r+b", bufsize=0)
#infinite loop so that function do not terminate and thread do not end.
while True:
#Receiving from client
data = sfh.readline().strip()
if data == "hello":
reply = 'OK...Hello back to you'
else:
reply = '01:OK - ' + data
if not data:
break
conn.sendall(reply)
#came out of loop
conn.close()
== PYTHON ==
Socket created
Socket bind complete
Socket now listening
Connected with 192.168.1.10:42749
== TELNET ==
$ telnet 192.168.1.106 8888
Trying 192.168.1.106...
Connected to 192.168.1.106.
Escape character is '^]'.
Welcome to the server. Type something and hit enter
hello
OK...Hello back to you
I have a device that continually outputs data and I would like to send that data to a client on the same network as it is produced and I'm not finding a good solution. Here is what I'm trying.
Server:
import SocketServer
from subprocess import Popen,PIPE
class Handler(SocketServer.BaseRequestHandler):
def handle(self):
if not hasattr(self, 'Proc'):
self.Proc = Popen('r.sh', stdout=PIPE)
socket = self.request[1]
socket.sendto(self.Proc.stdout.readline(),self.client_address)
if __name__ == "__main__":
HOST, PORT = "192.168.1.1", 6001
server = SocketServer.UDPServer((HOST, PORT), Handler)
server.serve_forever()
Client:
import socket
data = " ".join(sys.argv[1:])
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(data + "\n", ("192.168.1.1", 6001))
try:
received = sock.recv(1024)
while True:
print "Sent: {}".format(data)
print "Received: {}".format(received)
sock.sendto('more' + "\n", ("192.168.1.1", 6001))
received = sock.recv(1024)
except:
print "No more messages"
arg[1] for the client is a program that outputs lines of data for several minutes that I need to process as it is created. The problem seems to be that every time the client sends another request, a new Handler object is created, so I loose Proc. How can I stream Proc.stdout?
Edit: The device is a Korebot2, so I have limited access to other python libraries due to space.
Using UDP you get a new "connection" each time you send a datagram, which is the reason you notice that a new object instance is created each time you send something. You're probably using the wrong kind of protocol here... UDP is used mostly for sending distinct "datagrams", or when a longer connection is not needed. TCP is also called a "streaming" protocol, and is often used for data that has no fixed end.
Also remember that UDP is not a reliable protocol, if used over a network it is almost guaranteed that you will loose packets.