I am currently working on a small project experimenting with different regions of python. I decided to make a multi-client TCP server in python, and wanted to be able to send a "packet" through the server, it be received by the other clients then parsed. However, I get an error if I try to send the packet, saying I must send either bytes or a string, is there a way to convert the object into bytes, then back or send the packet object through the server itself.
## EDIT ##
I have researched into UDP servers, and I do not believe that is what I am looking for, I probably provided too little information. I am creating a small trial game, for me and some friends to mess about on, I need there to be a constant connection to the server, as information is going to be constantly being sent across the network such as location,direction,speech,inventory etc. and I wanted a way to turn the entire Entity class into a byte array then send that and it be turned back into an instance of the Entity class when it was received.
You could use pickle to serialize/deserialize objects to strings and back. https://docs.python.org/2/library/pickle.html
The simplest possible approach would be to send (gzipped?) JSON'd or msgpack'd objects.
For example, using UDP, this could look something like the below code; note that you would want to reuse the socket object rather than instantiating a new one every time.
import socket
import msgpack
def send_object(obj=None, ip, port):
if obj:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(msgpack.dumps(obj), (ip, port))
Related
I have a first sender script in Python 3.10 which needs to send some data
def post_updates(*args):
sender.send_message("optional_key", args)
Then a second receiver script in Python 3.7 which needs to receive this data
while True:
args = receiver.get_message("optional_key", blocking=True)
print("args received:", args)
Constraints:
Each script should not depend on the presence of the other to run.
The sender should try to send regardless if the receiver is running.
The receiver should try to receive regardless if the sender is running.
The message can consist of basic python objects (dict, list) and should be serialized automatically.
I need to send over 100 messages per second (minimizing latency if possible).
Local PC only (Windows) and no need for security.
Are there 1-liner solutions to this simple problem? Everything I look up seems overly complicated or requires a TCP server to be started beforehand. I don't mind installing popular modules.
UDP and JSON look perfect for what you're asking for, as long as
you don't need there to be more than one receiver
you don't need very large messages
you just need to send combinations of dicts, lists, strings, and numbers, not Python objects of arbitrary classes
you're not being overly literal about finding a "1-liner": it's a very small amount of code to write, and you're free to define your own helper functions.
Python's standard library has all you need for this. Encoding and decoding from JSON is as simple as json.dumps() and json.loads(). For sending and receiving, I suggest following the example on the Python wiki. You need to create the socket first with
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
regardless of if you're making the sender or the receiver. The receiver will then need to bind to the local port to listen to it:
sock.bind(('127.0.0.1', PORT))
And then the sender sends with sock.sendto() and the receiver receives with sock.recvfrom().
The good old pipe might do the job, but you need to assess how big the buffer size needs to be (given the async nature of your sender/receiver), and change the default pipe buffer size.
So currently I am trying to send udp to a server with a web domain like this
www.example.com/path?stuff=exist
I am currently trying to use socket
and this is is an example of my code
import socket
IPADDR = '64.233.177.139'
that is the ip of google, and not the ip I am currently trying to send to
PORTNUM = 9001
PACKETDATA = '42["message","test"]'
s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM, 0)
s.connect((IPADDR, PORTNUM, '/path?stuff=exist'))
s.send(PACKETDATA)`
And I currently get this error
TypeError: getsockaddrarg() takes exactly 2 arguments (3 given)
I am new to using the socket library and did some digging to no avail.
For some more context I just want to send the data in PACKETDATA to the server google.com/path?stuff=exist (not really that url, just an example)
Anything helps, thanks in advance.
Your immediate programming problem is that socket.connect expects to be called with an argument that is a tuple containing exactly two items -- a hostname (or IP address) string and a port number. But this program passes an argument tuple that contains 3 items -- a host address, a port number and the string '/path?stuff=exist'. That's why the error message complains about finding 3 things where only 2 were expected.
If you want to send '/path?stuff=exist' over UDP then you'll have to include it in the data portion of the datagram. You'll also have to come up with some way of making it distinguishable from the PACKETDATA that you're already putting into the datagram body. (That doesn't need to be fancy. You could just insert a space between the path string and the PACKETDATA.)
There might also be a conceptual problem here. A web server only listens for TCP traffic. It does not listen for UDP traffic. So unless you've arranged for your specific server to have some sort of listener accepting datagrams on the target UDP port, nothing at the server side will collect this traffic even after you've figured out how to get Python to send it.
When you setup a simple TCP listening socket using the Python 'socket' module, what are the different steps involved doing?
The code I'm talking about looks like this:
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('localhost', 50000))
s.listen(1)
conn, addr = s.accept()
The s = ... seems pretty straightforward - you are expressing your intent to create an ipv4 TCP socket, without having done anything yet.
What I'm curious about is this:
What does it mean to bind to a socket, without listening?
How does limiting the number of unaccepted connections using listen(n) work?
If you have listen(1), you're in the middle of dealing with the first connection you accepted, and a second client tries to connect, is the second client waiting for the SYN-ACK? Or does the 3 way handshake happen, and he's waiting for actual data?
What happens if a third client tries to connect - does he immediately get a TCP RST?
Does setting the number of unaccepted connections here set some option in the kernel to indicate how many connections it should accept? Or is this all handled in Python?
How can you be listening without accepting? What does it mean to accept a connection?
Every article I've come across seems to just assume these steps make sense to everyone, without explaining what exactly it is that each one does. They just use generic terms like
listen() starts listening for connections
bind() binds to a socket
accept() just accepts the connection
Defining a word by using that word in the definition is kind of a dumb way to explain something.
it's basically a 1-to-1 from the POSIX c calls and as such I'm including links to the man pages, so that you can read their explanation and corresponding c code:
socket creates a communication endpoint by means of a file-descriptor in the namespace of the address-family you specified but assigns neither address nor port.
bind assigns an address and port to said socket, a port which may be chosen randomly if you request a port for which you do not have the privilige. (like < 1024 for non-root user)
listen makes the specific socket and hence address and port a passive one, meaning that it will accept incoming connections with the accept call. To handle multiple connections one after the other, you get to specify a backlog containing them, connections that arrive while you're handling one get appended. Once the backlog is full, the system will respond as such to those systems with an approach that makes them reconnect by withholding SYN, withholding ACK response etc..
As usual you can find someone explaining the previous to you a lot better.
accept then creates a new non-listening socket associated with a new file descriptor that you then use for communication with said connecting party.
accept also works as a director for your flow of execution, effectively blocking further progress until a connection is actually available in the queue for it to take, like a spinlock. The only way around that is to declare the socket non-blocking in which case it would return immediately with an error.
I have a multicast server sending data that must be captured by a python client. The problem is that recvfrom does not receive any data or at least receive the first packet and sorta caches it. If I use recvfrom in a loop then my data is received correctly.
My question is why I should use recvfrom in a loop to have the expected behavior?
from socket import *
s=socket(AF_INET, SOCK_DGRAM)
s.bind(('172.30.102.141',12345))
m=s.recvfrom(1024)
print m[0]
# sleep for x seconds here
m=s.recvfrom(1024)
print m[0]
# print the exact same thing as previously...
One thing is for sure, multicast is basically sending UDP packages and you have to keep listening for new packages. That is true even for TCP protocol based communication.
When you use low level interfaces for network communication, like socket is, it's up on both sides to define application level protocol.
That means, you define how receiving party concludes that message is complete. This is because message could get split in multiple parts/packets that get through the network. So receiving side has to assemble them in a proper way and then check if the message is whole. After that you push it up through the pipeline of processing messages or whatever you do in receiving side.
When using UDP, receiving side doesn't know if there is any packet on its way, so it just does try to recvfrom 1024 bytes and finishes. It doesn't know and should not care if there is more data on it's way. It's up to you to take care of that.
I'm new to the world of networking and wanted to clarify some of my thoughts regarding a problem I am facing at the moment. I saw this post which makes me think what I'm doing may be impossible, but I thought it would be worth a shot to ask on here and see what more qualified people think about it.
I am a TA for an intro computer science course, and I am writing a final project for students to complete at the end of the semester. Essentially, the project would be to fill in the holes in the implementation of a messaging client. I have set it up so each client would run two threads (one to listen for incoming messages, and one to wait for input to send messages to the other client). I have gotten this to work successfully on localhost with communication between two different port numbers, and am trying to find a way to have this work over the network so the two clients do not necessarily have to be on the same machine.
After struggling through a few methods, I came up with this solution: I would host a server on Heroku that would keep track of the clients' IPs and port numbers, and use a rest API so that one client could easily get the IP and port of the other client they are trying to communicate with. I have tested this, and the API seems to work. Thus, a client can create a socket endpoint and send it to this server to be entered into its database, and when the communication is terminated, it is removed from the database (this JSON would store a username as the primary key and internally manage an IP and port number) as the connection is now closed.
So, what I have is each client with an IP and port number knowing the IP and port number it is trying to communicate with. My last struggle is to actually form the connection. I understand there is a distinction between localhost (127.0.0.1) and the public IP for an internet endpoint. Upon searching, I found a way to find the public IP for the current user to share with the database, but I cannot bind to it. Whenever I try to, I get sockets error code 13: permission denied. I would imagine that if I tried connecting to the public IP of the other machine, I would get a similar error (but I cannot test the client until I can get a server running!).
I read online that some router work would be needed to actually form this connection between two machines. I guess I'm struggling to understand the practicality of socket programming if such a simple operation (connecting two socket endpoints on two different computers) requires so much tweaking. Is there something I am missing?
For reference, here is a general outline of my code thus far. The server:
# Server thread
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((LOCAL_IP, AVAILABLE_PORT))
s.listen(1)
# In my code, there is a quitting mechanism which closes s as well.
while True:
client_socket, addr = s.accept()
data = client_socket.recv(1024)
print "Received: " + data
client_socket.close()
...and the client:
# Client thread
# It is an infinite loop so I am always waiting for another potential message to send
while True:
x = raw_input()
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((OTHER_MACHINE_LOCAL_IP, OTHER_PORT))
sock.sendall(x)
sock.close()
Right now, I cannot make progress with these permission denied errors. Is there any way I could get this to work? Understand that, being that this would be for around 250ish students to use who are all intro CS students, I would want to avoid having to instruct them to do anything with their routers.
If there is another method to do this which would make this easier that I am missing, I would also love to hear any suggestions :) Thanks in advance!