I have a first sender script in Python 3.10 which needs to send some data
def post_updates(*args):
sender.send_message("optional_key", args)
Then a second receiver script in Python 3.7 which needs to receive this data
while True:
args = receiver.get_message("optional_key", blocking=True)
print("args received:", args)
Constraints:
Each script should not depend on the presence of the other to run.
The sender should try to send regardless if the receiver is running.
The receiver should try to receive regardless if the sender is running.
The message can consist of basic python objects (dict, list) and should be serialized automatically.
I need to send over 100 messages per second (minimizing latency if possible).
Local PC only (Windows) and no need for security.
Are there 1-liner solutions to this simple problem? Everything I look up seems overly complicated or requires a TCP server to be started beforehand. I don't mind installing popular modules.
UDP and JSON look perfect for what you're asking for, as long as
you don't need there to be more than one receiver
you don't need very large messages
you just need to send combinations of dicts, lists, strings, and numbers, not Python objects of arbitrary classes
you're not being overly literal about finding a "1-liner": it's a very small amount of code to write, and you're free to define your own helper functions.
Python's standard library has all you need for this. Encoding and decoding from JSON is as simple as json.dumps() and json.loads(). For sending and receiving, I suggest following the example on the Python wiki. You need to create the socket first with
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
regardless of if you're making the sender or the receiver. The receiver will then need to bind to the local port to listen to it:
sock.bind(('127.0.0.1', PORT))
And then the sender sends with sock.sendto() and the receiver receives with sock.recvfrom().
The good old pipe might do the job, but you need to assess how big the buffer size needs to be (given the async nature of your sender/receiver), and change the default pipe buffer size.
Related
I'm currently trying to figure out how to send OSC messages from Python to Max/MSP. I'm currently using osc4py3 to do so, and I have a sample code from the documentation that should hypothetically be working, written out here:
from osc4py3.as_eventloop import *
from osc4py3 import oscbuildparse
# Start the system.
osc_startup()
# Make client channels to send packets.
osc_udp_client("127.0. 0.1", 5000, "tester")
msg = oscbuildparse.OSCMessage("/test/me", ",sif", ["text", 672, 8.871])
osc_send(msg, "tester")
The receiver in Max is just a udprecieve object listening to port 5000. I managed to get Processing to send OSC messages to Max and it worked pretty simply using the oscp5 library, but I can't seem to have the same luck in Python.
What is it I'm missing? Moreover, I don't entirely understand the structure for building OSC messages in osc4py3, even after doing my best with the documentation; if someone would be willing to explain what exactly is going on (namely, the arguments) in something like
msg = oscbuildparse.OSCMessage("/test/me", ",sif", ["text", 672, 8.871])
then I would be forever grateful.
I'm entirely open to using another OSC library, but all I ask is a run-through on how to send a message (I've attempted using pyOSC but that too proved too confusing for me).
Maybe you already solved it but in the posted code there are two problems. One is the IP address format (there is a space before the second "0"). Then you need the command osc.process() at the end. So the following way should work
from osc4py3.as_eventloop import *
from osc4py3 import oscbuildparse
# Start the system.
osc_startup()
# Make client channels to send packets.
osc_udp_client("127.0.0.1", 5000, "tester")
msg = oscbuildparse.OSCMessage("/test/me", ",sif", ["text", 672,
8.871])
osc_send(msg, "tester")
osc_process()
Hope it will work out
There are different possible scheduling policies in osc4py3. The documentation uses the event-loop model with as_eventloop, where user code must periodically call osc_process() to have osc4py3 deal with internal messages queues and communications.
The client example for sending OSC messages wrap osc_process() call in a loop (generally it is into an event processing loop).
You may dismiss osc_process() call simply by importing names with full multithreading scheduling policy at the beginning of your code:
from osc4py3.as_allthreads import *
The third scheduling policy is as_comthreads, where communications are processed in background threads, but received messages (in server side) are processed synchronously at the osc_process() call.
(by the author of osc4py3)
I am currently implementing a socket server using Python's socketServer module. I am struggling to understand how a client 'signals' the server to perform certain tasks.
As you can tell, I am a beginner in this area. I have looked at many tutorials, however, these only tell you how to perform singular tasks in the server e.g. modify a message from the client and send it back.
Ideally what I want to know is there a way for the client to communicate with the server to perform different kinds of tasks.
Is there a standard approach to this issue?
Am I even using the correct type of server?
I was thinking of implementing some form of message passing from the client that tells the server which task it should perform.
I was thinking of implementing some form of message passing from the client that tells the server which task it should perform.
That's exactly what you need: an application protocol.
A socket (assuming a streaming Internet socket, or TCP) is a stream of bytes, nothing more. To give those bytes any meaning, you need a protocol that determines which byte (or sequence thereof) means what.
The main problem to tackle is that the stream that such a socket provides has no notion of "messages". So when one party sends "HELLO", and "BYE" after that, it all gets concatenated into the stream: "HELLOBYE". Or worse even, your server first receives "HELL", followed by "OBYE".
So you need message framing, or rules how to interpret where messages start and end.
You generally don't want to invent your own application protocol. Usually HTTP or other existing protocols are leveraged to pass messages around.
I am currently working on a small project experimenting with different regions of python. I decided to make a multi-client TCP server in python, and wanted to be able to send a "packet" through the server, it be received by the other clients then parsed. However, I get an error if I try to send the packet, saying I must send either bytes or a string, is there a way to convert the object into bytes, then back or send the packet object through the server itself.
## EDIT ##
I have researched into UDP servers, and I do not believe that is what I am looking for, I probably provided too little information. I am creating a small trial game, for me and some friends to mess about on, I need there to be a constant connection to the server, as information is going to be constantly being sent across the network such as location,direction,speech,inventory etc. and I wanted a way to turn the entire Entity class into a byte array then send that and it be turned back into an instance of the Entity class when it was received.
You could use pickle to serialize/deserialize objects to strings and back. https://docs.python.org/2/library/pickle.html
The simplest possible approach would be to send (gzipped?) JSON'd or msgpack'd objects.
For example, using UDP, this could look something like the below code; note that you would want to reuse the socket object rather than instantiating a new one every time.
import socket
import msgpack
def send_object(obj=None, ip, port):
if obj:
sock = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
sock.sendto(msgpack.dumps(obj), (ip, port))
I'm working on a really basic "image streaming" server as a school subject, and I've done most of the work but I'm still stuck on the separation between data and control related sockets:
My structure is : TCPServer (my server, used as control socket) contains a dataSocket (only used to send images and initialized within my TCPServer object, when I receive a certain query)
When I'm sending data (images) through my dataSocket, I still need to see if the client sent a PAUSE or STOP request, but if I use python's self.request.recv(1024) the server awaits a response instead of continuing to send data (which is quite logical).
What should I do to prevent this behavior ? Should I launch my recv(1024) on a separate thread and run it at each loop (and check if I get any relevant data in between two iterations) ?
Twisted should do the trick! It handles asynchronous sockets in Python
I'm trying to send a message through ICMP packets but I don't know how to do it.
This is the code I currently have, but obviously doesn't work:
s = socket(AF_INET, SOCK_RAW, IPPROTO_ICMP)
s.setsockopt(IPPROTO_IP, IP_HDRINCL, 1)
s.settimeout(3.0)
s.sendto("Hello!" + "\r\n", (server, 7))
msg = s.recvfrom(buff_size)
s.close()
I have to receive an answer from server if string "Hello!" is sent, but I don't get it.
I suppose, that "Hello!" string will be encapsulated into Data field:
In order to construct an ICMP packet, you have to create the whole packet yourself using a raw socket. The struct module is useful for this.
Secondly, in order to even use raw sockets in the first place, you need to have permission to do so—you should be running as root (I know this is a sufficient condition, but I'm not 100% certain that it's a necessary condition). The ping(1) executable is able to do this because it's a setuid executable that runs as root when you run it. Since scripts cannot be made setuid on Linux, you'll have to make a wrapper setuid program in C that just executes your Python script.
I don't think that SOCK_RAW is going an ICMP datagram for you just because you set the protocol field to IPPROTO_ICMP! You have to construct the packet yourself.
Take a look at the source of ping.
There are (at least) two popular packages that provide ping in GNU/Linux operating systems. One is netkit and the other iputils. (netkit-combo is a tarball which has all the netkit utilities in one: telnet, FTP, ...) The *BSD guys probably have their own.