Develop Reference

XSUB not receiving in ZeroMQ XSUB/PUB setup

I would like to use XSUB/XPUB to enable multiple ZMQ publishers and subscribers. Everything works when I use zmq.proxy(xpub_socket, xsub_socket), but I need something custom because I need to write code between XSUB and XPUB that examines the messages.
Here's where I'm at:
import time
import zmq
context = zmq.Context()
address = '127.0.0.1'
pub_port = '3000'
sub_port = '3001'
# XSUB socket
xsub_socket = context.socket(zmq.XSUB)
xsub_socket.bind(f'tcp://{address}:{pub_port}')
# XPUB socket
xpub_socket = context.socket(zmq.XPUB)
xpub_socket.bind(f'tcp://{address}:{sub_port}')
time.sleep(1)
# PUB socket
pub_socket = context.socket(zmq.PUB)
pub_socket.connect(f'tcp://{address}:{pub_port}')
# SUB socket
sub_socket = context.socket(zmq.SUB)
sub_socket.subscribe('')
sub_socket.connect(f'tcp://{address}:{sub_port}')
time.sleep(1)
pub_socket.send_string('test')
time.sleep(1)
print(poller.poll(0))
The values sent from the PUB socket do not reach the XSUB socket.
I read here that the first byte needs to be 1. Both of these also don't work:
pub_socket.send(b'\x01')
pub_socket.send_multipart([b'\x01', 'test'.encode('utf-8')])
What am I doing wrong here?

A PUB socket won't send any messages to an XSUB socket unless it has received a subscription request, which you get by calling subscribe on a SUB socket.
The only way those subscription messages get passed through is if you set up your XSUB/XPUB proxy.
Here's a simple proxy that connects an XPUB and XSUB socket, printing out messages it receives in either direction:
import zmq
ctx = zmq.Context()
xpub_sock = ctx.socket(zmq.XPUB)
xpub_sock.bind("tcp://127.0.0.1:3000")
xsub_sock = ctx.socket(zmq.XSUB)
xsub_sock.bind("tcp://127.0.0.1:3001")
poller = zmq.Poller()
poller.register(xpub_sock, zmq.POLLIN)
poller.register(xsub_sock, zmq.POLLIN)
while True:
socks = dict(poller.poll())
if xpub_sock in socks and socks[xpub_sock] == zmq.POLLIN:
msg = xpub_sock.recv_multipart()
print("(sub)", msg)
xsub_sock.send_multipart(msg)
elif xsub_sock in socks and socks[xsub_sock] == zmq.POLLIN:
msg = xsub_sock.recv_multipart()
print("(pub)", msg)
xpub_sock.send_multipart(msg)
If I connect to this with an PUB socket, like this...
import zmq
import time
ctx = zmq.Context()
pub_sock = ctx.socket(zmq.PUB)
pub_sock.connect("tcp://localhost:3001")
while True:
pub_sock.send_string("test")
time.sleep(1)
...I won't see any messages arriving at the XSUB socket, because
there are no active subscriptions. However, if I connect a SUB
socket to the XPUB socket and set a subscription...
import zmq
ctx = zmq.Context()
sub_sock = ctx.socket(zmq.SUB)
sub_sock.connect("tcp://localhost:3000")
sub_sock.subscribe("")
while True:
msg = sub_sock.recv()
print(msg)
...then I will start to see messages passing from the PUB socket to
the XSUB socket, and then from the XPUB socket to the SUB
socket.

Why send message in diffrent time but receive mssage in the same time when use zmq delear/delear model

When I use DEALER-DEALER model to communicate, I find: server send messages in different time, but client receive messages in the same. For example (Python code):
# server
context = zmq.Context()
sock_conn = self.context.socket(zmq.DEALER)
sock_conn.bind("tcp://*:%d" % port)
sock_conn.send("msg1")
time.sleep(0.05)
sock_conn.send("msg2")
# client
context_send = zmq.Context()
sock_send = self.context.socket(zmq.DEALER)
sock_send.connect("tcp://%s:%d" % (ip, port1))
while True:
msg = sock_send.recv()
print(msg)
The server sends the message in different time, but sometimes the client receives the messages at the same time, who can explain this problem, thanks.

Unable to receive multiple clients data using ZeroMQ

I am executing this program of server, receiving from multiple clients, but after the first message for one client, the server is receiving the data and then it does not wait for other client request and just display the first clients data and stops.
Also, I am unable to be a client from other VM, which uses the same linux machine. I am not sure how to modify it in order to let the server receive from more clients.
1.I am running the same client program in different VM-s and trying to send the data to the server. Only one client message is delivered and other client is unable to send and just waits.
Thanks
import zmq
import json
ports = ["192.168.1.24:10000"]
context = zmq.Context()
print("Connecting to server")
socket = context.socket(zmq.DEALER)
socket.setsockopt(zmq.LINGER, 0)
for port in ports:
socket.connect("tcp://%s" % port)
print("Successfully connected to server %s" % port)
for request in range(len(ports)):
print("Sending request ", request, "...")
socket.send_string("", zmq.SNDMORE)
socket.send_string("Sensor Data")
# use poll for timeouts:
poller = zmq.Poller()
poller.register(socket, zmq.POLLIN)
socks = dict(poller.poll(5 * 1000))
if socket in socks:
try:
socket.recv() # discard delimiter
msg_json = socket.recv() # actual message
sens = json.loads(msg_json)
response = "am: %s :: pam: %s :: dam: %s" % (sens['a'], sens['b'], sens['c'])
print("Received reply ", request, "[", response, "]")
except IOError:
print("Could not connect to machine")
else:
print("Machine did not respond")
Client:
import zmq
import time
import json
port = "10000"
context = zmq.Context()
socket = context.socket(zmq.REP)
socket.bind("tcp://*:%s" % port)
while True:
# Wait for next request from server
message = str(socket.recv(), "utf-8")
print("Received request: ", message)
time.sleep(1)
msgDict = {
'am': "990",
'pam': "11",
'dam': "1",
}
ms = json.dumps(msgDict)
socket.send_string(ms)

There are few details that seem not to be well orchestrated inside your flow of events:
Given your code, the client-side makes a single .send_string( json.dumps( msgDict ) )
but on the server-side, your code does two consecutive .recv()-s, both of which are blocking.
So what happens is that after having read-out a message from the first .recv() read, the server steps right into a second, again blocking .recv(), where it remains waiting infinitely long ( as instructed ) in all cases no other message was delivered from distributed-system external worlds to the hands of your local Context() instance. In any such case, the blocking-form of the .recv() simply never returns.
If in doubts, whether there is anything at all to read, the POLLIN event is the conditio sine qua non, and using another test, until a call to socket.getsockopt( zmq.RCVMORE ) returns False, upon all first received message multipart-elements ( if any present ) were already depleted.
Given your client-code opts to use a REQ-archetype, it will never send any message, without having first been asked to do so from server. This if your server-code is hanging inside the above explained ( conceptually ill-defined ) blocking .recv(), it will never ask "another" REQ-client to reply.

ZMQ: REQ/REP fails with multiple concurrent requests and polling

I have run into a strange behaviour with ZeroMQ that I have been trying to debug the whole day now.
Here is a minimal example script which reproduces the problem. It can be run with Python3.
One server with a REP socket is started and five clients with REP sockets connect to it basically simultaneously. The result is that the server starts to block for some reason after the first few messages. It seems like the poller.poll(1000) is what blocks indefinitely.
This behavior also seems to be timing-dependant. Insert a sleep(0.1) in the loop that starts the clients and it works as expected.
I would have expected the REP socket to queue all incoming messages and release them one after the other via sock.recv_multipart().
What is happening here?
import logging
from threading import Thread
from time import sleep
import zmq
logging.basicConfig(level=logging.INFO)
PORT = "3446"
stop_flag = False
def server():
logging.info("started server")
context = zmq.Context()
sock = context.socket(zmq.REP)
sock.bind("tcp://*:" + PORT)
logging.info("bound server")
poller = zmq.Poller()
poller.register(sock, zmq.POLLIN)
while not stop_flag:
socks = dict(poller.poll(1000))
if socks.get(sock) == zmq.POLLIN:
request = sock.recv_multipart()
logging.info("received %s", request)
# sleep(0.5)
sock.send_multipart(["reply".encode()] + request)
sock.close()
def client(name:str):
context = zmq.Context()
sock = context.socket(zmq.REQ)
sock.connect("tcp://localhost:" + PORT)
sock.send_multipart([name.encode()])
logging.info(sock.recv_multipart())
sock.close()
logging.info("starting server")
server_thread = Thread(target=server)
server_thread.start()
sleep(1)
nr_of_clients = 5
for i in range(nr_of_clients):
Thread(target=client, args=[str(i)]).start()
stop_flag = True

For me the problem seems to be that you are "shutting down" the server before all clients have received their reply. So I guess its not the server who's blocking but clients are.
You can solve this by either waiting some time before you set the stop_flag:
sleep(5)
stop_flag = True
or, better, you explicitely join the client threads like:
nr_of_clients = 5
threads = []
for i in range(nr_of_clients):
thread = Thread(target=client, args=[str(i)])
thread.start()
threads.append(thread)
for thread in threads:
thread.join()
stop_flag = True

How to process concurrent client requests?

Here is sample code of the request / response pattern with zeroMQ in python. I would like to know if there is way to process requests from multiple clients concurrently?
import zmq
import time
def main():
context = zmq.Context()
serverSocket = StartServer(context,"9999")
processRequests(serverSocket)
def processRequests (socket):
while True:
print "waiting for request"
msg = socket.recv()
print msg
time.sleep(10)
socket.send("Request processed")
def StartServer(context, port):
socket = context.socket(zmq.REP)
socket.bind("tcp://*:%s" % port)
print "started server on", port
return socket
if __name__ == '__main__':
print "starting IPC server"
main()

The REQ-REP pattern is a synchronous pattern. If there are two REQ sockets connected to the same REP socket, the REP socket will process requests serially.
If you want to do asynchronous request-reply, you'll want to look into the ROUTER-DEALER pattern, which is the generalized analogue of REQ-REP.
If you want a brokered asynchronous request-reply, look at the "Figure 16 - Extended Request-Reply" section here.