Im trying to send 3 packets one after the other with python socket.
Python optimize it to one or 2 packets.
I prevented it by sleep command, but it takes too long time.
I thought to turn on the TCP urg flag, Does someone know how to do it?
or you have another solotion?
client side:
import socket
from time import sleep
IP = '127.0.0.1'
PORT = 5081
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((IP, PORT))
s.send('1'*5)
#sleep( 1)
s.send('2'*5)
#sleep( 1)
s.send('3'*5)
s.close()
server side:
import socket
IP = '0.0.0.0'
PORT = 5081
BUFFER_SIZE = 1024
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((IP, PORT))
s.listen(1)
connection, address = s.accept()
while 1:
#Here I expected to get the 1nd value
data1 = connection.recv(BUFFER_SIZE)
#end of communication
if not data1:
break
print 'data1', data1
#Here I expected to get the 2nd value,but both inputs arrived here, 22222 and 33333
data2 = connection.recv(BUFFER_SIZE)
print 'data2', data2
#Here I expected to get the 3nd value
data3 = connection.recv(BUFFER_SIZE)
print 'data3', data3
connection.close()
thanks
Avinoam
You should not even try. TCP is a stream protocol and should be used as a stream protocol (meaning a single sequence of bytes). Even if you manage to maintain the separation of packets when you use localhost on your system, it could break if you use it between different hosts, or simply after an upgrade of the TCP/IP stack. And as soon as your packets will pass through a proxy or a software filter, anything can happen.
The correct way to separate different objects on a stream is to use an upper level protocol encoding the objects sender side and decoding them client side. An example of that is one or two bytes (in network order if more than one byte) for the size followed by the relevant bytes. Or you could imagine a text protocol with commands, headers and data, or [put whatever you want here]
Related
I am running a simple closing socket listener to ingest a UDP stream containing json directed to port 5001 (in this instance) on localhost:
import socket
import json
from contextlib import closing
def monitor_stream():
with closing(socket.socket(socket.AF_INET, socket.SOCK_DGRAM)) as s:
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(('', 5001))
while True:
data, addr = s.recvfrom(4096)
try:
data = data.decode('ascii')
data = json.loads(data)
except:
print 'Not valid JSON: ', data
I need to re-broadcast the stream to another local port (arbitrarily 5002 in this example) so that a second program can access the data in as near as real time. Low latency is crucial. Is the following (using socket.sendto() within the while) an acceptable method:
while True:
data, addr = s.recvfrom(4096)
s.sendto(data, ('localhost', 5002))
If not, how else might I achieve the same result?
I have assumed that there is no way of multiple programs ingesting the original stream in a simultaneous fashion as they originate as unicast packets so only the first bound socket receives them.
Secondly, how might I cast the same stream to multiple ports (local or not)?
I am unable to change the incoming stream port / settings.
This was a simple misuse of socket.sendto() which can only be called on a non-connected socket.
I was trying to use the bound-listener to send-on the ingested stream.
I seems you need two socket objects to re-broadcast to a different address. The second socket is an un-bound echo client as the address is specified in .sendto(string, address)
I am trying simple client and sinple server , client send 2 massages and server recieve 2 massages . I am printg first massage print(data1) and second massage print(data2) and somehow it print both massages in the same line . Can you help ?
server
import socket
SERVER_IP = '0.0.0.0'
DEST_PORT = 1731
server_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
server_socket.bind((SERVER_IP,DEST_PORT))
server_socket.listen(1)
client_socket,address=server_socket.accept()
data1 = client_socket.recv(512).decode()
print(data1)
data2 = client_socket.recv(512).decode()
print(data2)
client_socket.close()
server_socket.close()
cliend code
#client
import socket
HOST_IP = '127.0.0.1'
DEST_PORT = 1731
my_socket = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
my_socket.connect((HOST_IP,DEST_PORT))
data1 = "test1"
print("client send")
my_socket.sendall(data1.encode())
my_socket.sendall("second send ".encode())
my_socket.close()
TCP sockets are a stream-oriented protocol, not a message-oriented one.
What's happening is that your client ends up sending b"test1second send " before your server reads anything from the socket.
When it does, it reads up to 512 characters – more than enough to contain your message – and again, up to 512 characters yet there are zero to read at that point and an empty line gets printed.
Depending on your actual requirements, you will need to figure out a way to delimit or encapsulate your messages; for text, a popular choice is by newline characters (though it will land you in some hot water should you need to transmit an actual newline in a message).
A better choice still is something like netstrings, where each message is preceded by its length (in ASCII for netstrings, but it could be a binary unsigned integer (4 bytes), or whatever you specify), or some sort of TLV scheme.
I have a client class, in which it should connect to the server (in this case, iKettle) and sends and receives data. The IP address should come from another main server (so another class), and this IP address can change.
Below is the client code:
#command codes
_ON = "0x4<LF>"
_OFF = "0x0<LF>"
#base command
_BASE_COMMAND = "set sys output "
_SLEEP_TIME = 0.5
#size of buffer when receiving data
_BUFFER_SIZE = 1024
ip_address = ""
port = 2000
def initialiseSocket(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
return s
def setOn(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send("set sys output 0x4<LF>")
time.sleep(self._SLEEP_TIME)
self.kettleResponse(ip_address)
def setOff(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send(self._BASE_COMMAND + self._OFF)
time.sleep(self._SLEEP_TIME)
self.kettleResponse(ip_address)
def kettleResponse(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
data = s.recv(self._BUFFER_SIZE)
print(data)
And this is an example of commands from another class:
kettle.setOn(KETTLEIP)
However, when running this code, it does not print anything.
Can anyone help please?
You're probably connecting and reconnecting too much. Sometimes that simplifies things, but in this case I believe it's making you lose your response.
Try just connecting once, and reusing the socket.
Also, keep in mind that TCP is a byte-oriented protocol, not a message-oriented protocol. IOW, if you send 10k, 10k, 10k, the other side of the TCP socket may receive 5k, 8k, 17k - or even more bizarre possibilities. The total number of bytes will be the same, and the data will arrive in the correct order, but the sizes of the chunks could be totally scrambled. For this reason, most folks use REST with http these days - it's simpler in some ways.
If you're married to TCP, perhaps try my bufsock module (or actually, it's Opensource with a Univ Calif Irvine copyright, but I wrote it while I was working for them, and obtained permission from them to release it). It's at http://stromberg.dnsalias.org/~strombrg/bufsock.html . Its methods behave closer to what most people expect out of TCP.
HTH.
My guess is that you should reuse the socket as #dstromberg indicates:
class Kettle:
# other stuff omitted...
def setOn(self,ip_address):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((ip_address, self.port))
s.send("set sys output 0x4<LF>")
time.sleep(self._SLEEP_TIME)
self.kettleResponse(s) # pass the socket on
def kettleResponse(self, s):
# don't open a new socket, just reuse s
data = s.recv(self._BUFFER_SIZE)
print(data)
I'm working with python sockets for a while, and I wrote some simple programs.
The problem that I encountered is related to sending/receiving methods in python sockets.
Giving you a basic example:
This is the receiver (server):
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 4001))
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.listen(5)
while True:
conn, addr = s.accept()
print conn, addr
data1 = conn.recv(64)
data2 = conn.recv(64)
print 'uname is %s , password is: %s' %(data1, data2, )
conn.close()
And this is the sender (or client):
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('', 4001))
uname = raw_input('enter username>')
passw = raw_input('enter password>')
s.send(uname)
s.send(passw)
print 'exiting ...'
s.close()
So the problem is: why server receives both uname and passw in first s.recv() method? It means data2 is always empty!
I have no idea what happens when client executes the s.send() method. I was thinking that each s.send() actually sends a "packet" to the destination (ip, port)!
Can someone explain to me why the second code is working correctly?
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('', 4001))
uname = raw_input('enter username>')
s.send(uname)
passw = raw_input('enter password>')
s.send(passw)
print 'exiting ...'
s.close()
socket.SOCK_STREAM means you're communicating via TCP . This means, if you call send, your data is pushed to the system's networking stack. Both send calls are called shortly one after another.
If you use TCP, your system decides about the packet size. So the uname and passw might be sent in one packet or even split in any other way.
On the receiver's side, data1 receives up to 64 bytes, which is enough for uname and passw.
The explanation above also shows, why the second one works:
Here you need some time between sending uname and passw. During this time, your OS decides to send the packet (i.e. to flush the network buffer).
When you are using streams, you should not think in terms of packets but in terms of streams. There a send call only means: push some data on my network stack(like a pipeline).
If you are interested in packets,
you might try to experiment with UDP:
socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
With such kind of socket your first sender would work as expected
I also faced similar problem. Then I implemented a protocol to send and receive one message at a time. Hope this link will help a lot : LINK
I'm writing a multithreaded distributed networking algorithm.
I've one thread that listens to new connections. Every time a new connection is established a separate thread is started for listening to messages from that connection.
My problem is that the socket I open works perfectly in both directions inside the connection listener. After I pass the socket object for that connection to the message listener I can read data from the socket, but sending data through it doesn't reach the remote host.
Here's the essential snip from my code:
def connection_listener(port, start_e, terminate_e):
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.settimeout(1)
s.bind(('', port))
s.listen(1)
while (not start_e.isSet()):
try:
conn, addr = s.accept()
msg_in = conn.recv(1024).split(":")
if (msg_in[1]=="hello"):
# If addr sends us a 'id:hello', we reply with a 'my_id:welcome'
conn.send(str(my_id)+":welcome")
t = Thread(target=message_listener, args=(conn, addr[0], terminate_e, ))
t.start()
except:
pass # timeout
def message_listener(conn, address, terminate_e):
while (not terminate_e.isSet()):
try:
msg_in = conn.recv(1024)
# Here I can receive everything that I send from the other end of conn,
# but conn.send("any data") doesn't reach the remote host
What I'd like to do is send acknowledgement-like messages from the message listener thread using the conn. Is this possible somehow or am I thinking and doing it wrong?
I sorted this out myself, so I'll share my answer.
I made the protocol exchange fixed size messages by padding with zeroes up to the desired length. I used a length of 32 bytes, which might be quite tiny from the hardware's point of view. Nevertheless it seems to work as supposed.
Pragmatically my solution looks like:
def send_everyone(message):
for i in range(len(peers)):
chunk = (str(my_id)+":"+message).rjust(32, '0')
peers[i].send(chunk)
And on the receiving side we want only 32 bytes at a time:
def message_listener(conn, address, terminate_e):
while (not terminate_e.isSet()):
try:
msg_in = conn.recv(32)
...