Develop Reference

How to receive data and use it on if () statement?

i have tried to receive data form my connection by this code on python 2.7 :
server = socket(AF_INET, SOCK_STREAM)
server.bind(('0.0.0.0', 21))
server.listen(1)
client , addr = server.accept()
data = client.recv(2048)
When i have to print or send to another connection my data it's working , however i want to add those lines :
if(data == "/disconnect") :
<disconnect blha blha... you know >
else :
<print the data and send it back blha blha... >
( i have checked without that if statement and " disconnect blha blha .. " works nicely )
it's just pass the code so when my client requests to disconnect the request is sent to the server as the "original" messages .. (the sever don't kick him)
what should i do ? Thanks !

You have two problems, and you need to fix both of them.
First, a TCP socket is just a stream of bytes. When you do a recv, you're not going to get exactly one message sent by send from the other side, you're going to get whatever's in the buffer at the moment—which could be two messages, or half a message, or anything else. When you're just testing with localhost connections on a computer that isn't heavily loaded, on many platforms, it will do what you're hoping for >99% of the time—but that just makes the problem hard to debug, it doesn't fix it. And as soon as you try to access the same code over the internet, it'll start failing most of the time instead of rarely.
Fortunately, the client appears to be sending messages as text, without any embedded newlines, with a \r\n Windows-style end-of-line between each message. This is a perfectly good protocol; you just have to write the code to handle that protocol on the receive side.
The second problem is that, even if you happen to get exactly one message send, that message includes the \r\n end-of-line. And '/disconnect\r\n' == '/disconnect' is of course going to be false. So, as part of your protocol handler, you need to strip off the newlines.
As it happens, you could solve both problems by using the makefile method to give you a file object that you can iterate, or readline on, etc., just like you do with a file that you open from disk, which you probably already know how to handle.
But it's worth learning how to do this stuff, so I'll show you how to do it manually. The key is that you keep a buffer around, add each recv onto that buffer, and then split it into lines, put any remainder back on the buffer, and process each line as a message. There are more elegant/concise ways to write this, but let's keep it dead simple:
buf = ''
while True:
data = client.recv(2048)
buf += data
lines = buf.split('\r\n')
buf = lines.pop()
for line in lines:
# line is now a single message, with the newline stripped
if line == "/disconnect":
# do disconnect stuff
else:
# do normal message stuff
That's all you need to get the basics working. But in a real server, you also need some code to handle two other conditions—because clients don't always shut down cleanly. For example, if a client gets disconnected from the internet before it can send a /disconnect message, you don't want to keep spinning and reading nothing forever, you want to treat it as a disconnect.
if not data: means the client has done a clean (at the TCP level) shutdown. So, you need to disconnect and break out of the receive loop.
Depending on your design, it may be legal to shutdown only the send side and wait for a final reply from the server, so you want to make sure you've finished sending whatever you have. (This is common in many internet protocols.)
It may even be legal to not send a final newline before shutting down; if you want to support this, you should check if buf: and if so, treat buf as one last command. (This is not common in many protocol—but is a common bug in clients, so, e.g., many web servers will handle it.)
try:/except Exception as e: will catch all kinds of errors. These errors mean the socket is no longer usable (or that there's a serious error in your code, of course), so you want to handle this by throwing away the connection and breaking out of the receive loop, without first sending any final response or reading any final message.
It's almost always worth logging that e in some way (maybe just print 'Error from', addr, repr(e)), so if you're getting unexpected exceptions you have something to debug.

How does the python socket.recv() method know that the end of the message has been reached?

Let's say I'm using 1024 as buffer size for my client socket:
recv(1024)
Let's assume the message the server wants to send to me consists of 2024 bytes.
Only 1024 bytes can be received by my socket. What's happening to the other 1000 bytes?
Will the recv-method wait for a certain amount of time (say 2 seconds) for more data to come and stop working after this time span? (I.e., if the rest of the data arrives after 3 seconds, the data will not be received by the socket any more?)
or
Will the recv-method stop working immediately after having received 1024 bytes of data? (I.e. will the other 1000 bytes be discarded?)
In case that 1.) is correct ... is there a way for me to to determine the amount of time, the recv data should wait before returning or is it determined by the system? (I.e. could I tell the socket to wait for 5 seconds before stopping to wait for more data?)
UPDATE:
Assume, I have the following code:
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((sys.argv[1], port))
s.send('Hello, world')
data = s.recv(1024)
print("received: {}".format(data))
s.close()
Assume that the server sends data of size > 1024 bytes. Can I be sure that the variable "data" will contain all the data (including those beyond the 1024th byte)?
If I can't be sure about that, how would I have to change the code so that I can always be sure that the variable "data" will contain all the data sent (in one or many steps) from the server?

It depends on the protocol. Some protocols like UDP send messages and exactly 1 message is returned per recv. Assuming you are talking about TCP specifically, there are several factors involved. TCP is stream oriented and because of things like the amount of currently outstanding send/recv data, lost/reordered packets on the wire, delayed acknowledgement of data, and the Nagle algorithm (which delays some small sends by a few hundred milliseconds), its behavior can change subtly as a conversation between client and server progresses.
All the receiver knows is that it is getting a stream of bytes. It could get anything from 1 to the fully requested buffer size on any recv. There is no one-to-one correlation between the send call on one side and the recv call on the other.
If you need to figure out message boundaries its up to the higher level protocols to figure that out. Take HTTP for example. It starts with a \r\n delimited header and then has a count of the remaining bytes the client should expect to receive. The client knows how to read the header because of the \r\n then knows exactly how many bytes are coming next. Part of the charm of RESTful protocols is that they are HTTP based and somebody else already figured this stuff out!
Some protocols use NUL to delimit messages. Others may have a fixed length binary header that includes a count of any variable data to come. I like zeromq which has a robust messaging system on top of TCP.
More details on what happens with receive...
When you do recv(1024), there are 6 possibilities
There is no receive data. recv will wait until there is receive data. You can change that by setting a timeout.
There is partial receive data. You'll get that part right away. The rest is either buffered or hasn't been sent yet and you just do another recv to get more (and the same rules apply).
There is more than 1024 bytes available. You'll get 1024 of that data and the rest is buffered in the kernel waiting for another receive.
The other side has shut down the socket. You'll get 0 bytes of data. 0 means you will never get more data on that socket. But if you keep asking for data, you'll keep getting 0 bytes.
The other side has reset the socket. You'll get an exception.
Some other strange thing has gone on and you'll get an exception for that.

How to make an endmark in python TCP socket calls send() and recv()

I am relatively new to sockets and very new to python. How would you go about making an endmark for python send() and recv().
I have searched all over and there is no easy tutorial. I have read the man page for recv(2) a thousand times and it ironically makes less sense to me each time I read it.
I would like to use the send() function in a server to let the client calling recv() know when the end of the send() is.
Do you use the flag argument of send?
Or do you use something like "|".join(str1, str2) and use an if statement in the client to recognize the | and parse the statement?

TCP is not message oriented protocol. It does not maintain message boundaries nor it does not help in other ways to achieve it. It is up-to the application to mark boundaries. Client and server can agree upon a method in exchange of data. Common method is to put in the message length in the data you send.
[2 byte message length][Actual Data of Interest]
The end which receives packets will always look for two byte length indicator. recv as much data indicated by length bytes, process them and again go to recv length bytes and so on.
Another method is that the application can mark the start and end of the message with markers. It also needs to handle cases where the markers can also be part of actual data.
[Start Indicator][Actual Data of Interest][ End Indicator]

Can the python client tcp socket be used for multiple sends?

I tried using a socket for 2 sends. The first one succeeds and the next one does not.
From the http://docs.python.org/howto/sockets.html
it would appear that multiple sends should be allowed. For Better or worse, I don't really need to read from the socket.
I have used twisted, but for the present purpose, I would like to stick to a socket, if I can help it(partly because I am using it within an application already using twisted to communicate.. this is a seperate connection).
"When the connect completes, the socket s can be used to send in a request for the text of the page. The same socket will read the reply, and then be destroyed. That’s right, destroyed. Client sockets are normally only used for one exchange (or a small set of sequential exchanges)."
return value for the send that succeeds = 35
return value for the send that FAILS = 32
code with some minor editing to remove any business logic.
self._commandSock = socket.socket(socket.AF_INET,socket.SOCK_STREAM)
def sendPrereqs(self,id,prereqs):
self._commandSock.connect(self._commandConnection)
#parse prereqs
temp = prereqs.split(',')
for pair in temp:
tup = pair.partition(':')
try:
command = 'some command'
logging.info('sending command: ' + command)
ret = self._commandSock.send(command)
if ret == None:
logging.info('send called successfully: ' + command)
else:
logging.info('socket returned non-None: ' + str(ret))
except:
print 'Unexpected Exception ', sys.exc_info()[0]()
print sys.exc_info()
#logging.info('Unexpected Exception '+ str(sys.exc_info()[0]()))
#logging.info(' ' + str(sys.exc_info()))
self._commandSock.close()`

return value for the send that succeeds = 35 return value for the send that FAILS = 32
Documentation says that successful send should return None.
No it doesn't. Documentation says:
Returns the number of bytes sent. Applications are responsible for checking that all data has been sent; if only some of the data was transmitted, the application needs to attempt delivery of the remaining data. For further information on this concept, consult the Socket Programming HOWTO.
You still haven't explained what you mean by "FAILS". The send call is returning successfully, and it's almost certainly placed 32 bytes into the socket write buffer.
If the only reason you think it's failing is that it returns the correct value, then the answer is obviously that it's not failing.
If something else is going wrong, there are all kinds of things that could be wrong at a higher level. One likely one is this: The server (especially if it was coded by someone who doesn't understand sockets well) is coded to expect one recv() of 35 bytes, and one recv() of 32 bytes. But it's actually getting a single recv() of 67 bytes, and waiting forever for the second, which never comes. There is no rule guaranteeing that each send() on a stream (TCP) socket corresponds to one recv() on the other side. You need to create some kind of stream protocol that demarcates the separate messages.
Meanwhile, the quote you're referring to is irrelevant. It's describing how client sockets are used by simple web browsers: They make a connection, do one send, receive all the data, then destroy the connection. I can see why it misled you, and you may want to file a documentation bug to get it improved. But for now, just ignore it.
If you want to make absolutely sure that the client is sending the right data, and the problem is in the server, there are two easy ways to do that:
Use netcat as a trivial substitute server (e.g., nc -kl 6000, replacing the "6000" with the actual port) and making sure it logs what you think the server should be seeing.
Use Wireshark to watch the connection between the client and server.
Once you've verified that the problem is on the server side, you need to debug the server. If you need help with that, that's probably best done in a new question, where you can post the server code instead of the client, and explain (with a link here) that you're sure the client is sending the right information.

The documentation is only referring to a common scenario. You can call send, sendall, and sendto on all sockets as often as you want - as long as the socket is not closed.
Note that these methods return the number of bytes sent, 32 and 35 simply mean you sent 32 bytes the first and 35 bytes the second time.
The fact that socket.send returns without an exception means that the data got handed to the operating system, but not that it actually reached the endpoint (or has been read correctly by an application there).

"Message Must be multiple of 16" Encrypted Audio over the Network with Python

I am sending encrypted audio through the network in Python. This app works momentarily then breaks saying it must be a multiple of 16.
Not sure what I am doing wrong. Or where to look in the code to solve this.
I would appreciate any help you have to offer
EDIT * I believe I have it working now if anyone is interested in taking a look I made a google code project
http://code.google.com/p/mii-chat/

msg = conn.recv(2024)
if msg:
cmd, msg = ord(msg[0]),msg[1:]
if cmd == CMD_MSG:
listb1.insert(END, decrypt_my_message(msg.strip()) + "\n")
The snippet above from your code reads 2024 bytes of data (which isn't a multiple of 16) and then (if the "if" statements are True) then calls decrypt_my_message with msg.strip() as the argument. Then decrypt_my_message complains that it was given a string whose length wasn't a multiple of 16. (I'm guessing that this is the problem. Have a look in the traceback to see if this is the line that causes the exception).
You need to call decrypt_my_message with a string of length n*16.
You might need to rethink your logic for reading the stream - or have something in the middle to buffer the calls to decrypt_my_message into chunks of n*16.

I did a quick scan of the code. All messages are sent after being encrypted, so the total data you send is a multiple of 16, plus 1 for the command. So far, so good.
On the decrypting side, you strip off the command, which leaves you with a message that is a multiple of 16 again. However, you are calling msg.strip() before you call decrypt_my_message. It is possible that the call to strip corrupts your encrypted data by removing bytes from the beginning or the end.
I will examine the code further, and edit this answer if I find anything else.
EDIT:
You are using space character for padding, and I suppose you meant to remove the padding using the strip call. You should change decrypt_my_message(msg.strip()) to decrypt_my_message(msg).strip().
You are using TCP to send the data, so your protocol is bound to give you headaches in the long term. I always send the length of the payload in my messages with this sort of custom protocol, so the receiving end can determine if it received the message block correctly. For example, you could use: CMD|LEN(2)|PAYLOAD(LEN) as your data frame. It means, one byte for command, two more bytes to tell the server how many bytes to expect, and LEN bytes of actual message. This way, your recv call can loop until it reads the correct amount. And more importantly, it will not attempt to read the next packet when/if they are sent back-to-back.
Alternatively, if your packets are small enough, you could go for UDP. It opens up another can of worms, but you know that the recv will only receive a single packet with UDP.