I have .NET client/server applications that communicate with a line protocol using TcpClient and an SSLStream. Basically they exchange some information and then enter a heartbeat loop where the client periodically sends a message to the server and receives an answer.
This works well and can do so for days, but sometimes the connection breaks with a Socket error code of either 10054 or 10060, saying the host is unresponsive.
For testing I got rid of most code and tried it with the very minimal components.
On the server I use a TcpListener and run this code for new clients in a new task (please note that I have stripped logging code from the example):
Private Sub HandleClient(c As TcpClient)
c.NoDelay = True 'I tried with and without this algorithm
Dim stream = c.GetStream
Dim buffer(1024) As Byte
c.ReceiveTimeout = 60000
Do
Dim value As Integer = 0
Try
value = stream.Read(buffer, 0, buffer.Length)
Catch ex As Exception
End Try
For i = 0 To value - 1
buffer(i) = 0 'Just to get rid of residuals for error checking
Next
stream.Write({1, 45, 26}, 0, 3)
Loop
End Sub
Most is just logging, the values I send are just some random numbers. So here I read the values from a NetworkStream and write some numbers back. Note: In the real application I use WriteLine and ReadLine.
The client code is equally short, it is also started in a new task:
Private Sub Connect()
'Get the ip to connect to from the command line, or use localhost
Dim ip = If(My.Application.CommandLineArgs.Count > 0, My.Application.CommandLineArgs(0), "127.0.0.1")
log("Trying to parse " & ip)
Dim parsed As IPAddress = Nothing
If IPAddress.TryParse(ip, parsed) = False Then parsed = IPAddress.Parse("127.0.0.1")
Dim c As New TcpClient()
c.NoDelay = True
c.Connect(parsed, 16543)
Dim stream = c.GetStream
Dim buffer(1024) As Byte
Do
stream.Write({33, 75, 123, 3}, 0, 4)
Dim value As Integer
Try
value = stream.Read(buffer, 0, buffer.Length)
Catch ex As Exception
End Try
For i = 0 To value - 1
buffer(i) = 0 'Just to get rid of residuals for error checking
Next
'Write a message every 5 seconds
Threading.Thread.Sleep(5000)
Loop
End Sub
I run this alongside my normal client and server applications and it works flawlessly but disconnects at the exact same time as my normal client/server connection. The quick conclusion is that there is a problem with the internet connection on this device.
However, I also adapted a small python script to do basically the some thing, it connects and exchanges Hello World strings:
Server:
import socket
import time
from datetime import tzinfo, timedelta, datetime
TCP_IP = 'this server'
TCP_PORT = 7777
BUFFER_SIZE = 20
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind((TCP_IP, TCP_PORT))
s.listen(1)
conn,addr = s.accept()
while 1:
while 1:
data = conn.recv(BUFFER_SIZE)
if not data: break
conn.send("World!")
conn.close()
Client:
import socket
import time
from datetime import tzinfo, timedelta, datetime
TCP_IP = 'my server'
TCP_PORT = 7777
BUFFER_SIZE = 1024
MESSAGE = "Hello"
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((TCP_IP, TCP_PORT))
while 1:
s.send(MESSAGE)
data = s.recv(BUFFER_SIZE)
time.sleep(5)
s.close()
I run this as well on the same machines and it has no problem whatsoever. It just keeps on sending, so I guess the internet connection is ok.
The client is behind a NAT router while the server is running on a vServer off-site.
So, to summarize:
Two .NET client/server applications experience disconnects at the same time, at random intervals (the can run for hours before a disconnect happens).
A python client/server application that seems to do the same task does not experience disconnects.
I am truly out of ideas what the cause of this problem may be.
Things I have tried are:
Using Async read/write operations instead of synchronous, enabling/disabling Nagle's algorithm (NoDelay), sending raw bytes instead of using WriteLine/ReadLine.
It may be machine or connection dependent, as two other test clients on other connections seem to be running stable for the last few days. But it also seems implementation dependent, because the Python script works forever.
Sorry for the wall of text/code, I'm grateful for any idea about where I went wrong.
Related
I have the following problem: I want a sever to send the contents of a textfile
when requested to do so. I have writen a server script which sends the contents to the client and the client script which receives all the contents with a revcall loop. The recvall works fine when
I run the server and client from the same device for testing.
But when I run the server from a different device in the same wifi network to receive the textfile contents from the server device, the recvall doesn't work and I only receive the first 1460 bytes of the text.
server script
import socket
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(("", 5000))
server.listen(5)
def send_file(client):
read_string = open("textfile", "rb").read() #6 kilobyte large textfile
client.send(read_string)
while True:
client, data = server.accept()
connect_data = client.recv(1024)
if connect_data == b"send_string":
send_file(client)
else:
pass
client script
import socket
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
client.connect(("192.168.1.10", 5000))
connect_message = client.send(b"send_string")
receive_data = ""
while True: # the recvall loop
receive_data_part = client.recv(1024).decode()
receive_data += receive_data_part
if len(receive_data_part) < 1024:
break
print(receive_data)
recv(1024) means to receive at least 1 and at most 1024 bytes. If the connection has closed, you receive 0 bytes, and if something goes wrong, you get an exception.
TCP is a stream of bytes. It doesn't try to keep the bytes from any given send together for the recv. When you make the call, if the TCP endpoint has some data, you get that data.
In client, you assume that anything less than 1024 bytes must be the last bit of data. Not so. You can receive partial buffers at any time. Its a bit subtle on the server side, but you make the same mistake there by assuming that you'll receive exactly the command b"send_string" in a single call.
You need some sort of a protocol that tells receivers when they've gotten the right amount of data for an action. There are many ways to do this, so I can't really give you the answer. But this is why there are protocols out there like zeromq, xmlrpc, http, etc...
I am trying to implement a simple chat program that uses sockets to transmit data via a UDP connection. However, I can't figure out how to correctly set it up so that people from outside my local network can access it if I am hosting it on my laptop. I am utilizing port 5000, and have port-forwarded that port on my router for my laptop. The port-forwarding doesn't seem to be the issue; at least the "Port Forward Network Utilities" from portforward.com seems to detect it as properly forwarded. Maybe I am mixing up the IP addresses I need to host from and connect with? The code in question is below:
import socket
import threading
import sys
class Server:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
connections = []
def __init__(self):
self.sock.bind(('192.168.1.5', 5000))
self.sock.listen(1)
def handler(self, c, a):
while True:
data = c.recv(1024)
for connection in self.connections:
print(data.decode())
connection.send(data)
if not data:
break
def run(self):
while True:
c, a = self.sock.accept()
cThread = threading.Thread(target=self.handler, args=(c, a))
cThread.daemon = True
cThread.start()
self.connections.append(c)
print(self.connections)
class Client:
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
usr_name = ""
def sendMsg(self):
while True:
self.sock.send(bytes(self.usr_name + ": " + input("-> "), 'utf-8'))
def __init__(self, address):
self.sock.connect((address, 5000))
self.usr_name = input("Enter a username: ")
iThread = threading.Thread(target=self.sendMsg)
iThread.daemon = True
iThread.start()
while True:
data = self.sock.recv(1024)
if not data:
break
print(data.decode())
if len(sys.argv) > 1:
client = Client(sys.argv[1])
else:
server = Server()
server.run()
As you can see, I have my current local IP address inputted for hosting the server, while the client asks for an IP to connect to. I'm not sure what to do now for hosting this over the internet, but I have tried every IP combination I can think of and it returns a number of errors.
Thanks in advance.
Edit:
The two main errors I was getting are:
Timeout Error [WinError 10060]
My friend received this when trying to connect from another network
[WinError 10061]
I would receive this when trying to connect using my public IP from the same computer
I'm sorry that I can't be more detailed in my errors and provide a full printout, and I will try to update this if I'm able to replicate them.
Edit:
I was able to rewrite it and get it to work, I don't need anymore help with this.
Thanks.
You're port-forwarding UDP port 5000 to 5000.
But you're opening TCP streams, not UDP. That's what SOCK_STREAM means. If you want UDP, you need to use SOCK_DGRAM.
So, you need to make these two consistent. The only problem is, I'm not sure which one you actually want here.
On the one hand, your code is doing connection-oriented recv, and seems to be assuming reliable transmission, which means you probably want TCP.
On the other hand, your code seems to be assuming that each recv(1024) is going to get exactly one send from the other side, which is only true for UDP; TCP sockets are byte streams, not message streams. When you do a recv(1024), you could easily get just the first 12 bytes of an 80-byte line, which means it could end in the middle of a UTF-8 character, which means decode will throw an exception.
I think you want TCP, but with a framing protocol on top of it. The simplest protocol that would probably make sense here is lines of text. Which is pretty easy to do on your own, but even easier to do with socket.makefile, given that you're dedicating a thread to each connection.
Good afternoon everyone reading this, I am new to programming with sockets, as well as new to asynchronous coding (I feel async may be part of the solution to my problem), so forgive me for any silly mistakes I make.
To start, I have a UDP Echo server that acts as a game server. Anytime it gets a ping sent to it, it adds the source ip and port to a list of "connected clients", and sends that exact ping out to everyone on the list, excluding the sender. This works fairly well, because it reacts upon receiving a message, so it can always just listen. The problem with the client however, is that I need to be constantly sending pings, while also listening.
This is currently what my client looks like:
import socket
from time import sleep
from contextlib import contextmanager
UDP_IP_ADDRESS = "127.0.0.1"
UDP_PORT_NO = 14004
Message = b"Hello World, From Client B"
#contextmanager
def socket_ctx():
""" Context Manager for the socket. Makes sure the socket will close regardless of why it exited."""
my_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
# Assign IP address and a RANDOM available port number to socket
my_socket.bind(('127.0.0.1', 0))
try:
# Let the rest of the app use the socket and wait for it to finish
yield my_socket
finally:
my_socket.close()
def send_data(client_sock):
client_sock.sendto(Message, (UDP_IP_ADDRESS, UDP_PORT_NO))
def listen(client_sock):
print(client_sock.recvfrom(100))
with socket_ctx() as sock:
while True:
send_data(sock)
listen(sock)
sleep(2)
Currently, it sends a ping once, then just idles as it presumably is listening. If it does happen to get a ping back, say, another client send a ping to the server, and the server sent the ping to this client, it hears it, prints it, and starts the loop again. The issue is, without another client sending something to jolt this one out of the listen, it doesn't send it's pings.
I think async might be my solution, but I would have no clue how to go about that. Does anyone have a solution for this problem?
Here's how I would implement a server with "receive and handle incoming UDP sockets, plus do some packet-sending once per second" behavior. Note that this uses the select() function to multiplex the two tasks, rather than asynchronous I/O; hopefully that is okay.
import socket
import select
import time
UDP_IP_ADDRESS = "127.0.0.1"
UDP_PORT_NO = 14004
Message = b"Hello World, From Client B"
udp_socket = socket.socket(socket.AF_INET, socket.SOCK_DGRAM)
udp_socket.bind(('127.0.0.1', 0))
print "UDP socket is listening for incoming packets on port", udp_socket.getsockname()[1]
# When we want to send the next periodic-ping-message out
nextPingTime = time.time()
while True:
secondsUntilNextPing = nextPingTime - time.time();
if (secondsUntilNextPing < 0):
secondsUntilNextPing = 0
# select() won't return until udp_socket has some data
# ready-for-read, OR until secondsUntilNextPing seconds
# have passed, whichever comes first
inReady, outReady, exReady = select.select([udp_socket], [], [], secondsUntilNextPing)
if (udp_socket in inReady):
# There's an incoming UDP packet ready to receive!
print(udp_socket.recvfrom(100))
now = time.time()
if (now >= nextPingTime):
# Time to send out the next ping!
print "Sending out scheduled ping at time ", now
udp_socket.sendto(Message, (UDP_IP_ADDRESS, UDP_PORT_NO))
nextPingTime = now + 1.0 # we'll do it again in another second
I am running a very simple python (3.x) client-server program (both locally on my PC) for a school project (not intended for the real world) which just sends messages back-and-forth (like view customers, add customer, delete customer, etc... real basic).
Sometimes the data can be multiple records which I had stored as namedTuples (just made sense) and then went down the path of using Pickle to transfer then.
So for example on the client I do something like this:
s.send(message.encode('utf-8'))
pickledResponse = s.recv(4096);
response = pickle.loads(pickledResponse)
Now ever so often I get the following error:
response = pickle.loads(pickledResponse)
EOFError: Ran out of input
My fear is that this has something to do with my socket (TCP) transfer and maybe somehow I am not getting all the data in time for my pickle.loads - make sense? If not I am really lost as to why this would be happening so inconsistently.
However, even if I am right I am not sure how to fix it (quickly), I was considering dropping pickle and just using strings (but couldn't this suffer from the same fate)? Does anyone have any suggestions?
Really my message are pretty basic - usually just a command and some small data like "1=John" which means command (1) which is FIND command and then "John" and it returns the record (name, age, etc...) of John (as a namedTuple - but honestly this isn't mandatory).
Any suggestions or help would be much appreciated, looking for a quick fix...
The problem with your code is that recv(4096), when used on a TCP socket, might return different amount of data from what you might have expected, as they are sliced at packet boundaries.
The easy solution is to prefix each message with length; for sending like
import struct
packet = pickle.dumps(foo)
length = struct.pack('!I', len(packet)
packet = length + packet
then for receiving
import struct
buf = b''
while len(buf) < 4:
buf += socket.recv(4 - len(buf))
length = struct.unpack('!I', buf)[0]
# now recv until at least length bytes are received,
# then slice length first bytes and decode.
However, Python standard library already has a support for message oriented pickling socket, namely multiprocessing.Connection, that supports sending and receiving pickles with ease using the Connection.send and Connection.recv respectively.
Thus you can code your server as
from multiprocessing.connection import Listener
PORT = 1234
server_sock = Listener(('localhost', PORT))
conn = server_sock.accept()
unpickled_data = conn.recv()
and client as
from multiprocessing.connection import Client
client = Client(('localhost', 1234))
client.send(['hello', 'world'])
For receiving everything the server sends until it closes its side of the connection try this:
import json
import socket
from functools import partial
def main():
message = 'Test'
with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as sock:
sock.connect(('127.0.0.1', 9999))
sock.sendall(message.encode('utf-8'))
sock.shutdown(socket.SHUT_WR)
json_response = b''.join(iter(partial(sock.recv, 4096), b''))
response = json.loads(json_response.decode('utf-8'))
print(response)
if __name__ == '__main__':
main()
I've used sendall() because send() has the same ”problem” as recv(): It's not guaranteed everything is sent. send() returns the number of bytes actually sent, and the programmer has to make sure that matches the length of the argument and if not to send the rest until everything is out. After sending the writing side of the connection is closed (shutdown()) so the server knows there is no more data coming from the client. After that, all data from the server is received until the server closes its side of the connection, resulting in the empty bytes object returned from the recv() call.
Here is a suitable socketserver.TCPServer for the client:
import json
from socketserver import StreamRequestHandler, TCPServer
class Handler(StreamRequestHandler):
def handle(self):
print('Handle request...')
message = self.rfile.read().decode('utf-8')
print('Received message:', message)
self.wfile.write(
json.dumps(
{'name': 'John', 'age': 42, 'message': message}
).encode('utf-8')
)
print('Finished request.')
def main():
address = ('127.0.0.1', 9999)
try:
print('Start server at', address, '...')
server = TCPServer(address, Handler)
server.serve_forever()
except KeyboardInterrupt:
print('Stopping server...')
if __name__ == '__main__':
main()
It reads the complete data from the client and puts it into a JSON encoded response with some other, fixed items. Instead of the low level socket operations it makes use of the more convenient file like objects the TCPServer offers for reading and writing from/to the connection. The connection is closed by the TCPServer after the handle() method finished.
I am totally new to socket programming.
I have a product and trying to connect.
I can send the data and see its result on product, but I cannot receive anything.
this is my script:
import socket
def ScktConn():
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(('127.0.0.1', 5006))
# our local IP is 192.168.2.1, but it works even with 127.0.0.1, I don't know from where #it is coming
Freq=raw_input('Frequency(450-2500): ')
CmdF='0 ace_set_frequency C1 '+str(Freq)+' \r\n'
s.send(CmdF)
# so far I sent a tcl command to product to set the frequency and it works
s.send('0 ace_azplayer_remove_player XXX \r\n')
# sending another tcl command and works
s.send('0 ace_azplayer_add_player \r\n')
# here it is working too
s.send('0 ace_azplayer_add_ace XXX C1\r\n')
Path='C:/Users/AM_RJ/Desktop/gridview_script/PBF/4x4U_wocorr_SNR.csv'
s.send('0 ace_azplayer_load_csvfile AzPlayer1 '+Path+' \r\n')
# here I should receive some numbers, but always returning me 0!
#even if I send ('hello!') and use recv(1024), it returns 0!
csvid=s.recv(4096)
print csvid
Path2='0 ace_azplayer_edit_playback_file AzPlayer1 '+str(csvid)+' -linkConfiguration "4x4" \r\n'
print Path2
s.send(Path2)
After using recv(4096), I should receive some numbers, but it always returning me 0!
even if I send ('hello!') and use recv(1024), it returns 0!
I'm using python 2.7.
I am not even sure whether or not the server and client sides are correct in my script!
Please help me out about it.
You need more than one socket, here is a minimal example (which would need more work to be made robust). ScktConn spawns a new thread which creates a server socket that listens for the connection from s.
import socket
import threading
import time
address = ('127.0.0.1', 5007)
def ScktRecv():
r = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
r.bind(address)
r.listen(1)
conn, _ = r.accept()
csvid = conn.recv(4096)
print "recv: %s" % csvid
def ScktConn():
recv_thread = threading.Thread(target=ScktRecv)
recv_thread.start()
time.sleep(1)
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(address)
# our local IP is 192.168.2.1, but it works even with 127.0.0.1, I don't know from where #it is coming
Freq=raw_input('Frequency(450-2500): ')
CmdF='0 ace_set_frequency C1 '+str(Freq)+' \r\n'
s.send(CmdF)