I have a gRPC project that works correctly running server and client on the same machine, but when I try it using different machines in a same wifi network using IPV4, it occurs the following error on client:
Traceback (most recent call last):
File "client.py", line 29, in <module>
run()
File "client.py", line 25, in run
add_reply = stub.Add(add_request)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown"
debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown {grpc_status:14, created_time:"2022-12-10T15:21:32.619696358-03:00"}"
>
I've seem people reporting that it worked for them, but it's seemed not be much different, so I have no idea what's missing.
server code:
from os import system
system('clear')
from concurrent import futures
import time
import grpc
from grpc import _server
import sum_pb2
import sum_pb2_grpc
PORT = 50051
HOST = "localhost"
class Servicer(sum_pb2_grpc.SumNumbersServicer):
def Add(self, request, context):
print(request)
add_reply = sum_pb2.AddReply()
add_reply.r = request.a + request.b
# return super().Add(request, context)
return add_reply
def serve():
server:_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
sum_pb2_grpc.add_SumNumbersServicer_to_server(Servicer(), server)
# serve._add_insecure_port(f"localhost:{PORT}")
server.add_insecure_port(f"{HOST}:{PORT}")
server.start()
print(f"Listenning on port {PORT}")
try:
server.wait_for_termination()
except KeyboardInterrupt:
server.stop(0)
if __name__ == "__main__":
serve()
client code:
from os import system
system('clear')
from concurrent import futures
import time
from random import randint
import grpc
from grpc import _server
import sum_pb2
import sum_pb2_grpc
PORT = 50051
HOST = "localhost"
HOST = "192.168.100.9"
min = 0
max = 100
def run():
with grpc.insecure_channel(f"{HOST}:{PORT}") as channel:
stub = sum_pb2_grpc.SumNumbersStub(channel)
a = randint(min, max)
b = randint(min, max)
add_request = sum_pb2.AddRequest(a= a, b= b)
add_reply = stub.Add(add_request)
print(f"{a} + {b} = {add_reply}")
if __name__ == "__main__":
run()
EDIT:
When using "0.0.0.0" as host in server.py it prints the following error (differently from last time, this time I'm using ethernet cable on both machines, idk if it makes difference ):
Traceback (most recent call last):
File "client.py", line 29, in <module>
run()
File "client.py", line 25, in run
add_reply = stub.Add(add_request)
File "/home/estagiarioti/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/estagiarioti/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.10.6:50051: Failed to connect to remote host: No route to host"
debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.10.6:50051: Failed to connect to remote host: No route to host {grpc_status:14, created_time:"2022-12-12T13:58:15.049856347-03:00"}"
>
Error message when running both machines in an wifi network and "0.0.0.0" ip in host:
Traceback (most recent call last):
File "client.py", line 29, in <module>
run()
File "client.py", line 25, in run
add_reply = stub.Add(add_request)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown"
debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown {created_time:"2022-12-12T19:12:13.762636131-03:00", grpc_status:14}"
>
ps: Sometimes i use WSL, sometimes i don't, maybe this is important to know.
localhost usually resolves to 127.0.0.1 which is a special so-called loopback network address. It is used when you know that you don't need external (to the host) connectivity because it generally doesn't use the host's network stack.
You will generally want to bind the server to 0.0.0.0 instead. This binds the server to all the host's network interfaces and is a simple way to ensure that your service is available to external hosts.
Also, you will need to be confident that the server's host's network address is 192.168.100.9 to ensure that the client tries to communicate with the correct host.
I don't know what I was doing wrong in WSL, but it worked when I used pure Windows and in the Firewall settings I allowed all the pythons in the list to have access in public and private networks. This with both machines on the same Wifi network
Related
I am trying to connect and upload / download a text file via Python, but I am getting this error:
Traceback (most recent call last):
File "C:\Users\Abdul\OneDrive\Desktop\SFTP neu\main3.py", line 8, in <module>
c.connect(hostname = "0.0.0.0",port = 22, username = "tester", pkey = k)
File "D:\Python\lib\site-packages\paramiko\client.py", line 349, in connect
retry_on_signal(lambda: sock.connect(addr))
File "D:\Python\lib\site-packages\paramiko\util.py", line 283, in retry_on_signal
return function()
File "D:\Python\lib\site-packages\paramiko\client.py", line 349, in <lambda>
retry_on_signal(lambda: sock.connect(addr))
OSError: [WinError 10049] Die angeforderte Adresse ist in diesem Kontext ungültig
I'm using Rebex TinySFTP Server.
At first I thought my host was wrong, but that wasn't it.
In this example I used 0.0.0.0.
Here is my code:
#!/usr/bin/python3
import paramiko
k = paramiko.RSAKey.from_private_key_file("C:\\Users\\Abdul\\OneDrive\\Desktop\\RebexServer\\private-rsa-key.pem")
c = paramiko.SSHClient()
c.set_missing_host_key_policy(paramiko.AutoAddPolicy())
print("connecting")
c.connect(hostname = "0.0.0.0",port = 22, username = "tester", pkey = k)
print("connected")
commands = ["C:\\Users\\Abdul\\OneDrive\\Desktop\\RebexServer\\data\\testfile.txt", "C:\\Users\\Abdul\\OneDrive\\Desktop\\SFTP neu\\data\\testfile1.txt"]
for command in commands:
print("Executing {0}".format( command ))
stdin , stdout, stderr = c.exec_command(command)
print(stdout.read())
print("Errors")
print(stderr.read())
c.close()
At first I thought my host was wrong, but that wasn't it. In this example I used 0.0.0.0
0.0.0.0 is often used when starting a server to indicate that the server should bind to all available IP addresses. But it's not what you should use in your client.
Use one of the actual IP addresses your server is binding to. If both are on the same machine, try 127.0.0.1 or whatever IP address you are using locally. Addresses starting with 192.168. are common in home and small office networks.
I'm testing basic code, and it is working, as it should
import websocket
ws = websocket.WebSocket()
ws.connect("wss://echo.websocket.org/")
send_string = "Testing WebSocket"
print "Sending: ",send_string
ws.send(send_string)
result = ws.recv()
print "Received: ",result
ws.close()
But after changing websocket address to my desire "wss://10.240.118.148:51443/ws/" I have:
Traceback (most recent call last):
File "testws.py", line 5, in <module>
ws.connect("wss://10.240.118.148:51443/ws/")
File "/usr/lib/python2.7/site-packages/websocket/_core.py", line 213, in connect
options.pop('socket', None))
File "/usr/lib/python2.7/site-packages/websocket/_http.py", line 72, in connect
sock = _tunnel(sock, hostname, port, auth)
File "/usr/lib/python2.7/site-packages/websocket/_http.py", line 224, in _tunnel
"failed CONNECT via proxy status: %r" % status)
websocket._exceptions.WebSocketProxyException: failed CONNECT via proxy status: 503
Of course connection to websocket is working:
wscat -n -c wss://10.240.118.148:51443/ws/
connected (press CTRL+C to quit)
> test
< {"error":{"code":-32700,"data":"parse error - unexpected 't'","message":"Parse error"},"id":null,"jsonrpc":"2.0"}
Do you have any idea what can be wrong ?
"failed CONNECT via proxy status: 503 is" really is a server error and not a Proxy error, you would have to consult your server's error log files to find out the reason for the problem.
So I have a python script that needs to send a packet to my server 'x.x.x.x'. I've been able to successfully initialise Tor through Python by setting up the SOCKS5 proxy, but upon trying to send a packet to my server I get the error:
Traceback (most recent call last):
File "test.py", line 18, in <module>
sock.sendto(bytes, ("x.x.x.x", 6000))
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 338, in sendto
self.bind(("", 0))
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 325, in bind
_, relay = self._SOCKS5_request(self._proxyconn, UDP_ASSOCIATE, dst)
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 494, in _SOCKS5_request
raise SOCKS5Error("{0:#04x}: {1}".format(status, error))
socks.SOCKS5Error: 0x07: Command not supported, or protocol error
From what I've seen, socket/SOCKS5 doesn't support connectionless UDP, so I attempted connecting to the port and then sending the packet once connected. I still get the same error as seen above, output can be seen below.
Traceback (most recent call last):
File "test.py", line 18, in <module>
sock.connect(("x.x.x.x", 6000))
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 698, in connect
self.bind(("", 0))
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 325, in bind
_, relay = self._SOCKS5_request(self._proxyconn, UDP_ASSOCIATE, dst)
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 494, in _SOCKS5_request
raise SOCKS5Error("{0:#04x}: {1}".format(status, error))
socks.SOCKS5Error: 0x07: Command not supported, or protocol error
Seeing as a UDP connection does not work either, I would prefer to stay connectionless as this makes my intended use simpler as the port is not necessarily active/open at any given time. The script I'm using to attempt to send the packet can be seen below. I've added but commented out the connectionless and connection methods I was using. Ignore all the extra imports at the top, these are for use later in the script development.
import socks
import socket
import requests
from TorCtl import TorCtl
import urllib2
import random
import math
import time
socks.setdefaultproxy(proxy_type=socks.PROXY_TYPE_SOCKS5, addr="127.0.0.1", port=9050)
socket.socket = socks.socksocket
print requests.get("http://icanhazip.com").text
sock = socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
bytes=random._urandom(1024)
# UDP CONNECTION METHOD
#sock.connect(("x.x.x.x", 6000))
#sock.send(bytes)
# UDP CONNECTIONLESS METHOD
#sock.sendto(bytes, ("x.x.x.x", 6000))
Which brings me to my question - is there any way to send UDP packets via a connectionless method through a SOCKS5 proxy in python?
UPDATE
I originally had SocksiPy installed instead of PySocks, so I've replaced the modules and removed the monkeypatch in the original script. But now, I'm instead getting 'Broken Pipe' errors, as seen below.
Traceback (most recent call last):
File "test.py", line 19, in <module>
s.sendto(bytes, ("x.x.x.x", 6000))
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 336, in sendto
return _BaseSocket.sendto(self, bytes, *args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/socks.py", line 223, in <lambda>
return lambda self, *pos, **kw: self._savedmethods[name](*pos, **kw)
socket.error: [Errno 32] Broken pipe
I've double checked to make sure Tor is actually working and the proxy is up, which it is as I can receive HTTP requests and responses through the Tor proxy - the responses to sites like http://icanhazip.com/ return a different IP that my actual IP, which suggests that Tor is indeed functioning. I also figured that removing the monkeypatch might make it work with better compatibility, seeing as with the new PySocks module on the OLD script it still fails with the original errors seen above.
The new script:
import socks
import socket
import random
import math
s = socks.socksocket()
s.set_proxy(socks.SOCKS5, "localhost", 9050)
bytes=random._urandom(1024)
# UDP CONNECTION METHOD
#sock.connect(("x.x.x.x", 6000))
#sock.send(bytes)
# UDP CONNECTIONLESS METHOD
#s.sendto(bytes, ("x.x.x.x", 6000))
The errors I'm describing above are with the connectionless method - using the connection method seems to possibly work, however it hangs when connecting to the port (which is the be expected, as the port isn't open).
As #gwyn pointed out, Tor only supports TCP streams, as specified on their website. Using UDP over Tor will not work, only TCP connections.
The following code runs a socket server on a thread. The client socket sends 'client: hello' to the server, and the server socket receives and replies 'server: world'.
import socket
import threading
def server():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind(('127.0.0.1', 12345))
sock.listen(1)
req, addr = sock.accept()
print req.recv(1024)
req.sendall('server: world')
def client():
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect(('127.0.0.1', 12345))
sock.sendall('client: hello')
print sock.recv(1024)
def main():
t = threading.Thread(target=server)
t.start()
client()
if __name__ == '__main__':
main()
It runs ok as expected the first time, but from the second time, if you do not wait for a good few seconds for the server to release the socket, and if you try this on a Linux machine or Mac (Windows do not get it somehow) you will run into this error:
Traceback (most recent call last):
File "socket_send_receive.py", line 24, in <module>
main()
File "socket_send_receive.py", line 21, in main
client()
File "socket_send_receive.py", line 14, in client
sock.connect(('127.0.0.1', 12345))
File "<string>", line 1, in connect
socket.error: [Errno 111] Connection refused
Exception in thread Thread-1:
Traceback (most recent call last):
File "/home/cxuan/python/2.6/lib/python2.6/threading.py", line 532, in __bootstrap_inner
self.run()
File "/home/cxuan/python/2.6/lib/python2.6/threading.py", line 484, in run
self.__target(*self.__args, **self.__kwargs)
File "socket_send_receive.py", line 6, in server
sock.bind(('127.0.0.1', 12345))
File "<string>", line 1, in bind
error: [Errno 98] Address already in use
I am looking for some insight into why this is happening and if it is possible to be genuinely resolved or what best practice should be adopted.
I know already using this option can be a workaround thanks to the other posts here on stackoverflow.
socket.SO_REUSEADDR
When a socket is closed, it ends up in a state called STATE_WAIT (see this diagram). While the socket is in this state, no one else can use the same address (ip-number/port pair) unless the SO_REUSEADDR option is set on the socket.
See e.g. the Wikipedia article on TCP for more information about how TCP works and the different states.
I'm having some difficulty attempting to create an SSL socket in Python to use a proxy that requires authentication. I am very sorry for the length, but I felt it was best to include as much detail as possible.
First, the server code looks like this:
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
def __init__(self, server_address, RequestHandlerClass, client_manager, recv_queue):
SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass, bind_and_activate=True)
<snipped out extra code>
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def setup(self):
while True:
try:
print 'trying to wrap in ssl'
self.request = ssl.wrap_socket(self.request,
certfile=(os.getcwd() + '/ssl_certs/newcert.pem'),
keyfile=(os.getcwd() + '/ssl_certs/webserver.nopass.key'),
server_side=True,
cert_reqs=ssl.CERT_NONE,
ssl_version=ssl.PROTOCOL_TLSv1,
do_handshake_on_connect=False,
suppress_ragged_eofs=True)
break
except Exception, ex:
print 'error trying to wrap in ssl %s' % ex
def handle(self):
# Display message that client has connected
print '\r[*] Received connection from %s:%s\r' % (self.client_address[0], self.client_address[1])
while self.stopped() == False:
recv_msg = self.request.read(1024)
if recv_msg == '':
self.stop.set()
server.recv_queue.put(recv_msg)
break
else:
server.recv_queue.put(recv_msg)
if self.stopped():
print '[!] Received STOP signal from %s:%s; Exiting!' % (self.client_address[0], self.client_address[1])
Second, this is the client code where I set up the information needed to connect via the proxy that requires authentication:
class proxyCommsHandler():
def __init__(self, user, password, remote_host, remote_port, list_of_proxies):
# information needed to connect
self.user = 'username'
self.passwd = 'password'
self.remote_host = 'remote_host_ip'
self.remote_port = 8008
self.list_of_proxies = [['proxyserver.hostname.com', 8080]]
# setup basic authentication to send to the proxy when we try to connect
self.user_pass = base64.encodestring(self.user + ':' + self.passwd)
self.proxy_authorization = 'Proxy-authorization: Basic ' + self.user_pass + '\r\n'
self.proxy_connect = 'CONNECT %s:%s HTTP/1.1\r\n' % (self.remote_host, self.remote_port)
self.user_agent = "User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1\r\n"
self.proxy_pieces = self.proxy_connect + self.proxy_authorization + self.user_agent + '\r\n'
Now, here's where I initially connect to the proxy, where I get no errors (I get a '200' status code):
self.proxy = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.proxy.connect( (proxy_host, proxy_port) )
self.proxy.sendall(self.proxy_pieces)
self.response = proxy.recv(1024)
Here's where the client fails (I think). I try to take self.proxy and wrap it in SSL, like this:
sslsock = ssl.wrap_socket(self.proxy, server_side=False, do_handshake_on_connect=True,
ssl_version=ssl.PROTOCOL_TLSv1)
This is the error that I see on the client:
Traceback (most recent call last):
File "C:\Python27\pyrevshell.py", line 467, in <module>
proxyCommsHandler(None, None, None, None, list_of_proxies).run()
File "C:\Python27\pyrevshell.py", line 300, in run
ssl_version=ssl.PROTOCOL_TLSv1)
File "C:\Python27\lib\ssl.py", line 372, in wrap_socket
ciphers=ciphers)
File "C:\Python27\lib\ssl.py", line 134, in __init__
self.do_handshake()
File "C:\Python27\lib\ssl.py", line 296, in do_handshake
self._sslobj.do_handshake()
SSLError: [Errno 8] _ssl.c:503: EOF occurred in violation of protocol
The client does connect, like shown from the output here:
trying to wrap in ssl
[*] Received connection from x.x.x.x:47144
[*] x.x.x.x:47144 added to the client list
But then it's immediately followed by an exception:
----------------------------------------
Exception happened during processing of request from ('x.x.x.x', 47144)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 582, in process_request_thread
self.finish_request(request, client_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 639, in __init__
self.handle()
File "shell_server.py", line 324, in handle
recv_msg = self.request.read(1024)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 138, in read
return self._sslobj.read(len)
SSLError: [Errno 1] _ssl.c:1348: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
----------------------------------------
While I realize that this sounds like an obvious problem based on the Exceptions that were thrown, the interesting parts are that:
I successfully connect initially via the proxy as shown above
I can successfully connect with a web browser behind behind the same proxy, and no exceptions are thrown; I can pass data to the browser
I've tried different SSL Protocol versions on both the server and client side, as shown in the table here in the Python documentation; it errors on the client-side each time
I've used Wireshark on both ends of the connection. While using a normal browser and connecting to the server, I see the entire SSL handshake and negotiation process, and everything runs smoothly.
However, when I use the client shown above, as soon as I connect with it, I see the client send a Client Hello message, but then my server sends a RST packet to kill the connection (I haven't determined if this is before or after the Exception is thrown).
Again, I apologize for the length, but I am in dire need of expert advice.
I've figured out the issue to my problem. I am sending the self.user_agent to the remote host when I connect via the proxy for the first time, which interferes with the SSL Handshake.
To solve this, I put an initial self.request.recv() in the def setup(self) function before I call ssl.wrap_socket on the socket.