Develop Reference

Connect two machines using gRPC

I have a gRPC project that works correctly running server and client on the same machine, but when I try it using different machines in a same wifi network using IPV4, it occurs the following error on client:
Traceback (most recent call last):
File "client.py", line 29, in <module>
run()
File "client.py", line 25, in run
add_reply = stub.Add(add_request)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown"
debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown {grpc_status:14, created_time:"2022-12-10T15:21:32.619696358-03:00"}"
>
I've seem people reporting that it worked for them, but it's seemed not be much different, so I have no idea what's missing.
server code:
from os import system
system('clear')
from concurrent import futures
import time
import grpc
from grpc import _server
import sum_pb2
import sum_pb2_grpc
PORT = 50051
HOST = "localhost"
class Servicer(sum_pb2_grpc.SumNumbersServicer):
def Add(self, request, context):
print(request)
add_reply = sum_pb2.AddReply()
add_reply.r = request.a + request.b
# return super().Add(request, context)
return add_reply
def serve():
server:_server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
sum_pb2_grpc.add_SumNumbersServicer_to_server(Servicer(), server)
# serve._add_insecure_port(f"localhost:{PORT}")
server.add_insecure_port(f"{HOST}:{PORT}")
server.start()
print(f"Listenning on port {PORT}")
try:
server.wait_for_termination()
except KeyboardInterrupt:
server.stop(0)
if __name__ == "__main__":
serve()
client code:
from os import system
system('clear')
from concurrent import futures
import time
from random import randint
import grpc
from grpc import _server
import sum_pb2
import sum_pb2_grpc
PORT = 50051
HOST = "localhost"
HOST = "192.168.100.9"
min = 0
max = 100
def run():
with grpc.insecure_channel(f"{HOST}:{PORT}") as channel:
stub = sum_pb2_grpc.SumNumbersStub(channel)
a = randint(min, max)
b = randint(min, max)
add_request = sum_pb2.AddRequest(a= a, b= b)
add_reply = stub.Add(add_request)
print(f"{a} + {b} = {add_reply}")
if __name__ == "__main__":
run()
EDIT:
When using "0.0.0.0" as host in server.py it prints the following error (differently from last time, this time I'm using ethernet cable on both machines, idk if it makes difference ):
Traceback (most recent call last):
File "client.py", line 29, in <module>
run()
File "client.py", line 25, in run
add_reply = stub.Add(add_request)
File "/home/estagiarioti/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/estagiarioti/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.10.6:50051: Failed to connect to remote host: No route to host"
debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.10.6:50051: Failed to connect to remote host: No route to host {grpc_status:14, created_time:"2022-12-12T13:58:15.049856347-03:00"}"
>
Error message when running both machines in an wifi network and "0.0.0.0" ip in host:
Traceback (most recent call last):
File "client.py", line 29, in <module>
run()
File "client.py", line 25, in run
add_reply = stub.Add(add_request)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
return _end_unary_response_blocking(state, call, False, None)
File "/home/caio/dev/laboratorio/mygrpc/.env/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
status = StatusCode.UNAVAILABLE
details = "failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown"
debug_error_string = "UNKNOWN:failed to connect to all addresses; last error: UNKNOWN: ipv4:192.168.100.9:50051: tcp handshaker shutdown {created_time:"2022-12-12T19:12:13.762636131-03:00", grpc_status:14}"
>
ps: Sometimes i use WSL, sometimes i don't, maybe this is important to know.

localhost usually resolves to 127.0.0.1 which is a special so-called loopback network address. It is used when you know that you don't need external (to the host) connectivity because it generally doesn't use the host's network stack.
You will generally want to bind the server to 0.0.0.0 instead. This binds the server to all the host's network interfaces and is a simple way to ensure that your service is available to external hosts.
Also, you will need to be confident that the server's host's network address is 192.168.100.9 to ensure that the client tries to communicate with the correct host.

I don't know what I was doing wrong in WSL, but it worked when I used pure Windows and in the Firewall settings I allowed all the pythons in the list to have access in public and private networks. This with both machines on the same Wifi network

Trouble connecting to Cloud SQL in python

I am trying to connect to a database on Cloud SQL, but I keep getting the same error. Not sure what it is, and tried several approaches.
Input:
import pymysql
connection = pymysql.connect(host='127.0.0.1',
user='',
password='XXXX',
db='cmcsql')
output:
C:\Users\Ejer\anaconda3\envs\pythonProject\python.exe C:/Users/Ejer/PycharmProjects/pythonProject/CloudSQL_test.py
Traceback (most recent call last):
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 569, in connect
sock = socket.create_connection(
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\socket.py", line 808, in create_connection
raise err
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\socket.py", line 796, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] Der kunne ikke oprettes forbindelse, fordi destinationscomputeren aktivt nægtede det
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/Ejer/PycharmProjects/pythonProject/CloudSQL_test.py", line 2, in <module>
connection = pymysql.connect(host='127.0.0.1',
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\__init__.py", line 94, in Connect
return Connection(*args, **kwargs)
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 327, in __init__
self.connect()
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 619, in connect
raise exc
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' ([WinError 10061] Der kunne ikke oprettes forbindelse, fordi destinationscomputeren aktivt nægtede det)")
Process finished with exit code 1

You should not be using 127.0.0.1 to connect to cloud sql instance.
127.0.0.1 is (in most cases) for localhost connection (when you run the db server locally on the same machine as where the client is). Instead you should be using the ip address given to your cloud sql instance. You can find it on the overview tab of the sql instance under Connect to this instance.
You should also create your user on the users tab and then use it in your code.
Don't forget about authentication, check out connections tab and read more about it here
Making sure that you have proper ip, user, existing db and connecting from authorized network should do the trick.

PyMysql connects to port 3306 by default, if your process is running on a different port it won't be able to connect.
Apart from the host, user, password parameters you also have to provide the port on which the sql process is running.

Using a Python script to establish FTP connection behind FTP proxy

Im trying to connect to an ftp server that is behind ftp proxy.
The proxy server does not require username and password, the ftp server does.
I have searched through several posts:
Proxies in python
How to use urllib2 to access ftp/http server using proxy with authentification
How to connect to ftp server via proxy using ftplib
How to specify an authenticated proxy for a python http connection?
Here is the last version of my code. So far I've figured out that to open the ftp server behind an ftp proxy, the notation ftp://username:password#server can be used. I'm using the urllib2 library to define the proxy server.
import urllib2
proxy_host = '101.11.44.84:8021' # only host name, no scheme (http/ftp)
proxy_handler = urllib2.ProxyHandler({'ftp': proxy_host})
auth = urllib2.FTPHandler()
try:
opener_thru_proxy = urllib2.build_opener(proxy_handler, auth)
except:
logger.exception('build_opener error')
raise
print opener_thru_proxy
try:
conn = opener_thru_proxy.open('ftp://user:password#100.159.66.113')
except:
logger.exception('opener thru proxy error')
raise
print conn.read()
conn.close()
The output of this code yields:
2016-05-23 16:15:28,286 - root - ERROR - opener thru proxy error
Traceback (most recent call last):
File "<string>", line 293, in <module>
File "C:\_Studna\Python\python-2.7.10.amd64\lib\urllib2.py", line 431, in open
File "C:\_Studna\Python\python-2.7.10.amd64\lib\urllib2.py", line 449, in _open
File "C:\_Studna\Python\python-2.7.10.amd64\lib\urllib2.py", line 409, in _call_chain
File "C:\_Studna\Python\python-2.7.10.amd64\lib\urllib2.py", line 1412, in ftp_open
File "C:\_Studna\Python\python-2.7.10.amd64\lib\urllib2.py", line 1434, in connect_ftp
File "C:\_Studna\Python\python-2.7.10.amd64\lib\urllib.py", line 875, in __init__
File "C:\_Studna\Python\python-2.7.10.amd64\lib\urllib.py", line 884, in init
File "C:\_Studna\Python\python-2.7.10.amd64\lib\ftplib.py", line 135, in connect
File "C:\_Studna\Python\python-2.7.10.amd64\lib\socket.py", line 575, in create_connection
URLError: <urlopen error ftp error: [Errno 10060] A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond>
It seems that I'm unable to establish the ftp connection with the ftp server.
However, when I use the same set of connection data and use it in Total Commander, the connection works correctly. See the image enclosed Total Commander ftp connection window.

Can't send E-Mails with Python

I have this code as I am trying to send E-Mails using Python.
def Mail():
import smtplib
import textwrap
SERVER = "localhost"
FROM = "fromemail"
TO = ["toemail"]
SUBJECT =(input('What is the subject of your E-Mail'))
TEXT =(input('What do you want the E-Mail to say?'))
message = textwrap.dedent("""\
From: %s
To: %s
Subject: %s
%s
""" % (FROM, ", ".join(TO), SUBJECT, TEXT))
server = smtplib.SMTP(SERVER)
server.sendmail(FROM, TO, message)
server.quit()
This is the error message I get.
Traceback (most recent call last):
File "<pyshell#0>", line 1, in <module>
Mail()
File "E:\Coursework\EMail.py", line 26, in Mail
server = smtplib.SMTP(SERVER)
File "C:\Python33\python-3.3.5\lib\smtplib.py", line 241, in __init__
(code, msg) = self.connect(host, port)
File "C:\Python33\python-3.3.5\lib\smtplib.py", line 322, in connect
(code, msg) = self.getreply()
File "C:\Python33\python-3.3.5\lib\smtplib.py", line 375, in getreply
raise SMTPServerDisconnected("Connection unexpectedly closed")
smtplib.SMTPServerDisconnected: Connection unexpectedly closed

The 'Connection unexpectedly closed' shows that the server is probably ON but unable to create a connection to smtplib.
You could check the status of the port 25 smtp service to verify that,
# netstat -an | grep -i :25
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN
Then, check the status of the service occupying the port 25,
# netstat -plnt | grep -i :25
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 30860/master
You probably will have the same,
then just check the status of the master/postfix process,
# systemctl status postfix
If you get the error highlighted as,
fatal: cannot handle socket type AF_INET6 with "inet_protocols = ipv4"
then quickly go to the postfix configuration file at /etc/postfix/main.cf and check if you have this text and uncomment it,
inet_protocols = ipv4
Then restart postfix,
# systemctl restart postfix
then check status, hopefully you shouldn't get any errors.
Then check if python is able to connect to the SMTP server!
$ python -c 'import smtplib; smtplib.SMTP("localhost");'
You shouldn't get any output which shows that python is able to connect to the SMTP server.
you can run your email sending script now!

Python SSLError: Client-side-error(EOF occurred in violation of protocol), Server-side-error(SSL3_GET_RECORD:wrong version number)

I'm having some difficulty attempting to create an SSL socket in Python to use a proxy that requires authentication. I am very sorry for the length, but I felt it was best to include as much detail as possible.
First, the server code looks like this:
class ThreadedTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
def __init__(self, server_address, RequestHandlerClass, client_manager, recv_queue):
SocketServer.TCPServer.__init__(self, server_address, RequestHandlerClass, bind_and_activate=True)
<snipped out extra code>
class ThreadedTCPRequestHandler(SocketServer.BaseRequestHandler):
def setup(self):
while True:
try:
print 'trying to wrap in ssl'
self.request = ssl.wrap_socket(self.request,
certfile=(os.getcwd() + '/ssl_certs/newcert.pem'),
keyfile=(os.getcwd() + '/ssl_certs/webserver.nopass.key'),
server_side=True,
cert_reqs=ssl.CERT_NONE,
ssl_version=ssl.PROTOCOL_TLSv1,
do_handshake_on_connect=False,
suppress_ragged_eofs=True)
break
except Exception, ex:
print 'error trying to wrap in ssl %s' % ex
def handle(self):
# Display message that client has connected
print '\r[*] Received connection from %s:%s\r' % (self.client_address[0], self.client_address[1])
while self.stopped() == False:
recv_msg = self.request.read(1024)
if recv_msg == '':
self.stop.set()
server.recv_queue.put(recv_msg)
break
else:
server.recv_queue.put(recv_msg)
if self.stopped():
print '[!] Received STOP signal from %s:%s; Exiting!' % (self.client_address[0], self.client_address[1])
Second, this is the client code where I set up the information needed to connect via the proxy that requires authentication:
class proxyCommsHandler():
def __init__(self, user, password, remote_host, remote_port, list_of_proxies):
# information needed to connect
self.user = 'username'
self.passwd = 'password'
self.remote_host = 'remote_host_ip'
self.remote_port = 8008
self.list_of_proxies = [['proxyserver.hostname.com', 8080]]
# setup basic authentication to send to the proxy when we try to connect
self.user_pass = base64.encodestring(self.user + ':' + self.passwd)
self.proxy_authorization = 'Proxy-authorization: Basic ' + self.user_pass + '\r\n'
self.proxy_connect = 'CONNECT %s:%s HTTP/1.1\r\n' % (self.remote_host, self.remote_port)
self.user_agent = "User-Agent: Mozilla/5.0 (Windows NT 6.1; rv:7.0.1) Gecko/20100101 Firefox/7.0.1\r\n"
self.proxy_pieces = self.proxy_connect + self.proxy_authorization + self.user_agent + '\r\n'
Now, here's where I initially connect to the proxy, where I get no errors (I get a '200' status code):
self.proxy = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.proxy.connect( (proxy_host, proxy_port) )
self.proxy.sendall(self.proxy_pieces)
self.response = proxy.recv(1024)
Here's where the client fails (I think). I try to take self.proxy and wrap it in SSL, like this:
sslsock = ssl.wrap_socket(self.proxy, server_side=False, do_handshake_on_connect=True,
ssl_version=ssl.PROTOCOL_TLSv1)
This is the error that I see on the client:
Traceback (most recent call last):
File "C:\Python27\pyrevshell.py", line 467, in <module>
proxyCommsHandler(None, None, None, None, list_of_proxies).run()
File "C:\Python27\pyrevshell.py", line 300, in run
ssl_version=ssl.PROTOCOL_TLSv1)
File "C:\Python27\lib\ssl.py", line 372, in wrap_socket
ciphers=ciphers)
File "C:\Python27\lib\ssl.py", line 134, in __init__
self.do_handshake()
File "C:\Python27\lib\ssl.py", line 296, in do_handshake
self._sslobj.do_handshake()
SSLError: [Errno 8] _ssl.c:503: EOF occurred in violation of protocol
The client does connect, like shown from the output here:
trying to wrap in ssl
[*] Received connection from x.x.x.x:47144
[*] x.x.x.x:47144 added to the client list
But then it's immediately followed by an exception:
----------------------------------------
Exception happened during processing of request from ('x.x.x.x', 47144)
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 582, in process_request_thread
self.finish_request(request, client_address)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 323, in finish_request
self.RequestHandlerClass(request, client_address, self)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/SocketServer.py", line 639, in __init__
self.handle()
File "shell_server.py", line 324, in handle
recv_msg = self.request.read(1024)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ssl.py", line 138, in read
return self._sslobj.read(len)
SSLError: [Errno 1] _ssl.c:1348: error:1408F10B:SSL routines:SSL3_GET_RECORD:wrong version number
----------------------------------------
While I realize that this sounds like an obvious problem based on the Exceptions that were thrown, the interesting parts are that:
I successfully connect initially via the proxy as shown above
I can successfully connect with a web browser behind behind the same proxy, and no exceptions are thrown; I can pass data to the browser
I've tried different SSL Protocol versions on both the server and client side, as shown in the table here in the Python documentation; it errors on the client-side each time
I've used Wireshark on both ends of the connection. While using a normal browser and connecting to the server, I see the entire SSL handshake and negotiation process, and everything runs smoothly.
However, when I use the client shown above, as soon as I connect with it, I see the client send a Client Hello message, but then my server sends a RST packet to kill the connection (I haven't determined if this is before or after the Exception is thrown).
Again, I apologize for the length, but I am in dire need of expert advice.

I've figured out the issue to my problem. I am sending the self.user_agent to the remote host when I connect via the proxy for the first time, which interferes with the SSL Handshake.
To solve this, I put an initial self.request.recv() in the def setup(self) function before I call ssl.wrap_socket on the socket.