Develop Reference

Connection Timing Out When Accessing Gcloud MySQL from Python

I have a python application where I'm trying to access a MySQL database on Google's cloud service.
I've been following this set up guide for connecting via an external application (Python) and I am using the pymysql package. I'm attempting to connect via the proxy and have already authenticated my connection via gcloud auth log in from the console.
As of now, I CAN access the database via the console, but I need to be able to make queries from my python script to build it out. When I try running it as is, I get the following error:
OperationalError: (2003, "Can't connect to MySQL server on '34.86.47.192' (timed out)")
Here's the function I'm using, with security sensitive info starred out:
def uploadData():
# cd to the directory with the MySQL exe
os.chdir('C:\\Program Files\\MySQL\\MySQL Server 8.0\\bin')
# Invoke the proxy
subprocess.call('start cloud_sql_proxy_x64.exe -instances=trans-cosine-289719:us-east4:compuweather', shell=True)
# Create connection
# I have also tried host = '127.0.0.1' for localhost here
conn = pymysql.connect(host='34.86.47.192',
user='root',
password='*******',
db='gribdata')
try:
c = conn.cursor()
# Use the right databse
db_query = 'use gribdata'
c.execute(db_query)
query = 'SELECT * FROM clients'
c.execute(query)
result = c.fetchall()
print(result)
except Error as e:
print(e)
finally:
conn.close()

Yeah, this one's pretty limited in documentation, but what you want to do is run it from it's hosted IP and configure access to your external IP address on your server. So you want use that IP (34.xxx.xxx.xxx) rather than the loopback 127 local host IP.
To get it to work, you want to go to your connections tab and add a new connection within Gcloud. Make sure the public address box is checked, the IP is correct, and you save once done.
There's some excellent details here from some Gcloud engineers. Looks like some of the source documentation is outdated and this is the way to connect now.

First of all, confirm that the Cloud SQL proxy is indeed installed in the directory that you are expecting it to be. The Cloud SQL proxy is not part of MySQL Server, hence you should not find it in C:\\Program Files\\MySQL\\MySQL Server 8.0\\bin, at least by default. Instead, the Cloud SQL proxy is a tool provided by Google and is just an .exe file that can be stored in any directory you wish. For instructions on how to download the Proxy you can check the docs
The Cloud SQL proxy creates a secure link between the Cloud SQL instance and your machine. what it does is forward a local port in your machine to the Cloud SQL instance. Thus, the host IP that you should use if you are using the proxy is 127.0.0.1
conn = pymysql.connect(host='127.0.0.1',
user='root',
password='*******',
db='gribdata')
When starting the Cloud SQL Proxy with TCP socket, you should add the port to which you want to forward Cloud SQL's traffic at the end of the start command =tcp:3306
subprocess.call('start cloud_sql_proxy_x64.exe -instances=trans-cosine-289719:us-east4:compuweather=tcp:3306', shell=True)

Have you tried to connect CloudSQL from the console? Once you connected, you should get a message in the console displaying "Listening on 127.0.0.1:3306".Your connection command should be
"cloud_sql_proxy_x64.exe -instances=trans-cosine-289719:us-east4:compuweather=tcp:3306"
Try to connect cloud proxy from the console and try to create a connection with pymysql. Use "127.0.0.1".

Jupyter Notebook unable to connect to local MySQL database

I'm trying to connect my Jupyter Notebook, I use Google Colab, with my local MySQL database. When I run this script in PyCharm, my preferred IDE, it works no problem. Only when I run it in Google Colab or Project Jupyter, both get the same error, does this happen.
import pymysql
import pandas as pd
mySQLuser = 'username'
mySQLpasswd = 'password'
mySQLhost = '127.0.0.1'
mySQLport = '3306'
mySQLdatabase = 'databasename'
connection = pymysql.connect(user = mySQLuser, passwd = mySQLpasswd, host = mySQLhost, port = mySQLport, db = mySQLdatabase)
I've also tried the same with sqlalchemy(create_engine) and mysql.connector, but get the same error.
OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' ([Errno 111] Connection refused)")
I've also tried granting all privileges to the user and that didn't change anything. Another suggestion I've seen online is to null or change the bind-address in the configuration file (my.ini), but that didn't affect anything either.

Are you sure your colab/jupyter instances are run locally?
If that's not the case, you will not be able to access the SQL database on localhost (127.0.0.1) until you make your it accessible remotely (which implies several steps: making it accessible, changing the mySQLhost address in your code, authorizing connections from the host server in your SQL settings).

Got it to work. The service provider was rejecting the requests despite me configuring the network to allow it, but I sorted it out.

Connect to remote Postgres server via SQLAlchemy

I am trying to send some commands to a remote Postgres server using SQLAlchemy but each time I receive an error.
Please note that I can connect to the remote Postgres using SSH username and password to login to the server. For that I have used my local terminal, PuTTY and WinSCP so the problem appears to be in the Python code I have written
# create postgres engine to connect to the database
engine = create_engine('postgres://server_username:server_password#server_name:port/database')
with engine.connect() as conn:
ex = conn.execute("SELECT version();")
conn.close() # not needed but keep just in case
print(ex)
Running the code above yields the following error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL SYSCALL error: Connection reset by peer (0x00002746/10054)
expected authentication request from server, but received S
I have also tried adding the SSL verification parameter as follows
create_engine('postgres://server_username:server_password#server_name:port/database?sslmode=verify-full')
which returned the error
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) root certificate file "C:\Users\aris.pavlides\AppData\Roaming/postgresql/root.crt" does not exist
Either provide the file or change sslmode to disable server certificate verification.
at which point I had nothing to lose so I disabled certificate verification altogether
create_engine('postgres://server_username:server_password#server_name:port/database?sslmode=disable')
which returned the initial error message.
Do you have any ideas on how I can modify the code to make it work?

Sending Emails through Django - WinError 10060 A connection attempt failed and GetAddrInfo Error

I've been running an instance of Django on Windows R2 2012 for over a year and I've come to a road block. Yesterday something happened, I don't know what it could be. The same two errors keep popping up at different times though when trying to send an email:
[WinError 10060] A connection attempt failed because the connected
party did not properly respond after a period of time, or established
connection failed because connected host has failed to respond
and
socket.gaierror: [Errno 11001] getaddrinfo failed
Users are able to connect to the IP address of the server and the port Django is running on 192.168.1.5:8000, but they cannot send emails anymore. Though a percentage do go through as described here, but very few.
Things I've tried
1) This solution
import socket
socket.getaddrinfo('localhost', 8000)
Since I'm doing python manage.py runserver 192.168.1.5:8000, I added that IP and nothing.
2) I went into the Firewall settings and made sure that the ports were all good. The SMTP one that is declared in the setting.py file in my Django project and 25. All of them, inbound and out.
3) I tried sending things on my local machine and it does work. I used other programs that do not use Django to send emails and they do process on all other machines except the Server. So I know it's not my email server.
4) I changed the email config to use my Gmail account and it does process on all other machines except for the server. So it has to be the environment.
5) Editing http_proxy environment variables
The problem, in my case, was that some install at some point defined
an
environment variable http_proxy on my machine when I had no proxy.
Removing the http_proxy environment variable fixed the problem.
As described here
and in my Django project in the wsgi.y file:
os.environ['http_proxy'] = "http://192.168.1.5:8080"
os.environ['https_proxy'] = "http://192.168.1.5:8080"
6) Given this answer here (can someone please explain how I would do it to a django email function), I've also tried this method of wrapping it from solutions here
import smtplib
import socks
#socks.setdefaultproxy(TYPE, ADDR, PORT)
socks.setdefaultproxy(socks.SOCKS5, '192.168.1.5', 8080)
socks.wrapmodule(smtplib)
smtpserver = 'smtp.live.com'
AUTHREQUIRED = 1
smtpuser = 'example#hotmail.fr'
smtppass = 'mypassword'
RECIPIENTS = 'mailto#gmail.com'
SENDER = 'example#hotmail.fr'
mssg = "test message"
s = mssg
server = smtplib.SMTP(smtpserver,587)
server.ehlo()
server.starttls()
server.ehlo()
server.login(smtpuser,smtppass)
server.set_debuglevel(1)
server.sendmail(SENDER, [RECIPIENTS], s)
server.quit()
Though I wouldn't like to use such a method as I'd prefer using Django's built in email service.

Since you have not changed the code and errors you shared shows that it's a network related problem.
It's most probably a DNS issue. In your settings.py you have specified the EMAIL_HOST, which is i believe a hostname. You need to check you server's DNS server.
You are mentioning about checking your firewall settings but what you are doing wrong is not checking the actual connection.
To address the problem you can use couple of command line utilities like telnet or nslookup. You can check if you can resolve a hostname:
nslookup smptp.mail_host.com
This command will fail most probably.
I would like to point what you did wrong in your steps:
1) You have tried to get your services getaddrinfo in which you needed to put your smtp servers hostname, which would result with the same error. Sockets are the very primitive part of the connection on the application layer, you don't really need to dig in to that.
2) Checking firewall settings is OK.
3) This is a good step which shows that there is a problem with your servers network connection.
4) That is another evidence :)
5) You have got it wrong, if you have a proxy server on your network to connect external networks, than you use this settings. But you have configured it wrong. You should not set your projects url as proxy server.
6) This is another deep level coding. You should not use such low level script, which will cause you numerious problems, which would have been handled in high level modules.

I focused my answer on the strange fact that you can get around the problem using a SOCKS5 proxy. (I believe you. There was no time to ask you for details.) You verified that your example solution by SOCKS5 works for you. Django uses the same smtplib and you can easily wrap it the same way by this code added to wsgi.py.
import smtplib
import socks # it is the package SocksiPy or PySocks
socks.setdefaultproxy(socks.SOCKS5, '192.168.1.5', 8080)
socks.wrapmodule(smtplib)
Http(s) proxy (paragraph 5)) is not related because it does not affect SMTP and other protocols except http(s) because "SOCKS operates at a lower level than HTTP proxying".

Mysql.connector to access remote database in local network Python 3

I used mysql.connector python library to make changes to my local SQL server databases using:
from __future__ import print_function
import mysql.connector as kk
cnx = kk.connect(user='root', password='password123',
host='localhost',
database='db')
cursor = cnx.cursor(buffered=True)
sql = "DELETE FROM examples WHERE id = 4"
number_of_rows = cursor.execute(sql)
cnx.commit()
cnx.close()
This works fine, but when i try the same code with a change only to the 'host' parameter, with something like,
host='xxx.xxx.xxx.xxx'
(where the IP is that of a server connected to my local network.), it won't update that particular data base in that server.
The error thrown is something like:
mysql.connector.errors.DatabaseError: 2003 (HY000): Can't connect to MySQL server on 'xx.xxx.x.xx' (10060)
Why wouldn't this work?

First, you must check if your local IP can acces to your remote server (check if you are an IP restriction on your server), after check if your mysql database use the default port or not, If not you must precise the port in your code.

Check if the database user you are using to connect to the database on the remote host has the correct access and privileges.
You can test this from the command line using:
mysql -u root -p password123 - h xxx.xxx.xxx.xxx db
If this does not work then debug as follow:
ping xxx.xxx.xxx.xxx. If host is reachable move on to next step, if not then this IP is blocked, not available or incorrect. Double check the IP and check that they are on the same network.
Check if mysqld is running on host. service mysqld restart. If it is move on to next step, if not start mysqld. If it does not want to start, install it, start the service and setup your database.
Telnet the specific port to see if the port is blocked. telnet xxx.xxx.xxx.xxx 3306. If this works, move on to the next step. If this does not work, check your IPTables and check if the port is open on the remote host.
Add a user to the mysql on the the host: https://dev.mysql.com/doc/refman/8.0/en/adding-users.html
Restart mysqld and try the command above again.