Develop Reference

"WinError 10061" When trying to connect to Cloud SQL using Cloud SQL Python Connector

I have created a Cloud SQL instance, and a new database. However, I can't seem to connect to the database using Cloud SQL Python Connector. I have followed the sample code and steps in the documentation, but still failed.
Error:
Traceback (most recent call last):
File "c:\Users\my_name\my_path\gl_cloudsql_mysql_fx.py", line 149, in <module>
print(bnm_data_db.isTableExist('MY_TABLE'))
File "c:\Users\my_name\my_path\gl_cloudsql_mysql_fx.py", line 81, in isTableExist
with self.__pool.connect() as db_conn:
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\engine\base.py", line 3245, in connect
return self._connection_cls(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\engine\base.py", line 145, in __init__
self._dbapi_connection = engine.raw_connection()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\engine\base.py", line 3269, in raw_connection
return self.pool.connect()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 455, in connect
return _ConnectionFairy._checkout(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 1270, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 719, in checkout
rec = pool._do_get()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\impl.py", line 168, in _do_get
with util.safe_reraise():
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\util\langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\impl.py", line 166, in _do_get
return self._create_connection()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 396, in _create_connection
return _ConnectionRecord(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 681, in __init__
self.__connect()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 905, in __connect
with util.safe_reraise():
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\util\langhelpers.py", line 147, in __exit__
raise exc_value.with_traceback(exc_tb)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 901, in __connect
self.dbapi_connection = connection = pool._invoke_creator(self)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\sqlalchemy\pool\base.py", line 368, in <lambda>
return lambda rec: creator_fn()
File "c:\Users\my_name\my_path\gl_cloudsql_mysql_fx.py", line 27, in getconn
conn: pymysql.connections.Connection = self.__connector.connect(
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\connector.py", line 154, in connect
return connect_task.result()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\concurrent\futures\_base.py", line 458, in result
return self.__get_result()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\concurrent\futures\_base.py", line 403, in __get_result
raise self._exception
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\connector.py", line 261, in connect_async
return await asyncio.wait_for(get_connection(), timeout)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\asyncio\tasks.py", line 445, in wait_for
return fut.result()
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\connector.py", line 257, in get_connection
return await self._loop.run_in_executor(None, connect_partial)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\concurrent\futures\thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\site-packages\google\cloud\sql\connector\pymysql.py", line 54, in connect
socket.create_connection((ip_address, SERVER_PROXY_PORT)),
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\socket.py", line 845, in create_connection
raise err
File "C:\Users\my_name\Anaconda3\envs\composer-bis-gl-env\lib\socket.py", line 833, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it
I run this python code on locally on my machine (in a Conda environment).
Here's my code:
from google.cloud.sql.connector import Connector
import sqlalchemy
import pymysql
connector = Connector()
# function to return the database connection
def getconn() -> pymysql.connections.Connection:
conn: pymysql.connections.Connection = connector.connect(
"my_project:my_location:my_instance",
"pymysql",
user="user",
password="password",
db="my_database",
)
return conn
# create connection pool
pool = sqlalchemy.create_engine(
"mysql+pymysql://",
creator=getconn,
)
# insert statement
check_table_statement = sqlalchemy.text(
"SELECT count(*) from information_schema.tables",
)
with pool.connect() as db_conn:
result = db_conn.execute(check_table_statement)
print(result)
connector.close()
I have tried to include my IP address in the Cloud SQL instance. However, I am not sure which one is the correct address. I have check my IP address multiple ways, but all of them are different:
Google
whatsmyip.org
cmd command nslookup myip.opendns.com resolver1.opendns.com
No luck with all of them.
Note:
I did not check the Allow only SSL connections box. So, I reckon, there is no need to include the CA certificate.
The CloudSQL instance is using Public IP.
Update:
I have added full error.

To solve this, check the following -
Make sure you enabled Public IP on the Cloud SQL instance
Then, create a user with hosts allowed as % (all IPs).
Next, ensure that you have a database created by the name of my_database by clicking on the Databases button on the left navigation inside Cloud SQL dashboard.
Ensure that you are including the right service credentials in your script with Cloud SQL Client role.
You should be able to connect. Do let me know if you face any issues, will update the answer accordingly :)

A couple things for you to try in order to resolve the ConnectionRefusedError.
Make sure the Cloud SQL Admin API is enabled within your Google Cloud Project.
Make sure the IAM principal you are using for authorization within your environment has the Cloud SQL Client Role granted to it.
Double check that your instance connection name is correct (first argument being passed to the .connect method) by verifying it on the Cloud SQL instance overview page.
If you are connecting to a Cloud SQL instance Private IP make sure you are connecting from a machine within the VPC network and that your code is updated for private IP connections:
from google.cloud.sql.connector import Connector, IPTypes
import sqlalchemy
import pymysql
connector = Connector()
# function to return the database connection
def getconn() -> pymysql.connections.Connection:
conn: pymysql.connections.Connection = connector.connect(
"my_project:my_location:my_instance",
"pymysql",
user="user",
password="password",
db="my_database",
ip_type=IPTypes.PRIVATE,
)
return conn
Let me know if any of these help resolve your connection, otherwise I can take a deeper look into it for you :)

What's the minimum amount of priviliges required to interact with a mysql database using python mysql connector?

I've created a custom user by executing:
mysql> CREATE USER 'myuser'#'localhost' IDENTIFIED BY 'mypwd';
mysql> GRANT select, update, alter ON mydb.* TO myuser#localhost;
My user only needs to read data from the database, update entries and change a few table's structure using the ALTER command.
Now I want to execute the following script:
import mysql.connector
db = mysql.connector.connect(
host="localhost",
user="myuser",
passwd="mypwd",
database="mydb")
print(db);
I don't fully understand why, but following error occurrs:
Traceback (most recent call last):
File "conneciton.py", line 3, in <module>
weewxDb = mysql.connector.connect(
File "/usr/lib/python3/dist-packages/mysql/connector/__init__.py", line 173, in connect
return MySQLConnection(*args, **kwargs)
File "/usr/lib/python3/dist-packages/mysql/connector/connection.py", line 102, in __init__
self.connect(**kwargs)
File "/usr/lib/python3/dist-packages/mysql/connector/abstracts.py", line 735, in connect
self._open_connection()
File "/usr/lib/python3/dist-packages/mysql/connector/connection.py", line 250, in _open_connection
self._do_auth(self._user, self._password,
File "/usr/lib/python3/dist-packages/mysql/connector/connection.py", line 172, in _do_auth
self._auth_switch_request(username, password)
File "/usr/lib/python3/dist-packages/mysql/connector/connection.py", line 216, in _auth_switch_request
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1044 (42000): Access denied for user 'myuser'#'localhost' to database 'mydb'
I think my user is missing some privileges. But which? I want to give him as few permissions as possible.

Python mysql connector change host link to ipaddress

I am coming across an issue with mysql connector.
I have an environment that i can't connect from my local machine.
I use simple command for connection:
from mysql.connector import connect
connection = mysql.connector.connect(user='dbuser', database='dbname',
host='amazon_link', password='dbpassword')
This works for all environment except one. And this also works in the server as well where the database lives.
I can access the database using my MySqlWorkbench from local machine. But when i try from my script, i get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/__init__.py", line 179, in connect
return MySQLConnection(*args, **kwargs)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 95, in __init__
self.connect(**kwargs)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/abstracts.py", line 719, in connect
self._open_connection()
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 210, in _open_connection
self._ssl)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 144, in _do_auth
self._auth_switch_request(username, password)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 177, in _auth_switch_request
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1045 (28000): Access denied for user 'dbuser'#'10.0.1.72' (using password: YES)
And the ip address that i get for the host link is different from what mysql connector is showing here as well.
Is there a way to avoid mysql connector to change link to url here, as when i use the link instead of ip address in my Workbench i am able to connect. So i doubt it is an issue related to vpn or security groups. Otherwise i wouldn't be able to connect to database via work bench at all.
Any help would be appreciated. Thanks.
And sorry in case i missed something basic here.

Try to use the global IP address of your server instead of the domain name.
Refer the stackoverflow link for more details:
Remotely connect to MySQL with Python mysql.connector

ssh connection in python

I'm trying to read a text file from a server using ssh from python 3.5. I'm using paramiko to connect to the server but unfortunately, I'm having trouble actually connecting to the server.
this is the code I'm using to connect to the server
import paramiko
ssh = paramiko.SSHClient()
ssh.set_missing_host_key_policy(paramiko.AutoAddPolicy())
ssh.connect('IP ADDRESS OF THE SERVER', key_filename="/home/user/.ssh/id_ecdsa" ,look_for_keys=True)
but every time I try to connect to the server I'm getting an authentication failed error message, can anyone see what I'm doing wrong here?
Any advice would be appreciated
This is the error I'm getting everytime i connect.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 380, in connect
look_for_keys, gss_auth, gss_kex, gss_deleg_creds, gss_host)
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 603, in _auth
raise saved_exception
File "/usr/local/lib/python3.5/site-packages/paramiko/client.py", line 580, in _auth
allowed_types = set(self._transport.auth_publickey(username, key))
File "/usr/local/lib/python3.5/site-packages/paramiko/transport.py", line 1331, in auth_publickey
return self.auth_handler.wait_for_response(my_event)
File "/usr/local/lib/python3.5/site-packages/paramiko/auth_handler.py", line 208, in wait_for_response
raise e
paramiko.ssh_exception.AuthenticationException: Authentication failed.

You should use your private key to connect to a remote server. Your public key must be already installed in the server side, i.e. it must be listed in ~/.ssh/authorized_keys.
Try first from the command line, and only then use Python/paramiko. Check the permissions of the files/directories if all that fails.

Turns out I simply just forgot to add in the username in the connection string. works perfectly now.

pymysql cannot connect with SSL

I am trying to connect to a MySQL database using python but I am getting a strange error. It is compounded by the fact that I can use the same connection values from the mysql console command and it connects with no problems.
Here is the exact code I am using:
import pymysql
from checks import AgentCheck
class DelayedJobCheck(AgentCheck):
def check(self, instance):
self.log.info("testing connection")
self.log.info(instance)
connection = pymysql.connect(**instance)
cur = cnx.cursor(buffered=True)
cur.execute("SHOW STATUS LIKE 'Ssl_cipher'")
print(cur.fetchone())
cur.close()
cnx.close()
self.gauge('hello.world', 1)
This is the error that I am getting:
Traceback (most recent call last):
File "/opt/datadog-agent/agent/checks/__init__.py", line 661, in run
self.check(copy.deepcopy(instance))
File "/opt/datadog-agent/agent/checks.d/delayed_job.py", line 10, in check
connection = pymysql.connect(**instance)
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/__init__.py", line 88, in Connect
return Connection(*args, **kwargs)
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/connections.py", line 644, in __init__
self._connect()
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/connections.py", line 869, in _connect
raise exc
OperationalError: (2003, u"Can't connect to MySQL server on '192.168.199.86' ([SSL: SSL_NEGATIVE_LENGTH] dh key too small (_ssl.c:590))")
I am running this code on a Ubuntu box and I though initially that it might be because the SSL CA is a self generated cert. So I followed the steps here But, it did not make any difference. Also I have verified that the process that is running this code has full access to the cert files
Any ideas what else might be causing this?

As the err info said dh key is too small, a larger one might help. Replace the default dh512.pem file with dh4096.pem
sudo wget "https://git.openssl.org/gitweb/?p=openssl.git;a=blob_plain;f=apps/dh4096.pem" -O dh4096.pem
Ref: http://www.alexrhino.net/jekyll/update/2015/07/14/dh-params-test-fail.html