Develop Reference

Trouble connecting to Cloud SQL in python

I am trying to connect to a database on Cloud SQL, but I keep getting the same error. Not sure what it is, and tried several approaches.
Input:
import pymysql
connection = pymysql.connect(host='127.0.0.1',
user='',
password='XXXX',
db='cmcsql')
output:
C:\Users\Ejer\anaconda3\envs\pythonProject\python.exe C:/Users/Ejer/PycharmProjects/pythonProject/CloudSQL_test.py
Traceback (most recent call last):
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 569, in connect
sock = socket.create_connection(
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\socket.py", line 808, in create_connection
raise err
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\socket.py", line 796, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] Der kunne ikke oprettes forbindelse, fordi destinationscomputeren aktivt nægtede det
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/Ejer/PycharmProjects/pythonProject/CloudSQL_test.py", line 2, in <module>
connection = pymysql.connect(host='127.0.0.1',
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\__init__.py", line 94, in Connect
return Connection(*args, **kwargs)
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 327, in __init__
self.connect()
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 619, in connect
raise exc
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' ([WinError 10061] Der kunne ikke oprettes forbindelse, fordi destinationscomputeren aktivt nægtede det)")
Process finished with exit code 1

You should not be using 127.0.0.1 to connect to cloud sql instance.
127.0.0.1 is (in most cases) for localhost connection (when you run the db server locally on the same machine as where the client is). Instead you should be using the ip address given to your cloud sql instance. You can find it on the overview tab of the sql instance under Connect to this instance.
You should also create your user on the users tab and then use it in your code.
Don't forget about authentication, check out connections tab and read more about it here
Making sure that you have proper ip, user, existing db and connecting from authorized network should do the trick.

PyMysql connects to port 3306 by default, if your process is running on a different port it won't be able to connect.
Apart from the host, user, password parameters you also have to provide the port on which the sql process is running.

Python clickhouse_driver through ssh connection error

struggling for a week already with my issue, hope you'll be able to explain to me what is wrong here.
I'm trying to connect Clickhouse DB I have on the remote server, which is on the absolutely default settings. So I'm connecting to a remote server and creating a tunnel on my machine.
import sshtunnel as sshtunnel
from clickhouse_driver import connect
server = sshtunnel.SSHTunnelForwarder(
('host', 22),
ssh_username='username',
ssh_pkey="username.openssh",
ssh_private_key_password="password",
remote_bind_address=('localhost', 8123),
local_bind_address=('localhost', 555)
)
server.start()
conn = connect(host='localhost', port=555, database='ertb')
cursor = conn.cursor()
cursor.execute('SHOW TABLES')
cursor.fetchall()
server.stop()
and I receive this error
Traceback (most recent call last):
File "C:/Users/user/PycharmProjects/alerts/ssh_coonection.py", line 42, in <module>
cursor.execute('SHOW TABLES')
File "C:\Users\user\PycharmProjects\alerts\venv\lib\site-packages\clickhouse_driver\dbapi\cursor.py", line 102, in execute
**execute_kwargs
File "C:\Users\user\PycharmProjects\alerts\venv\lib\site-packages\clickhouse_driver\client.py", line 205, in execute
self.connection.force_connect()
File "C:\Users\user\PycharmProjects\alerts\venv\lib\site-packages\clickhouse_driver\connection.py", line 180, in force_connect
self.connect()
File "C:\Users\user\PycharmProjects\alerts\venv\lib\site-packages\clickhouse_driver\connection.py", line 256, in connect
return self._init_connection(host, port)
File "C:\Users\user\PycharmProjects\alerts\venv\lib\site-packages\clickhouse_driver\connection.py", line 237, in _init_connection
self.send_hello()
File "C:\Users\user\PycharmProjects\alerts\venv\lib\site-packages\clickhouse_driver\connection.py", line 325, in send_hello
write_binary_str(self.user, self.fout)
File "C:\Users\user\PycharmProjects\alerts\venv\lib\site-packages\clickhouse_driver\writer.py", line 19, in write_binary_str
text = text.encode('utf-8')
AttributeError: 'NoneType' object has no attribute 'encode'
I really tried to understand from where this NoneType object appears, but a bit stuck in code.

There are two issues:
is not passed user and password that leads to the error 'NoneType' object has no attribute 'encode'
used the wrong port to access to CH (clickhouse-driver is designed to communicate with ClickHouse server over native protocol (TCP) that by default use port 9000 for non-secure communication (see Issue connecting to Dockerized Clickhouse Server with Python driver for details))
import sshtunnel as sshtunnel
from clickhouse_driver import connect
with sshtunnel.SSHTunnelForwarder(
('localhost', 22),
ssh_username="root",
ssh_password="root",
remote_bind_address=('localhost', 9000)) as server:
local_port = server.local_bind_port
print(local_port)
conn = connect(f'clickhouse://default:#localhost:{local_port}/ertb')
#conn = connect(host='localhost', port=local_port, database='ertb', user='default', password='')
cursor = conn.cursor()
cursor.execute('SHOW TABLES')
print(cursor.fetchall())

pymysql.err.InternalError: (1049, "Unknown database")

I want to connect MySQL RDS DB using python from raspberrypi.(i want to get seq from MySQL table 'face' using select query.)
and I have an error but i can not fix it.
This is rds mysql connection code:
import rds_config
import pymysql
rds_host = rds_config.rds_host
name = rds_config.rds_user
password = rds_config.rds_pwd
db_name = rds_config.rds_db
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name,
connect_timeout=10)
with conn.cursor() as cur:
cur.execute("select seq from face")
conn.commit()
rds_config:
rds_host='rds endpoint'
rds_port=3306
rds_user='user'
rds_pwd='password'
rds_db='db name'
and This is traceback:
Traceback (most recent call last):
File "getRds.py", line 18, in <module>
conn = pymysql.connect(rds_host, user=name, passwd=password, db=db_name, connect_timeout=10)
File "/usr/local/lib/python2.7/dist-packages/pymysql/__init__.py", line 94, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 327, in __init__
self.connect()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 598, in connect
self._request_authentication()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 862, in _request_authentication
auth_packet = self._process_auth(plugin_name, auth_packet)
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 933, in _process_auth
pkt = self._read_packet()
File "/usr/local/lib/python2.7/dist-packages/pymysql/connections.py", line 683, in _read_packet
packet.check_error()
File "/usr/local/lib/python2.7/dist-packages/pymysql/protocol.py", line 220, in check_error
err.raise_mysql_exception(self._data)
File "/usr/local/lib/python2.7/dist-packages/pymysql/err.py", line 109, in raise_mysql_exception
raise errorclass(errno, errval)
pymysql.err.InternalError: (1049, u"Unknown database 'bsb-rds'")
i alread added ip address in vpc security group and public access is on.
it was possible to connect through mysql cli or workbench.
can anyone help me?

tl;dr you need to create bsb-rd. Execute this command: create database bsb-rds either with cur.execute() in python or in your favorite sql cli.
If you get this error, good news! You can connect to your RDS instance. This means you have the security group set up right, the host url, username, password and port are all correct! What this error is telling you is that your database bsb-rds does not exist on your RDS instance. If you have just made the RDS instance it is probably because you have not created the database yet. So create it!
Here are two ways to do this
mysql cli
In your terminal run
mysql --host=the_host_address user=your_user_name --password=your_password
Then inside the mysql shell execute
create database bsb-rd;
Now try your code again!
Python with pymysql
import rds_config
conn = pymysql.connect('hostname', user='username', passwd='password', connect_timeout=10)
with conn.cursor() as cur:
cur.execute('create database bsb-rd;')
I came to this page looking for a solution and didn't get it. I eventually found the answer and now share what helped me.

I ran into your exact issue and I believe I have the solution:
Context:
My tech stack looks like the following
Using AWS RDS (MySQL)
Using Flask Development on local host
RDS instance name: "test-rds"
Actual DB Name: test-database
Here is where my issue was and I believe your issue is in the same place:
You are using the AWS RDS NAME in your connection rather than using the true Database name.
Simply changing the DB name in my connection to the true DB name that I had setup via MySQL Workbench fixed the issue.
Other things things to note for readers:
Please ensure the following:
If you are connecting from outside your AWS VPC make sure you have public access enabled. This is a huge "gotcha". (Beware security risks)
Make sure your connection isn't being blocked by a NACL
Make sure your connection is allowed by a Security Group Rule

pymysql cannot connect with SSL

I am trying to connect to a MySQL database using python but I am getting a strange error. It is compounded by the fact that I can use the same connection values from the mysql console command and it connects with no problems.
Here is the exact code I am using:
import pymysql
from checks import AgentCheck
class DelayedJobCheck(AgentCheck):
def check(self, instance):
self.log.info("testing connection")
self.log.info(instance)
connection = pymysql.connect(**instance)
cur = cnx.cursor(buffered=True)
cur.execute("SHOW STATUS LIKE 'Ssl_cipher'")
print(cur.fetchone())
cur.close()
cnx.close()
self.gauge('hello.world', 1)
This is the error that I am getting:
Traceback (most recent call last):
File "/opt/datadog-agent/agent/checks/__init__.py", line 661, in run
self.check(copy.deepcopy(instance))
File "/opt/datadog-agent/agent/checks.d/delayed_job.py", line 10, in check
connection = pymysql.connect(**instance)
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/__init__.py", line 88, in Connect
return Connection(*args, **kwargs)
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/connections.py", line 644, in __init__
self._connect()
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/connections.py", line 869, in _connect
raise exc
OperationalError: (2003, u"Can't connect to MySQL server on '192.168.199.86' ([SSL: SSL_NEGATIVE_LENGTH] dh key too small (_ssl.c:590))")
I am running this code on a Ubuntu box and I though initially that it might be because the SSL CA is a self generated cert. So I followed the steps here But, it did not make any difference. Also I have verified that the process that is running this code has full access to the cert files
Any ideas what else might be causing this?

As the err info said dh key is too small, a larger one might help. Replace the default dh512.pem file with dh4096.pem
sudo wget "https://git.openssl.org/gitweb/?p=openssl.git;a=blob_plain;f=apps/dh4096.pem" -O dh4096.pem
Ref: http://www.alexrhino.net/jekyll/update/2015/07/14/dh-params-test-fail.html

Peewee - Can't connect to MySQL server on host

I'm developing a Flask based python app using the peewee ORM. I was initially connecting to the database that was being stored locally on my machine and I'm now trying to transition to connecting to the db remotely. I've set up the database in phpmyadmin via my server's cpanel section.
The Issue
I've set up my IP address to be able to remotely access my databases but I am getting the following error when I attempt to connect to the database:
Traceback (most recent call last):
File "app.py", line 294, in <module>
models.initialize()
File "/Users/wyssuser/Desktop/dscraper/models.py", line 145, in initialize
DATABASE.connect()
File "/Library/Python/2.7/site-packages/peewee.py", line 2767, in connect
self.__local.closed = False
File "/Library/Python/2.7/site-packages/peewee.py", line 2688, in __exit__
reraise(new_type, new_type(*exc_value.args), traceback)
File "/Library/Python/2.7/site-packages/peewee.py", line 2766, in connect
**self.connect_kwargs)
File "/Library/Python/2.7/site-packages/peewee.py", line 3209, in _connect
return mysql.connect(db=database, **conn_kwargs)
File "/Library/Python/2.7/site-packages/pymysql/__init__.py", line 88, in Connect
return Connection(*args, **kwargs)
File "/Library/Python/2.7/site-packages/pymysql/connections.py", line 644, in __init__
self._connect()
File "/Library/Python/2.7/site-packages/pymysql/connections.py", line 869, in _connect
raise exc
peewee.OperationalError: (2003, "Can't connect to MySQL server on '142.157.25.22' ([Errno 61] Connection refused)")
This is the portion of my code that references the database connection:
app.py
if __name__ == '__main__':
models.initialize()
app.run(debug=DEBUG, port=PORT, host=HOST)
config.py
DATABASE = {
'db': 'my_dbname',
'host': '142.157.25.22',
'port': 3306,
'user': 'my_username',
'passwd': 'my_pswd',
}
models.py
from peewee import *
import config
DATABASE = MySQLDatabase(config.DATABASE['db'], host=config.DATABASE['host'], port=config.DATABASE['port'], user=config.DATABASE['user'], passwd=config.DATABASE['passwd'])
...all of my models related code
def initialize():
print 'starting db connection'
DATABASE.connect()
print 'connected'
DATABASE.create_tables([Batch, Company, User, Post],safe=True)
DATABASE.close()
I've also tried connecting to 'localhost' as the host but that doesn't seem to work here, is there a different host I should be connecting to?

Solution is bad default port:
the example from peewee doc is
# Connect to a MySQL database on network.
mysql_db = MySQLDatabase('my_app', user='app', password='db_password',
host='10.1.0.8', port=3316)
but defaul port is 3306