I am trying to connect to a MySQL database using python but I am getting a strange error. It is compounded by the fact that I can use the same connection values from the mysql console command and it connects with no problems.
Here is the exact code I am using:
import pymysql
from checks import AgentCheck
class DelayedJobCheck(AgentCheck):
def check(self, instance):
self.log.info("testing connection")
self.log.info(instance)
connection = pymysql.connect(**instance)
cur = cnx.cursor(buffered=True)
cur.execute("SHOW STATUS LIKE 'Ssl_cipher'")
print(cur.fetchone())
cur.close()
cnx.close()
self.gauge('hello.world', 1)
This is the error that I am getting:
Traceback (most recent call last):
File "/opt/datadog-agent/agent/checks/__init__.py", line 661, in run
self.check(copy.deepcopy(instance))
File "/opt/datadog-agent/agent/checks.d/delayed_job.py", line 10, in check
connection = pymysql.connect(**instance)
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/__init__.py", line 88, in Connect
return Connection(*args, **kwargs)
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/connections.py", line 644, in __init__
self._connect()
File "/opt/datadog-agent/embedded/lib/python2.7/site-packages/pymysql/connections.py", line 869, in _connect
raise exc
OperationalError: (2003, u"Can't connect to MySQL server on '192.168.199.86' ([SSL: SSL_NEGATIVE_LENGTH] dh key too small (_ssl.c:590))")
I am running this code on a Ubuntu box and I though initially that it might be because the SSL CA is a self generated cert. So I followed the steps here But, it did not make any difference. Also I have verified that the process that is running this code has full access to the cert files
Any ideas what else might be causing this?
As the err info said dh key is too small, a larger one might help. Replace the default dh512.pem file with dh4096.pem
sudo wget "https://git.openssl.org/gitweb/?p=openssl.git;a=blob_plain;f=apps/dh4096.pem" -O dh4096.pem
Ref: http://www.alexrhino.net/jekyll/update/2015/07/14/dh-params-test-fail.html
Related
Thanks for taking the time to read through this question.
I've spent quite a bit of time trying to find an answer to this and some time digging through code in pydobc and mysqldb to no avail.
Summary of Problem:
I can connect to a MySQL server using the same method+connection string from many machines, but cannot use the same method from a RHEL server. I continually get an Access denied error; it seems unlikely this is actually the issue though.
Background:
I am using sqlalchemy to connect to a MySQL server hosted on a remote network. I am able to connect to the MySQL server using the same credentials from MySQL Workbench, MySQL Shell, my local Jupyter Notebook, local Python scripts, a virtual Ubuntu 18.04 WSL1 instance, and the troublesome RHEL server- but I cannot use the same connection method on RHEL as the rest of the instances.
I am positive my user has access to read/write tables on the MySQL server (and ostensibly make ODBC connections since I believe I'm doing so from my laptop), so despite the error below, I'm having a hard time believing it is actually a true permission error. That said, I'm happy to listen to any suggestions.
Problem:
When connecting from any of the sources listed above except for the RHEL server, I am able to utilize this code to create an engine and subsequently a connection:
engine = sqlalchemy.create_engine('mysql+mysqldb://{user}:{password}#{server}:{port}/{database}?charset=utf8mb4'.format(user=user
,password=password
,server=server
,port=port
,database=database))
con = engine.connect()
I am then able to use that connection to do any number of things- pd.to_sql(), direct code executions- works like a charm.
The issue comes up when I try to use that same connection info on the RHEL Server, I get the following error:
>>> engine.connect()
Traceback (most recent call last):
File "/python_dir/python3.8/site-packages/sqlalchemy/engine/base.py", line 3211, in _wrap_pool_connect
return fn()
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/base.py", line 307, in connect
return _ConnectionFairy._checkout(self)
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/base.py", line 767, in _checkout
fairy = _ConnectionRecord.checkout(pool)
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/base.py", line 425, in checkout
rec = pool._do_get()
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/impl.py", line 146, in _do_get
self._dec_overflow()
File "/python_dir/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/python_dir/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/impl.py", line 143, in _do_get
return self._create_connection()
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/base.py", line 253, in _create_connection
return _ConnectionRecord(self)
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/base.py", line 368, in __init__
self.__connect()
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/base.py", line 611, in __connect
pool.logger.debug("Error on connect(): %s", e)
File "/python_dir/python3.8/site-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__
compat.raise_(
File "/python_dir/python3.8/site-packages/sqlalchemy/util/compat.py", line 207, in raise_
raise exception
File "/python_dir/python3.8/site-packages/sqlalchemy/pool/base.py", line 605, in __connect
connection = pool._invoke_creator(self)
File "/python_dir/python3.8/site-packages/sqlalchemy/engine/create.py", line 578, in connect
return dialect.connect(*cargs, **cparams)
File "/python_dir/python3.8/site-packages/sqlalchemy/engine/default.py", line 584, in connect
return self.dbapi.connect(*cargs, **cparams)
File "/python_dir/python3.8/site-packages/MySQLdb/__init__.py", line 130, in Connect
return Connection(*args, **kwargs)
File "/python_dir/python3.8/site-packages/MySQLdb/connections.py", line 187, in __init__
super().__init__(*args, **kwargs2)
MySQLdb._exceptions.OperationalError: (1045, "Access denied for user 'USER'#'SERVER_ADDRESS' (using password: YES)")
Obviously I've changed a few things in the error for security, but that last line is the only real error I see:
MySQLdb._exceptions.OperationalError: (1045, "Access denied for user 'USER'#'SERVER_ADDRESS' (using password: YES)")
Connecting from RHEL
Despite the failure using the connection info above, I am able to connect to the MySQL server from the RHEL server with a different connection setup:
engine = sqlalchemy.create_engine('mysql+pyodbc://my_dsn')
By using pyodbc, I'm able to connect utilizing a DSN (I think DSN is the right term- this is getting a bit murky for me) that I have configured on the RHEL server in /etc/odbc.ini
Here's an example of what I have for that specific entry (I've retained capitalizations + blanks where they exist in the file, but have changed all sensitive fields):
[ODBC Data Sources]
my_dsn = My Special DSN
[my_dsn]
Driver = /dir/libmyodbc5.so
Description = Description
SERVER = Server
PORT = Port
USER = User
Password = Password
Database = Database
OPTION = 3
SOCKET =
The only major difference that I see between the DSN configuration listed above and the connection string that I'm providing to mysqldb is that the DSN lets me specify the OPTION parameter. I can't figure out how pyodbc uses this, though.
I've tried digging through the documentation from quite a few sources (sqlalchemy,pyodbc,MySQL,StackOverflow,SuperUser, direct source code) to find if/how to specify the OPTION for mysqldb to no avail. I'm not sure if that OPTION is really making that big of a difference, but it's the only piece I can't provide.
That being said, I did find this question that explains what the options are a bit more and this corresponding page that the answer came from. Although that doesn't help me much.
Goal
Ideally, I'd like to be able to run my connections the exact same regardless of server or OS. I'm not sure if that's possible, but it's what I'm attempting to do. I would rather not set up the same DSN on every machine that needs to connect (so the pyodbc option I'm using on RHEL isn't ideal), but no other connection method from that RHEL server will work.
Troubleshooting
I don't have a stellar understanding of the drivers/Linux config, so there's definitely a possibility that I've done something wrong here. Here's some stuff I've tried:
Tried many different connection strings and different drivers/connection info specified
Tried installing various different MySQL drivers on RHEL (with varied success)
Read dozens of questions and articles on similar issues
Other janky configuration attempts that didn't help
Wept softly over my keyboard
Configuration Information:
Problem server: RHEL Version 7
Other machines: Ubuntu 18.04, Windows 10
Python: 3.8
SQLAlchemy: RHEL- 1.4.20, Windows- 1.4.15, Ubuntu- 1.4.22
pyodbc: RHEL- 4.0.30, Windows- 4.0.30, Ubuntu- 4.0.31
If I can provide any more information, please let me know! Really appreciate any ideas.
I am trying to connect to a database on Cloud SQL, but I keep getting the same error. Not sure what it is, and tried several approaches.
Input:
import pymysql
connection = pymysql.connect(host='127.0.0.1',
user='',
password='XXXX',
db='cmcsql')
output:
C:\Users\Ejer\anaconda3\envs\pythonProject\python.exe C:/Users/Ejer/PycharmProjects/pythonProject/CloudSQL_test.py
Traceback (most recent call last):
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 569, in connect
sock = socket.create_connection(
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\socket.py", line 808, in create_connection
raise err
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\socket.py", line 796, in create_connection
sock.connect(sa)
ConnectionRefusedError: [WinError 10061] Der kunne ikke oprettes forbindelse, fordi destinationscomputeren aktivt nægtede det
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "C:/Users/Ejer/PycharmProjects/pythonProject/CloudSQL_test.py", line 2, in <module>
connection = pymysql.connect(host='127.0.0.1',
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\__init__.py", line 94, in Connect
return Connection(*args, **kwargs)
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 327, in __init__
self.connect()
File "C:\Users\Ejer\anaconda3\envs\pythonProject\lib\site-packages\pymysql\connections.py", line 619, in connect
raise exc
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on '127.0.0.1' ([WinError 10061] Der kunne ikke oprettes forbindelse, fordi destinationscomputeren aktivt nægtede det)")
Process finished with exit code 1
You should not be using 127.0.0.1 to connect to cloud sql instance.
127.0.0.1 is (in most cases) for localhost connection (when you run the db server locally on the same machine as where the client is). Instead you should be using the ip address given to your cloud sql instance. You can find it on the overview tab of the sql instance under Connect to this instance.
You should also create your user on the users tab and then use it in your code.
Don't forget about authentication, check out connections tab and read more about it here
Making sure that you have proper ip, user, existing db and connecting from authorized network should do the trick.
PyMysql connects to port 3306 by default, if your process is running on a different port it won't be able to connect.
Apart from the host, user, password parameters you also have to provide the port on which the sql process is running.
I'm having trouble connecting to a db I set up in Atlas MongoDB.
I have already read multiple other questions but no answer made my connection work.
I want to point out that I am trying to access it behind a company's proxy (if that has anything to do with it).
My code is this and fails only on the last line where I try to count the documents:
from pymongo import MongoClient
client = MongoClient(
"mongodb+srv://name:pass#iliastrialcluster-1tl2y.azure.mongodb.net/test?retryWrites=true&w=majority&ssl=true&ssl_cert_reqs=CERT_NONE"
)
db = client.get_database('sample_airbnb')
print(db)
listings_and_reviews = db.listingAndReviews
print(listings_and_reviews)
listings_and_reviews.count_documents({})
The error I get is:
Traceback (most recent call last):
File "C:~/mongo_connection1.py", line 11, in <module>
listings_and_reviews.count_documents({})
File "C:~\lib\site-packages\pymongo\collection.py", line 1721, in count_documents
_cmd, self._read_preference_for(session), session)
File "C:~\lib\site-packages\pymongo\mongo_client.py", line 1454, in _retryable_read
read_pref, session, address=address)
File "C:~\lib\site-packages\pymongo\mongo_client.py", line 1253, in _select_server
server = topology.select_server(server_selector)
File "C:~\lib\site-packages\pymongo\topology.py", line 235, in select_server
address))
File "C:~\lib\site-packages\pymongo\topology.py", line 193, in select_servers
selector, server_timeout, address)
File "C:~\lib\site-packages\pymongo\topology.py", line 209, in _select_servers_loop
self._error_message(selector))
pymongo.errors.ServerSelectionTimeoutError: iliastrialcluster-shard-00-00-1tl2y.azure.mongodb.net:27017: timed out,iliastrialcluster-shard-00-02-1tl2y.azure.mongodb.net:27017: timed out,iliastrialcluster-shard-00-01-1tl2y.azure.mongodb.net:27017: timed out
ServerSelectionTimeout error is the client's way of telling you you can't connect to the server. The primary cause of these errors when using MongoDB Atlas is the failure to enable access for the IP address of the node the client is running on.
To verify try connecting using the MongoDB shell mongo using the same connection string. if you get a connection failed then you know it's not a Python problem. Your client code looks OK BTW so I am pretty sure this is what it is.
The connection to the server is lazily evaluated so we don't try to initiate a connection until you make an actual request. In this case the count_documents call. This is why this is the call that generates the error.
#JoeDrumgoole Thanks for your answer. The problem was the proxy after all, after deactivating it the connection works fine. Good to know that the connection to the server is lazily evaluated.
I am coming across an issue with mysql connector.
I have an environment that i can't connect from my local machine.
I use simple command for connection:
from mysql.connector import connect
connection = mysql.connector.connect(user='dbuser', database='dbname',
host='amazon_link', password='dbpassword')
This works for all environment except one. And this also works in the server as well where the database lives.
I can access the database using my MySqlWorkbench from local machine. But when i try from my script, i get the following error:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/__init__.py", line 179, in connect
return MySQLConnection(*args, **kwargs)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 95, in __init__
self.connect(**kwargs)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/abstracts.py", line 719, in connect
self._open_connection()
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 210, in _open_connection
self._ssl)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 144, in _do_auth
self._auth_switch_request(username, password)
File "/Users/rajatvij/Development/env/lib/python2.7/site-packages/mysql/connector/connection.py", line 177, in _auth_switch_request
raise errors.get_exception(packet)
mysql.connector.errors.ProgrammingError: 1045 (28000): Access denied for user 'dbuser'#'10.0.1.72' (using password: YES)
And the ip address that i get for the host link is different from what mysql connector is showing here as well.
Is there a way to avoid mysql connector to change link to url here, as when i use the link instead of ip address in my Workbench i am able to connect. So i doubt it is an issue related to vpn or security groups. Otherwise i wouldn't be able to connect to database via work bench at all.
Any help would be appreciated. Thanks.
And sorry in case i missed something basic here.
Try to use the global IP address of your server instead of the domain name.
Refer the stackoverflow link for more details:
Remotely connect to MySQL with Python mysql.connector
Relatively new to Python and MySQL, but I'm performing a simple query of a DB in a dev environment using the MySQL Python Connector. I've created a buffered cursor to return results as dictionaries. When I perform the simple query:
family_query = ("SELECT * FROM family as FF")
...I get a list of errors, all around this idea of error 2013: Lost Connection to MySQL server.
>python "FitMatch v1.5.py"
Traceback (most recent call last):
File "FitMatch v1.5.py", line 505, in <module>
fit_match_cursor.execute(fit_family_query, () )
File "C:\Python34\lib\site-packages\mysql\connector\cursor.py", line 507, in execute
self._handle_result(self._connection.cmd_query(stmt))
File "C:\Python34\lib\site-packages\mysql\connector\cursor.py", line 421, in _handle_result
self._handle_resultset()
File "C:\Python34\lib\site-packages\mysql\connector\cursor.py", line 895, in _handle_resultset
(self._rows, eof) = self._connection.get_rows()
File "C:\Python34\lib\site-packages\mysql\connector\connection.py", line 671, in get_rows
rows = self._protocol.read_text_result(self._socket, count)
File "C:\Python34\lib\site-packages\mysql\connector\protocol.py", line 309, in read_text_result
packet = sock.recv()
File "C:\Python34\lib\site-packages\mysql\connector\network.py", line 226, in recv_plain
raise errors.InterfaceError(errno=2013)
mysql.connector.errors.InterfaceError: 2013: Lost connection to MySQL server during query
I've tried increasing my connection_timeout to well over 10,000 (as I read on other stack overflow posts that could be the issue) but it had no effect.
Any ideas what could be causing the "Lost connection to MySQL server" error?
As a print of nested query result I face this problem
my Python query would fail with the error described in the question after returning just a subset of results.
As my way you Switched to PyMySQL and things work as you like
PyMySQL Link
My Sample Code For printing data
import pymysql
connection = pymysql.connect(user='XYZ', passwd='XYZ',host='XYZ',database='XYZ')
cursor = connection.cursor()
query = ("YOUR_QUERY")
cursor.execute(query)
for item in cursor:
print item
If any problem with this answer must comment me....:)