I am trying to connect to a GCP Cloud SQL Instance using Python 3 and do not want to use the cloud proxy, I just want to connect directly using SSL certs so I followed the GCP guide here to connect from a public IP secured with ssl keys.
Using this works for the mysql client:
mysql -uroot -pMyPassWord -h 1.2.3.4 --ssl-ca=server-ca.pem --ssl-cert=client-cert.pem --ssl-key=client-key.pem
But when I do what I believe is the same in Python I get an error:
from sqlalchemy import create_engine
db_connect_string='mysql+mysqldb://root:MyPassWord#1.2.3.4:3306/mydb'
ssl_args = {'ssl': {'ssl_cert':'./client-cert.pem', 'ssl_key':'./client-key.pem', 'ssl_ca':'./server-ca.pem'}}
engine = create_engine(db_connect_string, connect_args=ssl_args)
print(engine.table_names())
The error is:
sqlalchemy.exc.OperationalError:
(MySQLdb._exceptions.OperationalError) (1045, "Access denied for user
'root'#'1.2.3.4' (using password: YES)") (Background on this error at:
http://sqlalche.me/e/e3q8)
Which is straight forward enough if it wasn't for the fact that I:
Have already added my public IP address 1.2.3.4/32 as an authorised network.
I can access via mysql client so why does the restriction not apply there?
What am I missing?
OK finally got this working. The error message is misleading as you'd expect it to be related to whitelisting my IP but it's not. Here is the working code:
from sqlalchemy import create_engine
db_connect_string='mysql+mysqldb://root:MyPassWord#1.2.3.4:3306/mydb'
ssl_args = {'ssl': {'cert':'./client-cert.pem', 'key':'./client-key.pem', 'ca':'./server-ca.pem'}}
engine = create_engine(db_connect_string, connect_args=ssl_args)
print(engine.table_names())
The mysqlclient needs a dictionary called ssl with key pairs but all other answers I could find on stack had either the wrong ones or maybe they've been changed.
Here's the link to the dictionary required to pass as an argument:
https://mysqlclient.readthedocs.io/user_guide.html#installation
Here is the MySQL documentation which explains the arguments:
https://dev.mysql.com/doc/refman/8.0/en/mysql-ssl-set.html
Full list of arguments here:
mysql: The connection handler returned from mysql_init().
key: The path name of the client private key file.
cert: The path name of the client public key certificate file.
ca: The path name of the Certificate Authority (CA) certificate file.
This option, if used, must specify the same certificate used by the
server.
capath: The path name of the directory that contains trusted SSL CA
certificate files.
cipher: The list of permissible ciphers for SSL encryption.
Related
I'm trying to connect to a 21c ATP and 19c ADP (free tier, ACL enabled/configured with "My Address", TLS enabled (mTLS set to "Not required"), connection string contains "ssl_server_dn_match=yes") using Python's thin client but at the point of making a connection or setting up a connection pool, I get:
OperationalError: DPY-6005: cannot connect to database. Connection
failed with "[SSL: CERTIFICATE_VERIFY_FAILED] certificate verify
failed: self signed certificate in certificate chain (_ssl.c:1131)"
Envioronment:
DB: ATP 21c and ADP 19c
Python client library: oracledb-1.2.1 (I've tried 1.2.0 and 1.1.1, as well, but to no avail)
Environment: Python 3.10.4 and 3.8.10 (running on Mac OS)
Code sample:
import oracledb
# copied from the ATP's "Database Connection"
cs='''(description= (retry_count=20)(retry_delay=3)(address=(protocol=tcps)(port=1521)(host=adb.uk-london-1.oraclecloud.com))(connect_data=(service_name=xxxx.adb.oraclecloud.com))(security=(ssl_server_dn_match=yes)))'''
connection = oracledb.connect(user="admin", password="<password>", dsn=cs)
with connection.cursor() as cursor:
try:
sql = """select systimestamp from dual"""
for r in cursor.execute(sql):
print(r)
except oracledb.Error as e:
error, = e.args
print(error.message)
print(sql)
if (error.offset):
print('^'.rjust(error.offset+1, ' '))
References:
I've used the following documents as a reference:
https://blogs.oracle.com/opal/post/easy-way-to-connect-python-applications-to-oracle-autonomous-databases
https://blogs.oracle.com/developers/post/writing-a-flask-application-using-python-oracledb
https://python-oracledb.readthedocs.io/en/latest/user_guide/installation.html
https://docs.oracle.com/en/cloud/paas/autonomous-database/adbsa/connecting-python-tls.html#GUID-CA446B91-BC48-4A66-BF69-B8D54B9CBAD4
That error tells you that the certificate supplied by the server is not one that any local certificate authority recognizes (which is necessarily the case with self-signed certificates). Two options are available to resolve this:
Tell the OS the certificate is acceptable by adding it to the OS certificate "store"
Use an Oracle wallet (ewallet.pem) that contains the relevant certificates and set the wallet_location parameter appropriately. This was discussed in this issue.
I'm new to programming and am building my first app.. I'm building a kivy app trying to use mongoDB as the database. I can connect to a localhost to query and create documents. I cannot get it to connect to the atlas no matter what I try. I'm also using Pycharm and a venv.
Heres the basic info:
import pymongo
from pymongo.server_api import ServerApi
import mongoengine as mongo
import ssl
data = 'events'
username = 'admin'
password = 'abc123'
host_name = 'mongodb+srv://events.xfmhxnj.mongodb.net'
uri = f'mongodb+srv://{username}:{password}#events.xfmhxnj.mongodb.net/'
mongo.connect(db=db,
username=username,
password=password,
host=host_name)
class Obj(mongo.Document):
name = mongo.StringField(required=True)
div = Obj()
div.name = 'test'
div.save()
which gives me this error: raise ServerSelectionTimeoutError( pymongo.errors.ServerSelectionTimeoutError: ac-liums0m-shard-00-00.xfmhxnj.mongodb.net:27017: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate
Then i looked up this error and found someone recommended:
mongo.connect(db=db,
username=username,
password=password,
host=host_name,
ssl=True,
ssl_cert_reqs=ssl.CERT_NONE,)
)
error: raise ConnectionFailure(f"Cannot connect to database {alias} :\n{e}") mongoengine.connection.ConnectionFailure: Cannot connect to database default : Unknown option ssl_cert_reqs
I don't understand why its an unknown option. pymongo has it listed in the example.
https://api.mongodb.com/python/3.3.0/examples/tls.html
I've also tried the string straight from Atlas:
client = pymongo.MongoClient(f"mongodb+srv://{username}:{password}#events.xfmhxnj.mongodb.net/?retryWrites=true&w=majority", server_api=ServerApi('1'))
db = client.test
client.server_info()
error: raise ServerSelectionTimeoutError( pymongo.errors.ServerSelectionTimeoutError: ac-liums0m-shard-00-01.xfmhxnj.mongodb.net:27017: [SSL: CERTIFICATE_VERIFY_FAILED] certificate verify failed: unable to get local issuer certificate
or if i just paste in the uri string from compass
mongo.connect('mongodb+srv://admin:abc123#events.xfmhxnj.mongodb.net/test')
error: raise InvalidName("database names cannot contain the character %r" % invalid_char) pymongo.errors.InvalidName: database names cannot contain the character '.'
any help is super appreciated! I feel like i've tried every combination of connection settings. This is the last thing i need before pushing my app to x-code
Have you tried using MongoClient(connection_string, tlsCAFile=certifi.where()) ?
Certifi provides Mozilla’s carefully curated collection of Root
Certificates for validating the trustworthiness of SSL certificates
while verifying the identity of TLS hosts.
Before testing the new code remember to do "pip install certifi".
AWS says as below.
##To encrypt data in transit, download the public key for Amazon DocumentDB named rds-combined-ca-bundle.pem
import pymongo
##Create a MongoDB client, open a connection to Amazon DocumentDB as a replica set and specify the read preference as secondary preferred
client = pymongo.MongoClient('mongodb://<sample-user>:<password>#sample-cluster.node.us-east-1.docdb.amazonaws.com:27017/?ssl=true&ssl_ca_certs=rds-combined-ca-bundle.pem&replicaSet=rs0&readPreference=secondaryPreferred&retryWrites=false')
But I can connect aws-documentdb without using "rds-combined-ca-bundle.pem" on pymongo.
import ssl
from pymongo import MongoClient
client = MongoClient(db_host, db_port, username=db_user, password=db_password, ssl=True, ssl_cert_reqs=ssl.CERT_NONE)
Why can I connect without using "rds-combined-ca-bundle.pem" ??
If you check the Mongo documentation, when you specify ssl_cert_reqs=ssl.CERT_NONE, you are telling PyMongo to bypass the certificate validation and still allow connection to the server.
https://api.mongodb.com/python/3.3.0/examples/tls.html
However, when you specify the certificate, it would perform server validation using the certificate provided which is a recommended approach.
To get the latest .pem file use this link:
https://docs.aws.amazon.com/documentdb/latest/developerguide/ca_cert_rotation.html
I am trying to send some commands to a remote Postgres server using SQLAlchemy but each time I receive an error.
Please note that I can connect to the remote Postgres using SSH username and password to login to the server. For that I have used my local terminal, PuTTY and WinSCP so the problem appears to be in the Python code I have written
# create postgres engine to connect to the database
engine = create_engine('postgres://server_username:server_password#server_name:port/database')
with engine.connect() as conn:
ex = conn.execute("SELECT version();")
conn.close() # not needed but keep just in case
print(ex)
Running the code above yields the following error:
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) SSL SYSCALL error: Connection reset by peer (0x00002746/10054)
expected authentication request from server, but received S
I have also tried adding the SSL verification parameter as follows
create_engine('postgres://server_username:server_password#server_name:port/database?sslmode=verify-full')
which returned the error
sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) root certificate file "C:\Users\aris.pavlides\AppData\Roaming/postgresql/root.crt" does not exist
Either provide the file or change sslmode to disable server certificate verification.
at which point I had nothing to lose so I disabled certificate verification altogether
create_engine('postgres://server_username:server_password#server_name:port/database?sslmode=disable')
which returned the initial error message.
Do you have any ideas on how I can modify the code to make it work?
Setting up a new server that I want to communicate to a central MySQL database, using a TLS connection for security
Following steps like this I have been able to set up TLS for my MySQL server, I have made several users that are able to login from any host (%), and require SSL connections
+------------+-----------+----------+
| user | host | ssl_type |
+------------+-----------+----------+
| testuser | % | ANY |
+------------+-----------+----------+
I can confirm this on any host by connecting using tools like HeidiSQL or the MySQL CLI tool
Ex: mysql -u testuser -p -h mysql_server_IP this will initiate a TLS connection, as confirmed by \s
This rules out the majority of issues I have seen on this and other forums, which is caused by the host being set to localhost.
When accessing local databases, the following works fine. When connecting to non-TLS remote database servers, it also works fine.
import pymysql.cursors
connection = pymysql.connect(host=host,
user=user,
password=password,
db=db,
cursorclass=pymysql.cursors.DictCursor)
When attempting to access my server with require-tls I receive the following error
pymysql.err.OperationalError: (1045, "Access denied for user 'testuser'#'desktop.example.com' (using password: YES)")
The other findings I have suggest that that error is caused by:
Invalid username / password combinations, or, connection prohibited by that host. However I know I can make connections from this host, as demonstrated by the CLI.
When connecting to a server that only has the require_secure_transport = ON in my.cnf, PyMySQL gives a more obvious error around being unable to start a TLS connection. pymysql.err.InternalError: (3159, 'Connections using insecure transport are prohibited while --require_secure_transport=ON.' But if the MySQL user itself requires SSL, you get the more generic permission denied error from the question above.
On the github issue tracker there is mention of supplying the CA .pem file. If you don't have access to these files and want to trust the self signed cert implicitly. The docs mention the -ssl flag, which allows you to pass in paths for various cert files.
However, by passing in a valid dictionary, without any of the valid keys, you can effectively blanket trust self-signed certs. Example:
connection = pymysql.connect(host=host,
user=user,
password=password,
db=db,
cursorclass=pymysql.cursors.DictCursor,
ssl={"fake_flag_to_enable_tls":True})