My app is deployed in GCP, I'm trying to make a connection to DB using psycopg2. The SSL certificates and key are not stored as files, so I'll be getting them as strings.
When I try to make a connection by passing the filepath for these certificate pem files, it works.
psycopg2.connect(host='hostname',port=1234, connect_timeout=100, database='db', user='user', password='pwd',
sslrootcert="server-cert.pem",
sslcert="client-cert.pem",
sslkey="key.pem")
But when I pass certificates and key as strings, it doesn't work. It gives an error
FATAL: connection requires a valid client certificate\nconnection to
server at "hostname", port 1234 failed
SERVER_CERT = """-----BEGIN CERTIFICATE-----\nxxxxxx\n-----END CERTIFICATE-----"""
CLIENT_CERT = """-----BEGIN CERTIFICATE-----\nxxxxxx\n-----END CERTIFICATE-----"""
KEY = """-----BEGIN RSA PRIVATE KEY-----\nxxxxn-----END RSA PRIVATE KEY-----"""
psycopg2.connect(host='hostname',port=1234, connect_timeout=100, database='db', user='user', password='pwd',
sslrootcert=SERVER_CERT,
sslcert=CLIENT_CERT,
sslkey=KEY)
I also tried using ssl.DER_cert_to_PEM_cert(CERT) and RSA.importKey(KEY), but it still fails.
Is there a way to pass string instead of files? Thanks.
I was facing the same situation a couple of hours ago. What I did to resolve this is creating the files in python with the value of the variable:
cert = """cert"""
file = open("cert.txt","w")
file.write(cert)
file.close()
And then, just pass the path to the psycopg2 connection.
Related
I have been able to sshtunnel into a remote database via the following code utilizing passwords
import sshtunnel
import mysql.connector
import numpy as np
import pandas as pd
host =""
username=""
password=""
tunnel_username=""
tunnel_password=""
with sshtunnel.SSHTunnelForwarder(
(host, 22),
ssh_username=username,
ssh_password=password,
remote_bind_address=('localhost', 3306),
local_bind_address=('0.0.0.0', 3306)
) as tunnel:
connection = mysql.connector.connect(
user= tunnel_username,
password= tunnel_password,
host='localhost',
database= database,
port=3306)
data = pd.read_sql_query(query, connection)
connection.close()
print(data)
However, circumstances have changed, and I have been forced to only connect via SSH keys (generated with PuTTYgen). With that being said, I have the private key (ppk file), but it is unclear what I need to do (or if possible) to get the following code to work again.
I have not seen an option to reference the ppk file path instead of the sshtunnel password.
Use ssh_pkey parameter of SSHTunnelForwarder constructor to provide the path to your private key file.
And you will need to convert your private key file to the OpenSSH format, as Paramiko (used by sshtunnel) does not support PuTTY key format.
See How to ssh connect through python Paramiko with ppk public key
im struggling (since a few days) to connect to our db2 database on an as400 over ssl.
For the database connection im using jaydebeapi and to access the database with a secure connection i need to append the trustStore location and trustStore password to the JVM over a jpype function.
Here is a snipped what ive done so far:
import jaydebeapi
import os
import jpype
import pandas as pd
from credentials_getter import get_db2_credentials
class i5Connection:
def __init__(self, user, password):
if jpype.isJVMStarted():
print("already started!")
filepath_script = os.path.dirname(os.path.abspath(__file__))
ssl_trust_store_location = filepath_script + "/database/java-certs.jks"
jar = filepath_script + '/database/jt400.jar'
jvm_path = jpype.getDefaultJVMPath()
jpype.startJVM(jvm_path,
'-Djava.class.path=%s' % jar,
'-Djavax.net.ssl.trustStore=%s' % ssl_trust_store_location +
'-Djavax.net.ssl.trustStorePassword=pw')
connection_string = 'jdbc:as400://172.17.0.1/library'
print(connection_string)
self.conn = jaydebeapi.connect('com.ibm.as400.access.AS400JDBCDriver',
connection_string,
{'user': user,
'password': password,
'secure': 'true'},
filepath_script + '/database/jt400.jar')
Right now im running into the following error message:
jpype._jexception.java.sql.SQLExceptionPyRaisable: java.sql.SQLException: The application requester cannot establish the connection. (PKIX path building failed: sun.security.provider.certpath.SunCertPathBuilderException: unable to find valid certification path to requested target)
If i remove the trustStoreLocation, password from the startJVM call and the secure parameter for the jaydebeapi call, it works (without encryption)
Can anyone please tell me what im doing wrong? Is there probably a good ibm documentation i didnt find yet?
Edit: Ive found a workaround: I just saved the certificate to the cacert manually via keytool: How do I import an existing Java keystore (.jks) file into a Java installation?
not really a solution, but hopefully helpful if someone is stuck as well...
This isn't a JayDeBeApi solution, but I am able to connect to HANA using Python with an encrypted connection using this solution on a Mac and Windows.
I'm trying to use psycopg2 to connect to a cockroach DB server where client auth is required.
The connection strings typically look like this:
import psycopg2
c = psycopg2.connect("host=myhost dbname=mydb sslmode=verify-full sslcert=/root/.postgresql/aa/postgresql.crt sslkey=/root/.postgresql/aa/postgresql.key")
However, my key file is encrypted. My question is: Can psycopg2 handle password encrypted key files? What happens if postgresql.key is password protected? Does connect() prompt for a password to unwrap the key? Can a password to the key file be provided in the connection string?
I have to connect to other server in python using the pysftp library, the target server had a key value pair file (pem file), and I've got the following exception:
paramiko.ssh_exception.BadAuthenticationType: ('Bad authentication type', [u'publickey']) (allowed_types=[u'publickey'])
my Code:
import pysftp
pysftp.Connection(host="<IP address>", username="myUserName", password="no password", port=22, private_key="myPemFilePath.pem")
Please any help? and how can I fix such like this issue ?
From the documentation :
import pysftp
with pysftp.Connection('hostname', username='me', private_key='/path/to/keyfile') as sftp:
#
# ... do sftp operations
#
As you can see there is no password= "no password", in ther. Try, by just omitting that in your code, as it probably triggers the use of username/password authentication, skipping your private_key.
I want to connect to a remote PostgreSQL database through Python to do some basic data analysis. This database requires SSL (verify-ca), along with three files (which I have):
Server root certificate file
Client certificate file
Client key file
I have not been able to find a tutorial which describes how to make this connection with Python.
Any help is appreciated.
Use the psycopg2 module.
You will need to use the ssl options in your connection string, or add them as key word arguments:
import psycopg2
conn = psycopg2.connect(dbname='yourdb', user='dbuser', password='abcd1234', host='server', port='5432', sslmode='require')
In this case sslmode specifies that SSL is required.
To perform server certificate verification you can set sslmode to verify-full or verify-ca. You need to supply the path to the server certificate in sslrootcert. Also set the sslcert and sslkey values to your client certificate and key respectively.
It is explained in detail in the PostgreSQL Connection Strings documentation (see also Parameter Key Words) and in SSL Support.
You may also use an ssh tunnel with paramiko and sshtunnel:
import psycopg2
import paramiko
from sshtunnel import SSHTunnelForwarder
mypkey = paramiko.RSAKey.from_private_key_file('/path/to/private/key')
tunnel = SSHTunnelForwarder(
(host_ip, 22),
ssh_username=username,
ssh_pkey=mypkey,
remote_bind_address=('localhost', psql_port))
tunnel.start()
conn = psycopg2.connect(dbname='gisdata', user=psql_username, password=psql_password, host='127.0.0.1', port=tunnel.local_bind_port)
If you need to connect to your PostgresSQL database with an SSL certificate using psycopg2, you'll need to put your certificate SSL certificate in a subdirectory of your python program, and then you can reference the certificate in your connection string. I believe you could also set an environment variable as well, but in my example my SSL certificate will be in a subdirectory.
My python script is in a directory which looks like:
/Users/myusername/Desktop/MyCoolPythonProgram/test_database_connection.py
And my SSL certificate is in a directory which looks like:
/Users/myusername/Desktop/MyCoolPythonProgram/database/ssl_certificate/database/ssl_certificate/ca-certificate.crt
My HOSTNAME is a URL from DigitalOcean, but yours might be an IP Address instead.
This is what my test_database_connection.py script looks like:
import psycopg2
import os
POSTGRES_DATABASE_HOST_ADDRESS = "your-database-name-do-user-12345678-0.b.db.ondigitalocean.com"
POSTGRES_DATABASE_NAME = "defaultdb"
POSTGRES_USERNAME = "doadmin"
POSTGRES_PASSWORD = "$uperD00P3Rp#$$W0RDg0E$here"
# HOW TO (Relative Path Python): https://stackoverflow.com/questions/918154/relative-paths-in-python
path_to_current_directory = os.path.dirname(__file__)
relative_path_to_ssl_cert = 'database/ssl_certificate/ca-certificate.crt'
SSL_ROOT_CERT = os.path.join(path_to_current_directory , relative_path_to_ssl_cert )
POSTGRES_CONNECTION_PORT = "1234" # Set this to the correct port! Mine is provided by DigitalOcean and it's NOT 1234
db_info = "host='%s' dbname='%s' user='%s' password='%s' sslmode='require' sslrootcert='%s' port='%s'" % (POSTGRES_DATABASE_HOST_ADDRESS, POSTGRES_DATABASE_NAME, POSTGRES_USERNAME, POSTGRES_PASSWORD, SSL_ROOT_CERT, POSTGRES_CONNECTION_PORT)
postgres_connection = psycopg2.connect(db_info)
with postgres_connection:
with postgres_connection.cursor() as postgres_cursor:
sql = "SELECT * FROM your_table;"
postgres_cursor.execute(sql)
results = postgres_cursor.fetchall()
for row in results:
print("row in result")
print("Connection Success!")
# Close Database Cursor/Connection
postgres_cursor.close()
Adding this for completeness and because I couldn't find it anywhere else on SO. Like #mhawke says, you can use psycopg2, but you can also use any other Python database modules (ORMs, etc) that allow you to manually specify a database postgresql URI (postgresql://[user[:password]#][netloc][:port][/dbname][?param1=value1&...]) to connect to since the sslmode="require" parameter that psycopg2.connect uses to enforce ssl connections is just part of the postgresql:// URI that you use to connect to your database (see 33.1.2. Parameter Key Words). So, if you wanted to use sqlalchemy or another ORM instead of vanilla psycopg2, you can tack your desired sslmode onto the end of your database URI and connect that way.
import sqlalchemy
DATABASE_URI = "postgresql://postgres:postgres#localhost:5432/dbname"
# sqlalchemy 1.4+ uses postgresql:// instead of postgres://
ssl_mode = "?sslmode=require"
DATABASE_URI += ssl_mode
engine = sqlalchemy.create_engine(URI)
Session = sqlalchemy.orm.sessionmaker(bind=engine)
There's a nifty figure (Table 33.1) in the postgres documentation on SSL Support that breaks down the different options you can supply. If you want to use any of the fancier options that require you to specify a path to a specific certificate, you can drop it in with a format string.