I am trying to connect to a postgres instance I have in cloud sql. I have everything set up and am able to connect to it if ssl encryption is turned off. But now that I have it on I am trying to connect but running into some error.
def run():
connector = Connector()
def getconn():
conn = connector.connect(
os.getenv("CONNECTION_NAME"),
"pg8000",
user = os.getenv('DB_USERNAME'),
password = os.getenv("DB_PASSWORD"),
db=os.getenv('DB_NAME'),
ip_type= IPTypes.PRIVATE
)
return conn
pool = sqlalchemy.create_engine(
"postgresql+pg8000://",
creator=getconn,
pool.execute("CREATE TABLE........;")
All the certs are stored in secret manager as strings so I am using env variables to grab them, which is why I used cadata for example. But running into this error cadata does not contain a certificate why is this error coming up?
I'd recommend using the Cloud SQL Python Connector to connect to Cloud SQL from Python as it will generate the SSL context for you, meaning no need to manage SSL certificates! It also has additional benefits of not needing to authorize networks etc.
You can find a code sample for the Python Connector similar to the one you are using for establishing a TCP connection.
There is also an interactive getting started Colab Notebook that will walk you through using the Python Connector without you needing to change a single line of code!
It makes connecting to Cloud SQL both easy and secure.
Related
Moving this question from DevOps Stack Exchange where it got only 5 views in 2 days:
I would like to query an Azure Database for MySQL Single Server.
I normally interact with this database using a universal database tool (dBeaver) installed onto an Azure VM. Now I would like to interact with this database using Python from outside Azure. Ultimately I would like to write an API (FastAPI) allowing multiple users to connect to the database.
I ran a simple test from a Jupyter notebook, using SQLAlchemy as my ORM and specifying the pem certificate as a connection argument:
import pandas as pd
from sqlalchemy import create_engine
cnx = create_engine('mysql://XXX', connect_args={"ssl": {"ssl_ca": "mycertificate.pem"}})
I then tried reading data from a specific table (e.g. mytable):
df = pd.read_sql('SELECT * FROM mytable', cnx)
Alas I ran into the following error:
'Client with IP address 'XX.XX.XXX.XXX' is not allowed to connect to
this MySQL server'.
According to my colleagues, a way to fix this issue would be to whitelist my IP address.
While this may be an option for a couple of users with static IP addresses I am not sure whether it is a valid solution in the long run.
Is there a better way to access an Azure Database for MySQL Single Server from outside Azure?
As mentioned in comments, you need to whitelist the IP address ranges(s) in the Azure portal for your MySQL database resource. This is a well accepted and secure approach.
I connecting to database on Azure using authentication ActiveDirectoryPassword.
ss
cnxn = pyodbc.connect('DRIVER='+driver+';SERVER='+host+';UID='+user+';PWD='+password+';Authentication=ActiveDirectoryPassword')
It is working. The issue is that using this connection string I do not specify the DB. It just connecting me to master. How can I switch to DB I need. I have tried different connection strings (with database specified) but only this one works with ActiveDirectiryPassword.
You could try the below :
pyodbc.connect('DRIVER='+driver+';SERVER='+host+';DATABASE='+database+';UID='+user+';PWD='+password+';Authentication=ActiveDirectoryPassword')
Okay so I have an azure cosmos subscription, where I have created a Mongo DB resource, Now when I am using python SDK to connect it, now it's given when 104, error, connection reset by peer.
Now I am not sure what's the issue,
I am using endpoint with SSL True and Primary Key.
code
endpoint = "http://XXX.mongo.cosmos.azure.com:10255/?ssl=true"
key = 'xxxxxxxxxxxxxxxx'
# <create_cosmos_client>
client = CosmosClient(endpoint, key)
When choosing the MongoDB API, you must use a native MongoDB SDK (in your case, pymongo); the wire protocol is MongoDB, and operations are performed via the same protocol as MongoDB.
Your code is attempting to use the Cosmos DB SDK, which is specific to, and will only work with, the Core (SQL) API.
If you look in the portal blade for your MongoDB-API instance, you'll see examples under Quick Start tab, which each use a MongoDB SDK in its examples (or the mongo shell). Same thing with the Connection Strings tab, showing native MongoDB connection strings (as well as the separate parts of the connection string).
I'd like to develop a python project in client - server style.
My Python code will be deployed on a client machine and the MySQL database on the server, so all connections need to flow through a network connection.
I know how to use Python with MySQL with basics such as creating tables and queries etc, however I wanted to learn how to connect my python project to the database over a network protocol.
I searched on Google or YouTube, but only found resources that use a local database connection.
Where can I find Python resources (article, blog, sample, tutorial or video) that explains how to connect to MySQL via the network?
You have server with MySQL database running on it and a python client-application, which needs data from the database?
I think usually client software never connects directly to the database (for security reasons). Instead the client communicates with a server-application (maybe over a REST-API).
How i meant it
This is seems to be a good guide.
https://pynative.com/python-mysql-tutorial/
I tried PyMySql quickly. It works. Only problem is it is in Beta. https://pymysql.readthedocs.io/en/latest/user/examples.html
So I suggest you try the MySQL Connector Python will be slightly painful to setup initially but it comes from the vendor. so you can trust it a bit better.
or alternatively https://pypi.org/project/mysqlclient/
import pymysql.cursors
import pymysql
try:
conn = pymysql.connect(host='10.1.1.1 (remote server ip or hostname)',
user='db_user_name',
password='db_password',
db='name_of_db',
cursorclass=pymysql.cursors.DictCursor)
sql = "select * from users"
with conn.cursor() as cursor:
# Read a single record
sql = "select * from users"
cursor.execute(sql)
result = cursor.fetchall()
print(result)
except Exception as inst:
print(str(inst))
finally:
conn.close()
I am attempting to connected to a Google Cloud SQL instance in python and have gone through google's tutorial: https://cloud.google.com/appengine/docs/python/cloud-sql/
I am essentially cloning google's tutorial code and for some reason this line isn't working right for me:
if (os.getenv('SERVER_SOFTWARE') and
os.getenv('SERVER_SOFTWARE').startswith('Google App Engine/')):
This if statement is not being entered and I'm not sure why - it is then defaulting to accessing a local database based on the else statement. How is the os server_software environment set? I'm new to all of this but basically because that is not getting set, I am not able to access my google cloud sql instance. How do I make sure this if statement is entered?
SERVER_SOFTWARE is an environment variable that is automatically set by GAE. It could either be something like Google App Engine/x.x.xx when deployed or Development/x.x when running locally.
Basically the section of the code you're referring to checks whether your app is deployed and is running on GAE servers and if so - it will connect to a Google Cloud SQL instance, otherwise, if your app is running locally, it will attempt to connect to a local mysql instance.
It's done that way because you wouldn't normally want to mess with your production (deployed) data while developing & testing locally as many thing could go wrong.
Since you're stating that the if statement is not being entered - it's safe to assume that you are trying to run the program locally but are expecting it to connect to a Google Cloud SQL instance, for that, the next few lines in the example you provided are explaining how to do it:
db = MySQLdb.connect(host='127.0.0.1', port=3306, db='guestbook', user='root', charset='utf8')
# Alternatively, connect to a Google Cloud SQL instance using:
# db = MySQLdb.connect(host='ip-address-of-google-cloud-sql-instance', port=3306, user='root', charset='utf8')
so what you need to do is comment out the first line (the one it attempts to connect to a localhost mysql server) and uncomment the one where it connects to the Google Cloud SQL instance (note that you will have to update several parameters that reflect the configuration that you have, i.e. the host parameter and possible others).