Develop Reference

Accessing an Azure Database for MySQL Single Server from outside Azure

Moving this question from DevOps Stack Exchange where it got only 5 views in 2 days:
I would like to query an Azure Database for MySQL Single Server.
I normally interact with this database using a universal database tool (dBeaver) installed onto an Azure VM. Now I would like to interact with this database using Python from outside Azure. Ultimately I would like to write an API (FastAPI) allowing multiple users to connect to the database.
I ran a simple test from a Jupyter notebook, using SQLAlchemy as my ORM and specifying the pem certificate as a connection argument:
import pandas as pd
from sqlalchemy import create_engine
cnx = create_engine('mysql://XXX', connect_args={"ssl": {"ssl_ca": "mycertificate.pem"}})
I then tried reading data from a specific table (e.g. mytable):
df = pd.read_sql('SELECT * FROM mytable', cnx)
Alas I ran into the following error:
'Client with IP address 'XX.XX.XXX.XXX' is not allowed to connect to
this MySQL server'.
According to my colleagues, a way to fix this issue would be to whitelist my IP address.
While this may be an option for a couple of users with static IP addresses I am not sure whether it is a valid solution in the long run.
Is there a better way to access an Azure Database for MySQL Single Server from outside Azure?

As mentioned in comments, you need to whitelist the IP address ranges(s) in the Azure portal for your MySQL database resource. This is a well accepted and secure approach.

sshtunnel forwarding and mysql connection on google cloud function

I am trying to connect to a MySQL server through ssh tunnel on one of my google cloud functions. This works fine in my home environment. I assume it is some port issue on cloud function.
Edit: For clarification the MySQL server sits on a Namecheap shared hosting web server. Not Google Cloud SQL
Every time I run this I timeout with "unknown error". The tunnel appears to be successful. I am however unable to get the mysql connection to work.
import base64
import sshtunnel
import mysql.connector
def testing(event, context):
"""
Testing function
"""
with sshtunnel.SSHTunnelForwarder(
("server address", port),
ssh_username="user",
ssh_password="password",
remote_bind_address=("127.0.0.1",3306),
) as server:
print(server.local_bind_port)
with mysql.connector.connect(
user="user",
password="password",
host="localhost",
database="database",
port=server.local_bind_port
) as connection:
print(connection)

There's too many steps to list, but I'm wondering if the "connector" setup plays a difference even for SSH. Maybe you have to create a connector as shown here (notice how the instructions in "Private IP" tab are different than on your local computer). Then, configure Cloud Functions to use that connector. Make sure you also use the right port.
A Serverless VPC Access connector handles communication to your VPC
network. To connect directly with private IP, you need to:
Make sure that the Cloud SQL instance created above has a private IP
address. If you need to add one, see the Configuring private IP page
for instructions.
Create a Serverless VPC Access connector in the same
VPC network as your Cloud SQL instance. Unless you're using Shared
VPC, a connector must be in the same project and region as the
resource that uses it, but the connector can send traffic to resources
in different regions. Serverless VPC Access supports communication to VPC networks connected via Cloud VPN and VPC Network Peering. Serverless VPC Access does not support legacy networks.
Configure Cloud Functions to use the connector.
Connect using your
instance's private IP and port 3306.
Keep in mind, this "unknown" error could also very well be due to the Cloud SQL Admin API not being enabled here. As a matter of fact, make sure you follow that entire page as it's a broad question.
Let us know what worked for this type of error.

Config for using psycopg2 to access to Cloud SQL instance

I am using a PostgreSQL in Cloud SQL along with psycopg2 library to connect to database from my python code. The current one that I have is associated with a VPC network in which my google compute engines also in that VPC. So, in this case, my config to connect to this cloud sql instance can use private ip and my config look like this:
config['db-cloudsql'] = {
"host" : "10.x.x.x" # cloud sql private ip address
"user" : "postgres"
"password" :"xxxxx"
"database" : "postgres"
}
But now I have another vm instance from another VPC network that need to access to this cloud sql instance. I know that I can access to the cloud sql instance using public ip. (by adding my vm to authorised network) But this vm need to access to the cloud sql instance very often. So I am not sure that the cost from accessing with public ip will be higher compared with accessing with private ip (cannot find any related document about this). I tried peering 2 vpc networks for accessing with private ip but just found that I cannot use this method to connect to sql with private ip from this document
I have found from the document that I can use instance connection name as a host for my config so it should be something like:
config['db-cloudsql'] = {
"host" : "project-name:asia-southeast1:mydbname" #instance connection name
"user" : "postgres"
"password" :"xxxxx"
"database" : "postgres"
}
I haven't try this method yet, it might not working but if it somehow works, how will it differ from using public ip addess in term of cost?
Thank you

It seems like your question boils down into two parts:
Are there any alternatives to public IP or private IP?
No. You have to use one or the other to connect to your Cloud SQL instance. Private IP allows access from a VPC, Public IP is used pretty much everywhere else.
Cost of public IP vs private IP
You can find a breakdown of the costs here. In short, there is not really any extra charges with a Public IP. You do have to pay $.01 per hour while the instance is idle (to reserve the public IP address), and just like private IPs, you are responsible for costs of the Network Egress between regions.
I can use instance connection name as a host for my config
This is incorrect. If you are using the Cloud SQL Proxy to connect, it can create a Unix domain socket (at /cloudsql/INSTANCE_CONNECTION_NAME) that can be used to connect to your instance. However the proxy only authenticates your connection - it still needs a valid connection path (Public vs Private).

As you can see in this PostgreSQL pricing documentation there is no difference in the pricing as you use the public IP or the connection name on the Cloud SQL side. If you want to keep the cost as low as possible you can try to keep your Compute Engine instances in the same region, or at least in the same continent as your Cloud SQL instance.

Can`t connect MySQL database in phpMyAdmin by python

I need to use python to connect database in phpMyAdmin.
import MySQLdb
db = MySQLdb.connect(host="10.0.0.140",port=80,user="root",passwd="password")
cursor=db.cursor()
cursor.execute("SHOW DATABASES")
results=cursor.fetchall()
for result in results:
print row
and I get this error
2013,"Lost connection to MySQL server at'waiting for initial communication packet',system error:0"
I can access the database in chrome, so I don`t think it is a problem of remote access.
=======================update=======================
The real reason is that I am in a limited net segment.(maybe)
There is something wrong with SQL connection out of my code.
If changed code like this:(port is not needed)
MySQLdb.connect(host="10.0.0.140",user="root",passwd="password")
People in public net segment can connect the phpMyAdmin but I can`t.
I can not change my net segment so I can`t confirm it is the real reason.
But it is the only difference between me and others.

Check the connection, are you sure you run your mysql at port 80?
As I wrote in the comments, I doubt it runs port 80, this is the url to your phpMyAdmin and not the core mysql database server. Normally the web and database run on different machines, different IP addresses etc. Is this a hosted environment? Or do you run this in your local machine.
If it is your local machine then changing to port 3306 should work.
If it is a hosted environment by a hosting partner then you need to check the JDBC url from them. As you wrote it is XXXX hosted partner. Then check your XXX and it will provide you a XXX jdbc url for your mysql, and be sure to grant access to the user so you can logon to the mysql from your workstation remotly. The mysql url/host could be something like this NNNN-aaaa-bbbb-cccc-dddd.xxx.domain.xx
Also did you try the PyMYSQL
import pymysql
db = pymysql.connect(host='10.0.0.140',user='root',passwd='password')
cursor = db.cursor()
query = ("SHOW DATABASES")
cursor.execute(query)
for r in cursor:
print r

This is not the way you should do it. phpMyAdmin serves HTML to the user and should be used in a Web browser. You should connect directly to the MySQL host which is usually listening on port :3306. Also, you should keep in mind that, in production, MySQL servers are ordinarily not listening on public interfaces.

OpenShift mysql connection issues in python

I previously wrote my app using local development servers, and now that I have moved it onto an openshift small gear almost all works except for mysql connections.
In my code I have the line:
self.db = MySQLdb.connect(host, username, password, dbname)
When I review the openshift error log, the following error is reported:
_mysql_exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/var/lib/mysql/mysql.sock' (2)")
I think that python is trying to connect using a UNIX socket as opposed to an INET one, but I'm not sure how to change this behavior. Any help is much appreciated.

Not specific to MySQLdb: if you use localhost as hostname, a MySQL client using the MySQL C libraries will try to connect using UNIX socket (or named pipe on Windows). There are 2 ways around this, but you'll need to grant extra permissions to make it work for both:
Use IP address 127.0.0.1
Use IP address 127.0.0.1 instead of the localhost hostname. This will make MySQL client connect using TCP/IP.
Use option files
The other way is to force the protocol using using option files. For example, in your ~/.my.cnf (or any file you want), add the following:
[python]
protocol=tcp
Now use the connection arguments to read the option file and group:
import MySQLdb
cnx = MySQLdb.connect(host='localhost', user='scott', passwd='tiger',
read_default_file='~/.my.cnf',
read_default_group='python')
The group name does not need to be python, but it is good not to use mysql or client as it might interfere with other MySQL tools (unless you want that of course).
For setting up permissions, you'll need to use the IP address of localhost, something like:
mysql> GRANT SELECT TO yourdb.* TO 'scott'#'127.0.0.1' IDENTIFIED BY ...;
(Site note: MySQL database drivers such as MySQL Connector/Python do not consider localhost to be special and connect through TCP/IP right away and you have to explicitly use the unix_socket.)

As I later discovered, while the database server runs on localhost, it runs on a very specific localhost bind address. In my case it was an address that I would never have though to try if I hadn't noticed how phpmyadmin was connecting.