I am trying to access MySQL on GCE VM instance through Google Colaboratory though, unfortunately, it does not work properly but does work in the local VSCode environment.
While executing TestExec.py, it shows SSH Connected so it seems that ssh connection is successfully done though, however, it seems to be stuck on MySQL connection.
Output on Google Colab:
Please help with the solutions/tips?
sqlList = []
sqlList.append("select * from table name;")
HOST = 'ComputeEngine PublicIP'
PORT = 22
USER = 'username'
DBUSER = 'username for db'
KEY_FILE = 'private key file path'
DBNAME = 'dbname'
DBPORT = 3306
SSH_BASTION_ADDRESS = HOST
SSH_PORT = PORT
SSH_USER = USER
SSH_PKEY_PATH = KEY_FILE
MYSQL_HOST = HOST
MYSQL_PORT = 3306
MYSQL_USER = DBUSER
MYSQL_PASS = 'MySQL Login PW'
MYSQL_DB = DBNAME
with SSHTunnelForwarder(
(SSH_BASTION_ADDRESS, SSH_PORT),
ssh_pkey=SSH_PKEY_PATH,
ssh_username=SSH_USER,
# ssh_password=PASSPHRASE,
remote_bind_address=('localhost', MYSQL_PORT),
local_bind_address=('localhost', MYSQL_PORT)
) as ssh:
print("SSH Connected")
print(ssh.local_bind_port)
try:
connection = mysql.connector.connect(
host='localhost',
port = ssh.local_bind_port,
user=MYSQL_USER,
passwd=MYSQL_PASS,
db=MYSQL_DB,
charset='utf8'
)
print(connection.is_connected())
print("DB Connected")
cur = connection.cursor()
sql = "use dbname"
cur.execute(sql)
# rows = cur.fetchall()
# for row in rows:
# print(row)
for i in range(len(sqlList)):
print(sqlList[i])
sql = str(sqlList[i])
# sql = 'create table test (id int, content varchar(32))'
cur.execute(sql)
rows = cur.fetchall()
for row in rows:
print(row)
except mysql.connector.Error as err:
print("Something went wrong: {}".format(err))
connection.rollback()
raise err
finally:
cur.close()
connection.commit()
connection.close()
It seemed the error occurred because of the connection. I have changed the method of connection.
import pymysql.cursors
connection = pymysql.connect(
host='localhost',
user='user',
password='password!',
db='dbname',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor
)
I am having a weird issue with my python script. My script has to connect to MySQL DB. This is the code:
try:
conn = MySQLdb.connect( user='root', host = 'localhost')
cursor = conn.cursor()
databases = cursor.fetchall()
cursor.close()
except Exception as e:
print e
when I run this script I have and error like:
(1045, "Access denied for user 'root'#'localhost' (using password: NO")
in the other hand, I can connect to MySQL just by entering MySQL (without password).
Why am I having this error with my python script when there is no password to root user?
Provide empty password
try this
conn = MySQLdb.connect( user='root', host = 'localhost', passwd='')
This should be the syntax. You should have the MySql connector for Python
import mysql.connector
cnx = mysql.connector.connect(user='root', password='',
host='127.0.0.1',
database='database_name')
cnx.close()
try this (inside the bloc try except)
import mysql.connector
conn = mysql.connector.connect(user='root', password='',host='localhost',
database='your_database_name')
conn.close()
I have a postgres username that contains a hyphen (example: 'user-1').
The following code works with python 2.6.3, but not with Python 2.7.10 :: Anaconda 1.9.2 (64-bit). I tried both single and double quotes around the username.
Any ideas?
username = 'user-1'
try:
con = psycopg2.connect(
database = database,
user = username,
password = password,
host = hostname
)
except:
print "I am unable to connect to the database."
The code errors with:
"/usr/local/apps/python/anaconda-current/lib/python2.7/site-packages/psycopg2/init.py", line 164, in connect conn = _connect(dsn, connection_factory=connection_factory, async=async) psycopg2.OperationalError: FATAL: password authentication failed for user "user-1
I would like to connect to a postgresql database using python from a different server.
I triyed this :
conn_string = "host=192.168.1.1 dbname='"+db7+"' user='user' password='"+pw7+"'"
conn = psycopg2.connect(conn_string)
cursor = conn.cursor()
but I get the error:
conn = psycopg2.connect(conn_string)
File "/usr/lib/python2.7/dist-packages/psycopg2/__init__.py", line 179, in connect
connection_factory=connection_factory, async=async)
psycopg2.OperationalError: FATAL: database "database" does not exist
Remove unnecessary quotes in the syntax.
Follow this structure.
conn = psycopg2.connect(host = "localhost",database="ur_database_name", user="db_user", password="your_password")
Example.
conn = psycopg2.connect(host = "localhost",database="studentesdb", user="postgres", password="admin")
I'm using MySqldb with Python 2.7 to allow Python to make connections to another MySQL server
import MySQLdb
db = MySQLdb.connect(host="sql.domain.com",
user="dev",
passwd="*******",
db="appdb")
Instead of connecting normally like this, how can the connection be made through a SSH tunnel using SSH key pairs?
The SSH tunnel should ideally be opened by Python. The SSH tunnel host and the MySQL server are the same machine.
Only this worked for me
import pymysql
import paramiko
import pandas as pd
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
from os.path import expanduser
home = expanduser('~')
mypkey = paramiko.RSAKey.from_private_key_file(home + pkeyfilepath)
# if you want to use ssh password use - ssh_password='your ssh password', bellow
sql_hostname = 'sql_hostname'
sql_username = 'sql_username'
sql_password = 'sql_password'
sql_main_database = 'db_name'
sql_port = 3306
ssh_host = 'ssh_hostname'
ssh_user = 'ssh_username'
ssh_port = 22
sql_ip = '1.1.1.1.1'
with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
conn = pymysql.connect(host='127.0.0.1', user=sql_username,
passwd=sql_password, db=sql_main_database,
port=tunnel.local_bind_port)
query = '''SELECT VERSION();'''
data = pd.read_sql_query(query, conn)
conn.close()
I'm guessing you'll need port forwarding. I recommend sshtunnel.SSHTunnelForwarder
import mysql.connector
import sshtunnel
with sshtunnel.SSHTunnelForwarder(
(_host, _ssh_port),
ssh_username=_username,
ssh_password=_password,
remote_bind_address=(_remote_bind_address, _remote_mysql_port),
local_bind_address=(_local_bind_address, _local_mysql_port)
) as tunnel:
connection = mysql.connector.connect(
user=_db_user,
password=_db_password,
host=_local_bind_address,
database=_db_name,
port=_local_mysql_port)
...
from sshtunnel import SSHTunnelForwarder
import pymysql
import pandas as pd
tunnel = SSHTunnelForwarder(('SSH_HOST', 22), ssh_password=SSH_PASS, ssh_username=SSH_UNAME,
remote_bind_address=(DB_HOST, 3306))
tunnel.start()
conn = pymysql.connect(host='127.0.0.1', user=DB_UNAME, passwd=DB_PASS, port=tunnel.local_bind_port)
data = pd.read_sql_query("SHOW DATABASES;", conn)
credits to https://www.reddit.com/r/learnpython/comments/53wph1/connecting_to_a_mysql_database_in_a_python_script/
If your private key file is encrypted, this is what worked for me:
mypkey = paramiko.RSAKey.from_private_key_file(<<file location>>, password='password')
sql_hostname = 'sql_hostname'
sql_username = 'sql_username'
sql_password = 'sql_password'
sql_main_database = 'sql_main_database'
sql_port = 3306
ssh_host = 'ssh_host'
ssh_user = 'ssh_user'
ssh_port = 22
with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
ssh_password='ssh_password',
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
conn = pymysql.connect(host='localhost', user=sql_username,
passwd=sql_password, db=sql_main_database,
port=tunnel.local_bind_port)
query = '''SELECT VERSION();'''
data = pd.read_sql_query(query, conn)
print(data)
conn.close()
You may only write the path to the private key file: ssh_pkey='/home/userName/.ssh/id_ed25519' (documentation is here: https://sshtunnel.readthedocs.io/en/latest/).
If you use mysql.connector from Oracle you must use a construction
cnx = mysql.connector.MySQLConnection(...
Important: a construction
cnx = mysql.connector.connect(...
does not work via an SSh! It is a bug.
(The documentation is here: https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html).
Also, your SQL statement must be ideal. In case of an error on SQL server side, you do not receive an error message from SQL-server.
import sshtunnel
import numpy as np
with sshtunnel.SSHTunnelForwarder(ssh_address_or_host='ssh_host',
ssh_username="ssh_username",
ssh_pkey='/home/userName/.ssh/id_ed25519',
remote_bind_address=('localhost', 3306),
) as tunnel:
cnx = mysql.connector.MySQLConnection(user='sql_username',
password='sql_password',
host='127.0.0.1',
database='db_name',
port=tunnel.local_bind_port)
cursor = cnx.cursor()
cursor.execute('SELECT * FROM db_name.tableName;')
arr = np.array(cursor.fetchall())
cursor.close()
cnx.close()
This works for me:
import mysql.connector
import sshtunnel
with sshtunnel.SSHTunnelForwarder(
('ip-of-ssh-server', 'port-in-number-format'),
ssh_username = 'ssh-username',
ssh_password = 'ssh-password',
remote_bind_address = ('127.0.0.1', 3306)
) as tunnel:
connection = mysql.connector.connect(
user = 'database-username',
password = 'database-password',
host = '127.0.0.1',
port = tunnel.local_bind_port,
database = 'databasename',
)
mycursor = connection.cursor()
query = "SELECT * FROM datos"
mycursor.execute(query)
Someone said this in another comment. If you use the python mysql.connector from Oracle then you must use a construction cnx = mysql.connector.MySQLConnection(....
Important: a construction cnx = mysql.connector.connect(... does not work via an SSH! This bug cost me a whole day trying to understand why connections were being dropped by the remote server:
with sshtunnel.SSHTunnelForwarder(
(ssh_host,ssh_port),
ssh_username=ssh_username,
ssh_pkey=ssh_pkey,
remote_bind_address=(sql_host, sql_port)) as tunnel:
connection = mysql.connector.MySQLConnection(
host='127.0.0.1',
port=tunnel.local_bind_port,
user=sql_username,
password=sql_password)
query = 'select version();'
data = pd.read_sql_query(query, connection)
print(data)
connection.close()
If you are using python, and all the username, password, host and port are correct then there is just one thing left, that is using the argument (use_pure=True). This argument uses python to parse the details and password. You can see the doc of mysql.connector.connect() arguments.
with sshtunnel.SSHTunnelForwarder(
(ssh_host,ssh_port),
ssh_username=ssh_username,
ssh_pkey=ssh_pkey,
remote_bind_address=(sql_host, sql_port)) as tunnel:
connection = mysql.connector.MySQLConnection(
host='127.0.0.1',
port=tunnel.local_bind_port,
user=sql_username,
password=sql_password,
use_pure='True')
query = 'select version();'
data = pd.read_sql_query(query, connection)
print(data)
connection. Close()
Paramiko is the best python module to do ssh tunneling. Check out the code here:
https://github.com/paramiko/paramiko/blob/master/demos/forward.py
As said in comments this one works perfect.
SSH Tunnel for Python MySQLdb connection
Best practice is to parameterize the connection variables.
Here is how I have implemented. Works like charm!
import mysql.connector
import sshtunnel
import pandas as pd
import configparser
config = configparser.ConfigParser()
config.read('c:/work/tmf/data_model/tools/config.ini')
ssh_host = config['db_qa01']['SSH_HOST']
ssh_port = int(config['db_qa01']['SSH_PORT'])
ssh_username = config['db_qa01']['SSH_USER']
ssh_pkey = config['db_qa01']['SSH_PKEY']
sql_host = config['db_qa01']['HOST']
sql_port = int(config['db_qa01']['PORT'])
sql_username = config['db_qa01']['USER']
sql_password = config['db_qa01']['PASSWORD']
with sshtunnel.SSHTunnelForwarder(
(ssh_host,ssh_port),
ssh_username=ssh_username,
ssh_pkey=ssh_pkey,
remote_bind_address=(sql_host, sql_port)) as tunnel:
connection = mysql.connector.connect(
host='127.0.0.1',
port=tunnel.local_bind_port,
user=sql_username,
password=sql_password)
query = 'select version();'
data = pd.read_sql_query(query, connection)
print(data)
connection.close()