So, I'm struggling with mysql
I tried to use mysql.connector, though it turned out that it doesn't really want to cooperate when I'm connecting via sshtunnel
So I transfered to pymysql, and here's the most basic code I was able to write:
import pymysql
from sshtunnel import SSHTunnelForwarder
with SSHTunnelForwarder(('192.168.0.x', 22), ssh_username='pi', ssh_password='*********', remote_bind_address=('localhost', 3306)) as tunnel:
tunnel.start()
mydb = pymysql.connect(host="localhost",
user='Mashu',
passwd='******',
port=tunnel.local_bind_port,
db='Special_Channels')
print(mydb)
query = "SELECT * FROM Daily"
cur = mydb.cursor()
data = cur.execute(query)
print(data)
Though on
cur = mydb.cursor()
it raises an error:
ConnectionAbortedError: [WinError 10053] An established connection was aborted by the software in your host machine
Also on a higher level it is:
pymysql.err.OperationalError: (2013, 'Lost connection to MySQL server during query ([WinError 10053] An established connection was aborted by the software in your host machine)')
I'm sure the database and table existst, and that this mysql account is accessible, as I have it opened and made changes to it in other software (DataGrip if anyone wonders)
I suggest mariadb instead of pymysql for your case
pip install mariadb
Solution is pretty easy.
it should all be inside with statement as tunnel connection (therefore database access) is closed outside statement, so it should look like this:
import pymysql
from sshtunnel import SSHTunnelForwarder
with SSHTunnelForwarder(('192.168.0.x', 22), ssh_username='pi', ssh_password='*********', remote_bind_address=('localhost', 3306)) as tunnel:
tunnel.start()
mydb = pymysql.connect(host="localhost",
user='Mashu',
passwd='******',
port=tunnel.local_bind_port,
db='Special_Channels')
print(mydb)
query = "SELECT * FROM Daily"
cur = mydb.cursor()
data = cur.execute(query)
print(data)
I'm downloading the tables from my SQL server on PythonAnywhere.com using a ssh tunnel following their description. Using the following code everything works fine in terms of downloading the tables, but the code then hangs at tunnel.close(). Any suggestions on how to stop it from hanging?
from __future__ import print_function
from mysql.connector import connect as sql_connect
import sshtunnel
from sshtunnel import SSHTunnelForwarder
from copy import deepcopy
import cPickle as pickle
import os
import datetime
sshtunnel.SSH_TIMEOUT = 5.0
sshtunnel.TUNNEL_TIMEOUT = 5.0
remote_bind_address = ('{}.mysql.pythonanywhere-services.com'.format(SSH_USERNAME), 3306)
tunnel = SSHTunnelForwarder(('ssh.pythonanywhere.com'),
ssh_username=SSH_USERNAME, ssh_password=SSH_PASSWORD,
remote_bind_address=remote_bind_address)
tunnel.start()
connection = sql_connect(user=SSH_USERNAME, password=DATABASE_PASSWORD,
host='127.0.0.1', port=tunnel.local_bind_port,
database=DATABASE_NAME)
print("Connection successful!")
cursor = connection.cursor() # get the cursor
cursor.execute("USE {}".format(DATABASE_NAME)) # select the database
# fetch all tables
cursor.execute("SHOW TABLES")
tables = deepcopy(cursor.fetchall()) # return data from last query
for (table_name,) in tables:
if 'contribute' in table_name:
print(table_name)
# may hang
connection.close()
tunnel.close()
I want create proxy/socks service from my ssh linux server. I already created port forwarder with sshtunnel for mysql, example code:
from sshtunnel import SSHTunnelForwarder
from sqlalchemy import create_engine
from sqlalchemy.engine.url import URL
with SSHTunnelForwarder(('mydomain.com', 22), ssh_username='root', ssh_password='password', local_bind_address=('127.0.0.1', 3306), remote_bind_address=('127.0.0.1', 3306)) as server:
myDB = URL(drivername='mysql+pymysql', host='127.0.0.1', database='test_db', username='root', password='THawr_tapH3f', port=3306)
engine = create_engine(name_or_url=myDB)
connection = engine.connect()
connection.close()
I want to transform my ssh linux machine in proxy/socks4/5 service with python; how can I do this?
I connect to my db manually using these steps:
1>load a putty session with ip 1.1.1.1 and port 1111
2>login as: login1
3>login1#1.1.1.1's password: pwd1
4>[login1#G ~]$ ssh login1#2.2.2.2
5>[login1#l ~]$ MySQL -h 3.3.3.3 -u login2 -p'pwd2' -D mydb
Now I can can query anything successfully like select * from my_table.
I have a python selenium webdriver code from where I want to read my db. I am not able to achieve it because of ssh tunnelling and 3 IP's involved.
from sshtunnel import SSHTunnelForwarder
import MySQLdb
with SSHTunnelForwarder(
('host', 1111),
ssh_password="pwd1
ssh_username="login1",
remote_bind_address=('2.2.2.2', 1111)) as server:
con = None
con = MySQLdb.connect(user='login2',passwd='pwd2',db='mydb',host=3.3.3.3,port=3306)
cur = con.cursor()
I am getting this error:
BaseSSHTunnelForwarderError: Could not resolve IP address for %s, aborting!
When instantiating SSHTunnelForwarder:
The param host is your remote ssh 2.2.2.2.
Argument for remote_bind_address will be for the tunnel, the port is the one you want to tunnel, so your mysql port. ('3.3.3.3', 3306)
Then when connecting to mysql, you want it to pass though the tunnel to access the mysql instance. The tunnel port is server.local_bind_port.
from sshtunnel import SSHTunnelForwarder
import MySQLdb
with SSHTunnelForwarder(
('2.2.2.2', 1111),
ssh_password="pwd1"
ssh_username="login1",
remote_bind_address=('3.3.3.3', 3306)) as server:
con = MySQLdb.connect(
user='login2',passwd='pwd2',
db='mydb',host=1.1.1.1,
port=server.local_bind_port)
I'm using MySqldb with Python 2.7 to allow Python to make connections to another MySQL server
import MySQLdb
db = MySQLdb.connect(host="sql.domain.com",
user="dev",
passwd="*******",
db="appdb")
Instead of connecting normally like this, how can the connection be made through a SSH tunnel using SSH key pairs?
The SSH tunnel should ideally be opened by Python. The SSH tunnel host and the MySQL server are the same machine.
Only this worked for me
import pymysql
import paramiko
import pandas as pd
from paramiko import SSHClient
from sshtunnel import SSHTunnelForwarder
from os.path import expanduser
home = expanduser('~')
mypkey = paramiko.RSAKey.from_private_key_file(home + pkeyfilepath)
# if you want to use ssh password use - ssh_password='your ssh password', bellow
sql_hostname = 'sql_hostname'
sql_username = 'sql_username'
sql_password = 'sql_password'
sql_main_database = 'db_name'
sql_port = 3306
ssh_host = 'ssh_hostname'
ssh_user = 'ssh_username'
ssh_port = 22
sql_ip = '1.1.1.1.1'
with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
conn = pymysql.connect(host='127.0.0.1', user=sql_username,
passwd=sql_password, db=sql_main_database,
port=tunnel.local_bind_port)
query = '''SELECT VERSION();'''
data = pd.read_sql_query(query, conn)
conn.close()
I'm guessing you'll need port forwarding. I recommend sshtunnel.SSHTunnelForwarder
import mysql.connector
import sshtunnel
with sshtunnel.SSHTunnelForwarder(
(_host, _ssh_port),
ssh_username=_username,
ssh_password=_password,
remote_bind_address=(_remote_bind_address, _remote_mysql_port),
local_bind_address=(_local_bind_address, _local_mysql_port)
) as tunnel:
connection = mysql.connector.connect(
user=_db_user,
password=_db_password,
host=_local_bind_address,
database=_db_name,
port=_local_mysql_port)
...
from sshtunnel import SSHTunnelForwarder
import pymysql
import pandas as pd
tunnel = SSHTunnelForwarder(('SSH_HOST', 22), ssh_password=SSH_PASS, ssh_username=SSH_UNAME,
remote_bind_address=(DB_HOST, 3306))
tunnel.start()
conn = pymysql.connect(host='127.0.0.1', user=DB_UNAME, passwd=DB_PASS, port=tunnel.local_bind_port)
data = pd.read_sql_query("SHOW DATABASES;", conn)
credits to https://www.reddit.com/r/learnpython/comments/53wph1/connecting_to_a_mysql_database_in_a_python_script/
If your private key file is encrypted, this is what worked for me:
mypkey = paramiko.RSAKey.from_private_key_file(<<file location>>, password='password')
sql_hostname = 'sql_hostname'
sql_username = 'sql_username'
sql_password = 'sql_password'
sql_main_database = 'sql_main_database'
sql_port = 3306
ssh_host = 'ssh_host'
ssh_user = 'ssh_user'
ssh_port = 22
with SSHTunnelForwarder(
(ssh_host, ssh_port),
ssh_username=ssh_user,
ssh_pkey=mypkey,
ssh_password='ssh_password',
remote_bind_address=(sql_hostname, sql_port)) as tunnel:
conn = pymysql.connect(host='localhost', user=sql_username,
passwd=sql_password, db=sql_main_database,
port=tunnel.local_bind_port)
query = '''SELECT VERSION();'''
data = pd.read_sql_query(query, conn)
print(data)
conn.close()
You may only write the path to the private key file: ssh_pkey='/home/userName/.ssh/id_ed25519' (documentation is here: https://sshtunnel.readthedocs.io/en/latest/).
If you use mysql.connector from Oracle you must use a construction
cnx = mysql.connector.MySQLConnection(...
Important: a construction
cnx = mysql.connector.connect(...
does not work via an SSh! It is a bug.
(The documentation is here: https://dev.mysql.com/doc/connector-python/en/connector-python-connectargs.html).
Also, your SQL statement must be ideal. In case of an error on SQL server side, you do not receive an error message from SQL-server.
import sshtunnel
import numpy as np
with sshtunnel.SSHTunnelForwarder(ssh_address_or_host='ssh_host',
ssh_username="ssh_username",
ssh_pkey='/home/userName/.ssh/id_ed25519',
remote_bind_address=('localhost', 3306),
) as tunnel:
cnx = mysql.connector.MySQLConnection(user='sql_username',
password='sql_password',
host='127.0.0.1',
database='db_name',
port=tunnel.local_bind_port)
cursor = cnx.cursor()
cursor.execute('SELECT * FROM db_name.tableName;')
arr = np.array(cursor.fetchall())
cursor.close()
cnx.close()
This works for me:
import mysql.connector
import sshtunnel
with sshtunnel.SSHTunnelForwarder(
('ip-of-ssh-server', 'port-in-number-format'),
ssh_username = 'ssh-username',
ssh_password = 'ssh-password',
remote_bind_address = ('127.0.0.1', 3306)
) as tunnel:
connection = mysql.connector.connect(
user = 'database-username',
password = 'database-password',
host = '127.0.0.1',
port = tunnel.local_bind_port,
database = 'databasename',
)
mycursor = connection.cursor()
query = "SELECT * FROM datos"
mycursor.execute(query)
Someone said this in another comment. If you use the python mysql.connector from Oracle then you must use a construction cnx = mysql.connector.MySQLConnection(....
Important: a construction cnx = mysql.connector.connect(... does not work via an SSH! This bug cost me a whole day trying to understand why connections were being dropped by the remote server:
with sshtunnel.SSHTunnelForwarder(
(ssh_host,ssh_port),
ssh_username=ssh_username,
ssh_pkey=ssh_pkey,
remote_bind_address=(sql_host, sql_port)) as tunnel:
connection = mysql.connector.MySQLConnection(
host='127.0.0.1',
port=tunnel.local_bind_port,
user=sql_username,
password=sql_password)
query = 'select version();'
data = pd.read_sql_query(query, connection)
print(data)
connection.close()
If you are using python, and all the username, password, host and port are correct then there is just one thing left, that is using the argument (use_pure=True). This argument uses python to parse the details and password. You can see the doc of mysql.connector.connect() arguments.
with sshtunnel.SSHTunnelForwarder(
(ssh_host,ssh_port),
ssh_username=ssh_username,
ssh_pkey=ssh_pkey,
remote_bind_address=(sql_host, sql_port)) as tunnel:
connection = mysql.connector.MySQLConnection(
host='127.0.0.1',
port=tunnel.local_bind_port,
user=sql_username,
password=sql_password,
use_pure='True')
query = 'select version();'
data = pd.read_sql_query(query, connection)
print(data)
connection. Close()
Paramiko is the best python module to do ssh tunneling. Check out the code here:
https://github.com/paramiko/paramiko/blob/master/demos/forward.py
As said in comments this one works perfect.
SSH Tunnel for Python MySQLdb connection
Best practice is to parameterize the connection variables.
Here is how I have implemented. Works like charm!
import mysql.connector
import sshtunnel
import pandas as pd
import configparser
config = configparser.ConfigParser()
config.read('c:/work/tmf/data_model/tools/config.ini')
ssh_host = config['db_qa01']['SSH_HOST']
ssh_port = int(config['db_qa01']['SSH_PORT'])
ssh_username = config['db_qa01']['SSH_USER']
ssh_pkey = config['db_qa01']['SSH_PKEY']
sql_host = config['db_qa01']['HOST']
sql_port = int(config['db_qa01']['PORT'])
sql_username = config['db_qa01']['USER']
sql_password = config['db_qa01']['PASSWORD']
with sshtunnel.SSHTunnelForwarder(
(ssh_host,ssh_port),
ssh_username=ssh_username,
ssh_pkey=ssh_pkey,
remote_bind_address=(sql_host, sql_port)) as tunnel:
connection = mysql.connector.connect(
host='127.0.0.1',
port=tunnel.local_bind_port,
user=sql_username,
password=sql_password)
query = 'select version();'
data = pd.read_sql_query(query, connection)
print(data)
connection.close()