Develop Reference

Connection to remote mysql server through ssh using python mysqldb connector passwordless

Python 3.8
Mysql 8.0.23-0ubuntu0.20.04.1 for Linux on x86_64 ((Ubuntu))
Hi,
I want to connect to a distant mysql server using python's mysqldb connector and paramiko's sshtunnelforwarder.
I can connect to the database remotely without any problems by executing the following:
Connecting to database using mysql password authentication
server = new_ssh_server(config)
with server:
print('Connection', server.local_bind_address)
cnx = MySQLdb.connect(host = '127.0.0.1',
port = server.local_bind_port,
user = config['user'],
passwd = config['password'],
db = config['db'])
Queries work, I can read/write to database, no problem.
I would like to connect to database without supplying mysql password, by using mysql auth_socket authentication method, through ssh.
My attempts at this can be resumed by the following code:
Connecting to database using mysql auth_socket authentication
with server as tunnel:
print('Tunnel:', tunnel.local_bind_address)
cnx = MySQLdb.connect(host = 'localhost', user = 'hillbilly', password = '', db='tutut')#, unix_socket="/tmp/mysql.sock")
res = pd.read_sql('select * from users;', cnx)
print(res)
Which throws the following error:
File "connect_ssh_mysql_auth_socket.py", line 12, in <module>
cnx = MySQLdb.connect(host = 'localhost', user = 'hillbilly', password = '', db='rsotest2')#, unix_socket="/tmp/mysql.sock")
File "...../lib/python3.8/site-packages/MySQLdb/__init__.py", line 84, in Connect
return Connection(*args, **kwargs)
File "...../lib/python3.8/site-packages/MySQLdb/connections.py", line 179, in __init__
super(Connection, self).__init__(*args, **kwargs2)
MySQLdb._exceptions.OperationalError: (2002, "Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)")
I have and existing mysqld.sock on the distant server that I symlinked to /tmp/mysql.sock, but the error remains. I have also added the next line to /etc/mysql/mysql.conf.d/mysql.cnf:
socket=/run/mysqld/mysqld.sock
But I still get the same error when trying to connect remotely.
Specifying the unix_socket='/run/mysqld/mysqld.sock' to mysqldb connector (commented in Connecting to database using mysql auth_socket authentication) does not fix the issue.
I seem to misunderstand the use of mysql.sock, mysqld.sock.
I was not able to find nor create a mysql.sock socket.
Is what I am trying to do possible? I remember reading somewhere that unix sockets only work locally, does this mean it is not achievable?
Any help/explanation would be appreciated.
(EDIT AND CLOSING)
So this is not possible. Following this thread, auth_socket needs local access to the socket file (usually /tmp/mysql.sock) to run autentication tests, so not accessible through ssh tunneling.

Authentication to remote mysql server using auth_socket plugin is not possible through sshtunnel, as the plugin requires local access to the socket file. See this thread for more information.

Unable to connect to hive using python using impyla/dbapi.py

I am trying to connect to hive[with default derby db] using python:
from impala.dbapi import connect
conn = connect( host='localhost', port=10000)
cursor = conn.cursor()
cursor.execute('SELECT * FROM employee')
print cursor.description # prints the result set's schema
results = cursor.fetchall()
but I am getting error:
Traceback (most recent call last):
File "hivetest_b.py", line 2, in <module>
conn = connect( host='localhost', port=10000)
File "/home/ubuntu/.local/lib/python2.7/site-packages/impala/dbapi.py", line 147, in connect
auth_mechanism=auth_mechanism)
File "/home/ubuntu/.local/lib/python2.7/site-packages/impala/hiveserver2.py", line 758, in connect
transport.open()
File "/home/ubuntu/.local/lib/python2.7/site-packages/thrift/transport/TTransport.py", line 149, in open
return self.__trans.open()
File "/home/ubuntu/.local/lib/python2.7/site-packages/thrift/transport/TSocket.py", line 101, in open
message=message)
thrift.transport.TTransport.TTransportException: Could not connect to localhost:10000
entry in my /etc/hosts is:
127.0.0.1 localhost
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
ff02::3 ip6-allhosts
I am using default hive-site.xml and defult derby database for running my hive. When I run hive through shell it shows me that table:
hive> show databases;
OK
default
test
test_db
Time taken: 0.937 seconds, Fetched: 3 row(s)
hive> show tables;
OK
employee
Time taken: 0.054 seconds, Fetched: 1 row(s)
hive> describe employee;
OK
empname string
age int
gender string
income float
department string
dept string
# Partition Information
# col_name data_type comment
dept string
Time taken: 0.451 seconds, Fetched: 11 row(s)
I am not sure what exactly am I missing here. Any quick references/pointers would be appreciated.
Regards,
Bhupesh

You can check and validate the port with:
hive> set hive.server2.thrift.port;
And try 0.0.0.0 and 127.0.0.1 instead of localhost as your host for the connection.

Issue to connect into a mysql database with MySQLdb module (python)

I have a database in mysql and I want to connect to it. I am trying to use this module from Python called MySQLdb. I created an user (called abc) and password (abc) for this database (abc) that has one table and it is connecting ok (when I connect by mysql command line).
But when I run my python script there is an error in the connection.
My script is:
#!/usr/bin/python
import MySQLdb
# Open database connection
db = MySQLdb.connect("localhost","abc","abc","abc")
# prepare a cursor object using cursor() method
cursor = db.cursor()
# execute SQL query using execute() method.
cursor.execute("SELECT VERSION()")
# Fetch a single row using fetchone() method.
data = cursor.fetchone()
print "Database version : %s " % data
# disconnect from server
db.close()
My error is:
Traceback (most recent call last):
File "./test.py", line 8, in <module>
db = MySQLdb.connect("localhost","abc","abc","abc")
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/__init__.py", line 81, in Connect
return Connection(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/MySQLdb/connections.py", line 193, in __init__
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (1044, "Access denied for user 'abc'#'localhost' to database 'abc'")
What is wrong? My script or something in my mysql?
I changed my localhost to 127.0.0.1 (as suggested in another post, but did not solve my issue.
I also checked my permissions for this mysql user:
SHOW GRANTS;
+------------------------------------------------------------------------------+
| Grants for abc#localhost |
+------------------------------------------------------------------------------+
| GRANT USAGE ON *.* TO 'abc'#'localhost' |
| GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, DROP, INDEX, CREATE VIEW ON `abc`.* TO 'abc'#'localhost' |
+------------------------------------------------------------------------------+

You might need to restart the mysql daemon for the privileges to take affect. Or use the flush privileges command. https://dev.mysql.com/doc/refman/5.7/en/privilege-changes.html

MySQLPython is ignoring my my.cnf file. Where does it get its information?

When I try to use MySQLPython (via SQLAlchemy) I get the error
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.6/lib/python2.6/site-packages/MySQL_python-1.2.3c1-py2.6-macosx-10.6-x86_64.egg/MySQLdb/connections.py", line 188, in __init__
super(Connection, self).__init__(*args, **kwargs2)
sqlalchemy.exc.OperationalError: (OperationalError) (2002, "Can't connect to local MySQL server through socket '/opt/local/var/run/mysql5/mysqld.sock' (2)") None None
but no other mysql client on my machine sees it fine!
My my.cnf file states:
[client]
port = 3306
socket = /tmp/mysql/mysql.sock
[safe_mysqld]
socket = /tmp/mysql/mysql.sock
[mysqld_safe]
socket = /tmp/mysql/mysql.sock
[mysqld]
socket = /tmp/mysql/mysql.sock
port = 3306
and the mysql.sock file is, indeed, located in /tmp/mysql
I verified that ~/.my.cnf and /var/lib/mysql/my.cnf aren't overriding it. The mysql5 client program, etc, has no trouble connecting and neither does a groovy/grails installation on the same machine using jdbc/mysql connection
thrilllap-2:~ swirsky$ mysql5
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 6
Server version: 5.1.47 Source distribution
Copyright (c) 2000, 2010, Oracle and/or its affiliates. All rights reserved.
This software comes with ABSOLUTELY NO WARRANTY. This is free software,
and you are welcome to modify and redistribute it under the GPL v2 license
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
+--------------------+
| Database |
+--------------------+
| information_schema |
| test |
+--------------------+
2 rows in set (0.00 sec)
mysql>
Why can't MySQLdb for python figure this out? Where would it look if not the my.cnf files?

Have you tried adding the option read_default_file='~/.my.cnf' to your connect() call, to tell the Python database driver to read your configuration file? I think that otherwise it ignores the file by default.

MysqlDBlib doesn't read /etc/my.cnf by default. Adding read_default_group="client" to the MySQLdb.connect call will make it read the client section.

FWIW: I "fixed" it by changing the .sock file location in my my.cnf to be where MySQLdb wanted it (/opt/local/var/run/mysql5/mysqld.sock) but I hate doing things like that.

connect takes unix_socket as a parameter which is the path to the UNIX socket file. MySQLdb won't read your my.cnf (by design).

python mysql connection problem

This code works fine:
import MySQLdb
db = MySQLdb.connect("localhost", "root", "","bullsorbit")
cursor = db.cursor()
cursor.execute("Select * from table where conditions'")
numrows = int(cursor.rowcount)
print 'Total number of Pages : %d ' % numrows
but if I give my IP address
db = MySQLdb.connect("192.168.*.*", "root", "","bullsorbit")
it will give this error
super(Connection, self).__init__(*args, **kwargs2)
_mysql_exceptions.OperationalError: (2003, "Can't connect to MySQL server on 'ip address' (111)")

Code 2003 is a standard MySQL client error:
Error: 2003 (CR_CONN_HOST_ERROR) Message: Can't connect to MySQL server on '%s' (%d)
I'd guess that your user is not allowed to connect to the MySQL server using an iP-address. What happens if you try a connection usign the MySQL commandline client?
$ mysql --host=192.168.1.1 -u root bullsorbit

with localhost you connect via loopback-interface.
with ip-addr you connect - as you connect from extern.
if your server allows only the local (loopback) connection
your ip-addr connection fails. (security!)
look at your mysql config, maybe only local-connections are allowed.
is the skip-networking switch off (in the mysql-conf) ?
#skip-networking

For the 2003 error, another possibility is that too many connections are being attempted in too short of time. I noticed this same behavior when I had 127.0.0.1 as my connect string. First I replaced it with localhost which got me further but still the same 2003 error. Then I saw that if I scaled back the number of calls the error disappeared completely.

Instead of:
db = MySQLdb.connect("192.168..", "root", "","bullsorbit")
try:
db = MySQLdb.connect(user="root", host="192.168..", db="bullsorbit")
Password will default to empty string and try the usual identity methods. Host will also default to localhost on default port.