I have a mysql database on a server for work. I ssh ont the server and then enter the database using mysql -u username -p at which time the command line will prompt me for my password.
I'd like to access the database remotely for some development. I see that mysql.connector is a library for connecting to mysql databases, but can I ssh onto the server and then access the database using python?
You can use SSH tunneling to redirect a port listening on your local machine to a port on the remote machine.
ssh -L9999:localhost:3306 me#my.work.com
This will redirect traffic from port 9999 on your machine to port 3306 on my.work.com. We gave localhost to -L since we want to tunnel to the server itself. We could also create a tunnel through your work server to some machine accessible to it only.
Now you can connect your connector on your own machine using port 9999 and the traffic is tunneled to my.work.com:3306.
You can use the tunnel.py code from https://gist.github.com/shnjp/856179.
with make_tunnel('me#mywork.com:3306') as t:
mysql.connector.connect(host='localhost',
database='mysql',user='root',password='PASS')
This assumes that your localhost doesn't have any application running on port 3306. If you have some, then you need to use "port=" argument in make_tunnel and provide a different port to use on the localhost.
You can also connect from python to the database without using any tunnel.
For that, enable the mysql server to allow connections from the outside uncommenting and changing the bind-address line in the /etc/mysql/my.cnf file to bind-address = 0.0.0.0. After that, restart the server with sudo service mysql restart and finally grant permissions to your user to access from the outside GRANT SELECT,INSERT,UPDATE,DELETE ON your_database.* TO 'your_user'#'%' IDENTIFIED BY 'your_pass';
Now you'll be able to connect from python to the database
import mysql.connector
ip_of_the_database = 'x.x.x.x'
cnx = mysql.connector.connect(host = ip_of_the_database,
user = 'your_user',
password = 'your_pass',
database = 'your_database')
Related
I have a python script to add data to a postgres database. During testing, I had created a database in the localsystem and it worked perfectly. Now I have the database running in linode. I am not sure how to connect to the database from the localsytem. I have the database details stored in a python file as follows
DATABASE = 'my_database'
HOST = <IP_ADDRESS> # the ip_address of the linode instance
PORT = '5432'
USER = 'database_user'
PASSWORD = 'database_password'
I use the above code in access the database. in case of the database running in the localhost, the host was equal to HOST = 'localhost'.
When I run the python script from the localsystem to connect to the database this is the message I get
psycopg2.OperationalError: connection to server at "<IP_ADDESS>", port 5432 failed: Connection refused
Is the server running on that host and accepting TCP/IP connections?
I am not sure how to connect to the database running in linode.
I solved this by editing the pg_hba.conf.
In the linode instance, navigate to /var/lib/pgsql/data.
In the directory, open the pg_hba.conf and add the following line under the IPv4 local connections
host all all <local system's Public IP ADDRESS>/32 md5
Save the file and restart postgres.
Start SSH tunneling to the linode instance
Change the host and port in the database details to 127.0.0.1 and the port selected during tunneling. In my case it was 5433
Running the python script now should enable the local system to connect with the postgres server running in the linode instance.
I can read from my local psql instance like this:
engine = create_engine('postgresql://postgres:postgres#localhost/db_name')
df = pd.read_sql("select * from table_name;", engine)
I have a remote postgresql sever which I successfully accessed with ssh tunneling both in PgAdmin4 and pycharm. I use public key file to login into remote server. Now, my question is how do I access that database with pandas. I tried:
engine = create_engine('postgresql://username:password#localhost/db_name')
Here, username and password are of remote database. I get sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user. However, with the same username and password I can access the table in PgAdmin.
From what I read, because of ssh tunneling I have to use localhost and not the remote server address, right? In pgAdmin I can see that the server is running. So, my question is how do I read the table from remote postgresql database with ssh tunneling? In examples I have seen people using different port (different than 5432) but for me the setup only works if I use port 5432. I have disconnected all other servers to avoid the port conflict but I get the same error.
The tunnel created by pgAdmin4 is intended for its own use. It does not arrange for it to listen on 5432, it picks some arbitrary high numbered port and doesn't advertise what port that is. While you can discover what port it is listening on using system tools (like netstat) and then connect to it, you would probably be better served by finding some other way to set up your tunnel. There are python libraries that can help with that.
As for why you can connect to 5432 at all, clearly there is something listening there which is either PostgreSQL or pretending to be PostgreSQL, but it doesn't seem to be the one you intend. You can use netstat -ao to find the pid for it and then look up based on that.
I have a snippet of code that allows me to connect to my psql DB via ssh in Python. It works perfectly on Ubuntu 18.10 (via VirtualBox) but fails every time on windows with an error that it can't reach the remote host and port.
I'm been developing a user interface that can query data from a remote DB (logs etc.) and visualize it.
All of the development has been done using Spyder3 on Ubuntu 18.10. I never had an issue until I tried to execute the same code on Windows 10.
I tried Telnet to both the localhost:port and remote host:port (via ssh) and it works. Having looked up all the possible answers on stackoverflow and other places, I still haven't been able to fix the issue. The fact that it works on one environment and not on the other, while on the same machine, tells me it's some sort of environment setting but I don't know what it could be.
The code:
import psycopg2
import logging
logging.basicConfig(level=logging.DEBUG)
from sshtunnel import SSHTunnelForwarder
PORT = 5432
REMOTE_HOST = '111.222.111.222'
REMOTE_SSH_PORT = 22
curs = None
conn = None
server = SSHTunnelForwarder((REMOTE_HOST, REMOTE_SSH_PORT),
ssh_username='username',
ssh_password='password',
remote_bind_address=('localhost', PORT),
local_bind_address=('localhost', PORT))
server.start()
conn = psycopg2.connect(database='db_name', user='db_username', password='db_password', host='127.0.0.1', port='5432')
curs = conn.cursor()
Expected:
A successful connection to ssh and subsequent successful log-in to the database. This works on Ubuntu 18.10 via VirtualBox on the same machine.
Actual result:
2019-01-02 10:54:51,489 ERROR Problem setting SSH Forwarder up: Couldn't open tunnel localhost:5432 <> localhost:5432 might be in use or destination not reachable
I realized that my local postgres (psql) service was interfering with the port mapping as it was also using port 5432. Once I disabled the service, it worked like a charm.
I may be wrong, but I think the remote_bind_address should be set to your server's private IP. As this is where the remote machine would communicate to your machine.
remote_bind_address=(<PRIVATE_SERVER_IP>, PORT)
I am trying to create a postgresql database via sqlalchemy in pycharm (running python 3.6) but have trouble connecting / creating the database.
I initially started with sqlalchemy tutorial:
from sqlalchemy import create_table
engine =create_engine("postgresql+psycopg2://scott:tiger#localhost/test")
connection = engine.connect()
connection.close()
But it print's the error: could not connect to server: Connection refused (0x0000274D/10061)
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
How should I be opening up the port, or pointing it to the localhost?
Is there any specific thing that I need to do to create the database before i run create_engine?
I've also been searching online and found that perhaps I should be using psycopg2 to create the database, but I don't understand how this is to be done.
What I would like is to create a postgresql database on my desktop, in a specific folder.
Assuming you have the psycopg2 connector installed, then all you need to specify in the the connection string is postgresql, so your connection string would be something like
postgresql://tim:xyzzy#somehost/somedb
By default, PostgreSQL only listens on the localhost IP address, 127.0.0.1. So if the PostgreSQL server is on a different system you will need to configure the listen address in postgresql.conf
You will also need to have created the database before attempting to connect to it with SQLAlchemy. So you need to so a createdb when logged in as the PostgreSQL user.
tim#merlin:~$ sudo -i -u postgres
postgres#merlin:~$ createdb foo
postgres#merlin:~$
I am not a network/web/internet programmer so please excuse my noobness in this area. I have gotten a website using a free hosting service. They include one MySQL database. Here are the details for the database:
port = 3306
host = "fdb4.biz.nf"
database = "1284899_6067"
user = "1284899_6067"
password = "somepass9351"
I am using MySQLdb module (installed on my CLIENT machine - not server) to connect to this database:
db = MySQLdb.connect(host=host, user=user, passwd=password, db=database,port=port)
But I get the following error:
OperationalError: (2003, "Can't connect to MySQL server on 'fdb4.biz.nf' (10060)
What I have already tried
tried two different databases from different hosts
tried changing the port
tried searching SO for similar answers but all others connect to 'local host'
What I think:
could this be caused by my firewall? I am using my school's internet. I don't think this could be it because I am on CLIENT so if anything it is the SERVER'S firewall.
Two questions
Can MySQLdb be used to connect to a db on a SERVER when it is imported on a CLIENT?
If yes, what am I doing wrong?
Thank you so much for any help, its greatly appreciated! Been stuck the whole day on this.
For security reasons, mysql only listens for connections from localhost. Error code 10060 is basically that: you are not allowed to connect remotely.
Solution: find a my.ini (or my.cnf in linux) and try to find a line:
bind-address = 127.0.0.1
this line says: allow only local connections. So, you should comment-out this line, or set your IP address.
Yes, MySQLdb can connect to remote hosts.
And your usage of the connect method is correct.
You should first check if you can connect to the remote mysql server from your mysql client.
In terminal you can type mysql -h hostname -u username -p databasename
This should prompt you for the password. Enter the password. Can you connect?
If you can't connect, then you have an access problem, and its not a python - mysqldb problem
Either the server is not reachable because it is behind a firewall, in that case your client machine's ip needs to be whitelisted. Check your firewall settings
Or, the mysql server running on the remote machine is configured to accept only local connections. I think this is the default, but I'm not sure. You should ssh into the server remote host where the database server is running, locate the my.cnf file on the server and check the settings. Depending on your mysql version, the configuration would look slightly different.
Or, the user that you're trying to connect as is not associated with the ip that you're trying to connect from. Mysql users have two parts, like this: 'username'#'host'. To enable a user to connect from all ips the user needs to look like this 'user'#'%'.
I hope I've given you enough to try to debug this issue.