I have django project in which I can display records from raspberry pi device. I had mysql database and i have send records from raspberry there. I can display it via my api, but I want to work on this records.I want to change this to django database but I don't know how I can get access to django database which is on VPS server from raspberry pi device.
ALERT: THIS CAN LEAD TO SECURITY ISSUES
A Django database is no different from any other database. In this case a MySQL.
The VPS server where the MySQL is must have a public IP, the MySQL must be listening on that IP (if the VPS has a public IP but MySQL is not listening/bind on that IP, it won't work) and the port of the MySQL open (default is 3306), then you can connect to that database from any program with the required configurations params (host, port, user, password,...).
I'm not a sysadmin expert, but having a MySQL on a public IP is a security hole. So the best approach IMO is to expose the operations you want to do via API with Django.
Related
I have a client computer (now Windows 10, soon to be Ubuntu Server) connected through a LAN Ethernet interface to a remote DB server using SQLAlchemy. The same client holds a Wifi network interface connected to Internet in order to provide external access to the client. Within the client there is a Python script that runs reading the remote DB Server and updating a local DB with records of interest.
Everything works fine while the Wifi interface is disconnected, and SQLAlchemy's engine.connect() throws a connection error when the Wifi becomes active.
The question is how to force the connection to be done through the Ethernet interface for the following commands:
engine = create_engine(url)
engine.connect()
I am expecting some sort of default routing configuration for the SQLAlchemy engine, or a workaround that does not involve SQLAlchemy.
I can read from my local psql instance like this:
engine = create_engine('postgresql://postgres:postgres#localhost/db_name')
df = pd.read_sql("select * from table_name;", engine)
I have a remote postgresql sever which I successfully accessed with ssh tunneling both in PgAdmin4 and pycharm. I use public key file to login into remote server. Now, my question is how do I access that database with pandas. I tried:
engine = create_engine('postgresql://username:password#localhost/db_name')
Here, username and password are of remote database. I get sqlalchemy.exc.OperationalError: (psycopg2.OperationalError) FATAL: password authentication failed for user. However, with the same username and password I can access the table in PgAdmin.
From what I read, because of ssh tunneling I have to use localhost and not the remote server address, right? In pgAdmin I can see that the server is running. So, my question is how do I read the table from remote postgresql database with ssh tunneling? In examples I have seen people using different port (different than 5432) but for me the setup only works if I use port 5432. I have disconnected all other servers to avoid the port conflict but I get the same error.
The tunnel created by pgAdmin4 is intended for its own use. It does not arrange for it to listen on 5432, it picks some arbitrary high numbered port and doesn't advertise what port that is. While you can discover what port it is listening on using system tools (like netstat) and then connect to it, you would probably be better served by finding some other way to set up your tunnel. There are python libraries that can help with that.
As for why you can connect to 5432 at all, clearly there is something listening there which is either PostgreSQL or pretending to be PostgreSQL, but it doesn't seem to be the one you intend. You can use netstat -ao to find the pid for it and then look up based on that.
I wish to connect to a database server through my local machine at work, but I do not have direct access to the database server (due to security reasons). The database server is accessible through another intermediary server which I can connect to.
I understand I can connect to the database if I run my script on the intermediary server, but is there any way through which I can connect to the database server directly through my local machine?
I am trying to do this in a Python script as I wish to read the data into a pandas dataframe (I can do this part once I can set up the connection).
If you have SSH access to that intermediary server you can connect via an SSH tunnel. This post describes how to do that: Enable Python to Connect to MySQL via SSH Tunnelling
I'm creating a Python 3 spider that scrapes Tor hidden services for useful data. I'm storing this data in a PostgreSQL database using the psycopg2 library. Currently, the spider script and the database are hosted on the same network, so they have no trouble communicating. However, I plan to migrate the database to a remote server on a VPS so that I can have a team of users running the spider script from a number of remote locations, all contributing to the same database. For example, I could be running the script at my house, my friend could run it from his VPS, and my professor could run the script from a few different systems in the lab at the university, and all of these individual systems could synchronize with the PostgreSQL server runnning on my remote VPS.
This would be easy enough if I simply opened the database VPS to accept connections from anywhere, making the database public. However, I do not want to do this, for security reasons. I know I could tunnel the connection through SSH, but that would require giving each person a username and password that would grant them access to the server itself. I don't wish to do this. I'd prefer simply giving them access to the database without granting access to a shell account.
I'd prefer to limit connections to the local system 127.0.0.1 and create a Tor hidden service .onion address for the database, so that my remote spider clients can connect to the database .onion through Tor.
The problem is, I don't know how to connect to a remote database through a proxy using psycopg2. I can connect to remote databases, but I don't see any option for connecting through a proxy.
Does anyone know how this might be done?
This would be easy enough if I simply opened the database VPS to accept connections from anywhere
Here lies your issue. Just simply lock down your VPS using fail2ban and ufw. Create a ufw role to only allow connection to your Postgres port from the IP address you want to give access from to that VPS ip address.
This way, you don't open your Postgres port to anyone (from *) but only to a specific other server or servers that you control. This is how you do it. Don't run an onion service to connect Postgres content because that will only complicate things and slow down the reads to your Postgres database that I am assuming an API will be consuming eventually to get to the "useful data" you will be scraping.
I hope that at least points you in the right direction. Your question was pretty general, so I am keeping my answer along the same vein.
I am not a network/web/internet programmer so please excuse my noobness in this area. I have gotten a website using a free hosting service. They include one MySQL database. Here are the details for the database:
port = 3306
host = "fdb4.biz.nf"
database = "1284899_6067"
user = "1284899_6067"
password = "somepass9351"
I am using MySQLdb module (installed on my CLIENT machine - not server) to connect to this database:
db = MySQLdb.connect(host=host, user=user, passwd=password, db=database,port=port)
But I get the following error:
OperationalError: (2003, "Can't connect to MySQL server on 'fdb4.biz.nf' (10060)
What I have already tried
tried two different databases from different hosts
tried changing the port
tried searching SO for similar answers but all others connect to 'local host'
What I think:
could this be caused by my firewall? I am using my school's internet. I don't think this could be it because I am on CLIENT so if anything it is the SERVER'S firewall.
Two questions
Can MySQLdb be used to connect to a db on a SERVER when it is imported on a CLIENT?
If yes, what am I doing wrong?
Thank you so much for any help, its greatly appreciated! Been stuck the whole day on this.
For security reasons, mysql only listens for connections from localhost. Error code 10060 is basically that: you are not allowed to connect remotely.
Solution: find a my.ini (or my.cnf in linux) and try to find a line:
bind-address = 127.0.0.1
this line says: allow only local connections. So, you should comment-out this line, or set your IP address.
Yes, MySQLdb can connect to remote hosts.
And your usage of the connect method is correct.
You should first check if you can connect to the remote mysql server from your mysql client.
In terminal you can type mysql -h hostname -u username -p databasename
This should prompt you for the password. Enter the password. Can you connect?
If you can't connect, then you have an access problem, and its not a python - mysqldb problem
Either the server is not reachable because it is behind a firewall, in that case your client machine's ip needs to be whitelisted. Check your firewall settings
Or, the mysql server running on the remote machine is configured to accept only local connections. I think this is the default, but I'm not sure. You should ssh into the server remote host where the database server is running, locate the my.cnf file on the server and check the settings. Depending on your mysql version, the configuration would look slightly different.
Or, the user that you're trying to connect as is not associated with the ip that you're trying to connect from. Mysql users have two parts, like this: 'username'#'host'. To enable a user to connect from all ips the user needs to look like this 'user'#'%'.
I hope I've given you enough to try to debug this issue.