I'm new to Python yet managed to create a lot of good stuff for myself. The problem I faced is how to connect to an SQL database on a remote machine (VPS, VDS, Cloud)
I know that you would likely point me out to other answers on StackOverflow. Unfortunately, there is no one solved question on the website. None of the solutions worked for me.
One more time, I don't want to connect to an SQL database on a local machine. I need to access it remotely.
Can anyone provide me with working instructions?
[https://stackoverflow.com/questions/46913504/connecting-to-mysql-db-via-ssh-with-python][1]
[https://stackoverflow.com/questions/47069829/mysql-and-python-via-ssh][1]
[https://stackoverflow.com/questions/21903411/enable-python-to-connect-to-mysql-via-ssh-tunnelling][1]
[https://practicaldatascience.co.uk/data-science/how-to-connect-to-mysql-via-an-ssh-tunnel-in-python][1]
As you can see, there are numerous upvotes. But none of the approaches helped the topic starter. Otherwise, it would be marked as solved.
If you have a VPS that you can access via SSH then you can also use SSH to forward the port of the MySQL server on the VPS to your local machine.
This is something you would do for the development process of your application
Use this command to forward the remote ssh port to your local machine.
ssh -L 3306:localhost:3306 username#hostname
This way you will be able to access your VPS's MySQL server trough port 3306 on your local machine.
Here is some SSH documentation.
https://www.ssh.com/academy/ssh/tunneling/example
It is also possible to create ssh tunnels from within python but this is not recommended for a use case like yours.
Anyways, if you want to learn about this, you can read about it here
https://github.com/pahaz/sshtunnel/
Related
I have a server which contains a python file that connects to two external MySQL DB's. One of those DB's can be easily reached while the other server requires that IP's be whitelisted in order to have access to that DB. That server's IP is already whitelisted and works as intended when ran.
The problem arises however when I attempt to run the docker-ized variation of the application. The first DB runs just as it does before but the second DB no longer works. When inside the container, I can ping the second DB and it works, but whenever I try to access it via the code hosted on the server, it doesn't return data within any of the functions that utilize it. I noticed that the container has a separate IP, and may be causing issues since that docker container's IP would not have been whitelisted and may be where the problem begins. I am fairly new to Docker, so any documentation links that would assist me would be extremely helpful.
So for anyone who is dealing with this situation in the future, I added the line
network_mode: "host"
to my docker.compose.yaml file.
Here is some docs related to this: https://docs.docker.com/network/host/
Essentially what was happening is that the container could not be recognized by the whitelist and was not being allowed access to the second DB. With this change, it allowed the container to share the same network as the server it was being hosted on, and since that server has been whitelisted prior, it all worked out of the gate.
If you are using docker, then use
--net=host
within your run command. Here is a SO link about what this addition does:
What does --net=host option in Docker command really do?
I'm creating a Python 3 spider that scrapes Tor hidden services for useful data. I'm storing this data in a PostgreSQL database using the psycopg2 library. Currently, the spider script and the database are hosted on the same network, so they have no trouble communicating. However, I plan to migrate the database to a remote server on a VPS so that I can have a team of users running the spider script from a number of remote locations, all contributing to the same database. For example, I could be running the script at my house, my friend could run it from his VPS, and my professor could run the script from a few different systems in the lab at the university, and all of these individual systems could synchronize with the PostgreSQL server runnning on my remote VPS.
This would be easy enough if I simply opened the database VPS to accept connections from anywhere, making the database public. However, I do not want to do this, for security reasons. I know I could tunnel the connection through SSH, but that would require giving each person a username and password that would grant them access to the server itself. I don't wish to do this. I'd prefer simply giving them access to the database without granting access to a shell account.
I'd prefer to limit connections to the local system 127.0.0.1 and create a Tor hidden service .onion address for the database, so that my remote spider clients can connect to the database .onion through Tor.
The problem is, I don't know how to connect to a remote database through a proxy using psycopg2. I can connect to remote databases, but I don't see any option for connecting through a proxy.
Does anyone know how this might be done?
This would be easy enough if I simply opened the database VPS to accept connections from anywhere
Here lies your issue. Just simply lock down your VPS using fail2ban and ufw. Create a ufw role to only allow connection to your Postgres port from the IP address you want to give access from to that VPS ip address.
This way, you don't open your Postgres port to anyone (from *) but only to a specific other server or servers that you control. This is how you do it. Don't run an onion service to connect Postgres content because that will only complicate things and slow down the reads to your Postgres database that I am assuming an API will be consuming eventually to get to the "useful data" you will be scraping.
I hope that at least points you in the right direction. Your question was pretty general, so I am keeping my answer along the same vein.
I am trying to connect to remote machine by python script. Both the machines can be linux/windows.
I have to connect to various remote machines and it's not feasible for me to install or write some code on the remote side. I know the ip, username and password of the machine and i tried various options but was unsuccessful.
How shall I proceed.
Maybe you can use SSH to connect to a remote server.
paramiko will be good idea, it can use to connect linux/windows.
http://pxnet2768.pixnet.net/blog/post/157228756-%E7%B3%BB%E7%B5%B1%E9%81%8B%E7%B6%AD%E5%B7%A5%E7%A8%8B%E5%B8%AB%E7%9A%84%E6%B3%95%E5%AF%B6%EF%BC%9Apython-paramiko
Background
So after about a year of having a GoDaddy cloud service, and super disappointed with it from the get-go. Once they announced that they would be discontinuing Cloud Server services, it was like a sign from the heavens.
I then created a Google Cloud account. One of the biggest reasons I got a Cloud Server to begin with was to have an eclipse Che instance, an IDE wherever you are! I love it, but despite the temporary partnership between Bitnami and GoDaddy, launching a Eclipse instance with them with such a mind-numbing task since their internal Factory build still required a ton of Docker configurations...
And though I can appreciate the fact that I did learn the ins-and-outs of configuring Dockers Network settings, which is not something to wince at... As soon as I got my Google Cloud account it was simply a 1 2 3 and go!
Question
Whilst I'm running an Eclipse chat instance, what is the proper way to port-forward a given work space to my local machine? The scenario is simple...
I created a Python stack of which I am using Django but when I run server, of course default being the local IP to the project, I have yet to find the easy and more than likely existing standard way to run the Django server and have the eclipse Che create the URL to the project. I'm ninety-nine percent sure that I'm going about this the wrong way given the fact that even some of the demo stack projects with Node or Python are plug-and-play.
PS: I am able to ssh into the workspace no issue, I'm just confused on how to port forward from remote to local as I've only really done it the other way around.. ssh -R ... or -L?
What you need is SSH Tunnel, which is -L. If you need to send a port from local to server that is called a Reverse SSH Tunnel, which is -R.
so simple command
ssh -L <localport>:127.0.0.1:<remoteport> <user>#<server>
Some extension to the other answer mentioning ssh tunneling...
If you run a docker-dev on a server (e.g. 192.168.1.123) not being your local machine in eclipse-che that provides some web service you want to access, then find out the IP address of the docker-dev, e.g. by opening a terminal in your eclipse che workspace and executing ip addr. There you will see some 172.17.x.x that is accessible only from the server. Assume the service in docker-dev is listening on port 12345, then you need the following ssh port forwarding from your local machine to access it:
ssh -L 8888:172.17.0.2:12345 192.168.1.123
While the ssh connection is open, you can access the web service with you browser by accessing http://127.0.0.1:8888/
Things to note in advance:
I am using wampserver 2.2
Ive forwarded port 80
I added a rule to my firewall to accept traffic through port 3306
I have added "Allow from all" in directory of "A file i forget"
My friend can access my phpmyadmin server through his browser
I am quite the novice, so bear with me.
I am trying to get my friend to be able to alter my databases on my phpmyadmin server through
python. I am able to do so on the host machine using "127.0.0.1" as the HOST. My Question is, does he have to use my external ip as the HOST or my external ip/phpmyadmin/ as the HOST? And if using the external ip iscorrect...What could the problem be?
If your phpmyadmin runs on the same machine as mysql-server, 127.0.0.1 is enough (and safer if your mysql server binds to 127.0.0.1, rather than 0.0.0.0) if you use tcp(rather than unix socket).