I'm working on a simple Python program to query a Redshift cluster using psycopg2. When I run the code on my local machine it works as expected: it creates the connection, it runs the queries and I get the expected outcome. However, I loaded it on my EC2 instance because I want to schedule several runs a week and the execution fails with the following error:
psycopg2.OperationalError: could not connect to server: Connection timed out
Is the server running on host "xxxx" and accepting
TCP/IP connections on port 5439?
Considering that the code is working without problems on the local machine and the security settings should be the same as EC2, do you have any suggestions and/or workarounds?
Thanks a lot.
Related
I'm a student and I'm trying to write some sensor values into a MySQL database.
As IDE I'll be using Inteliji.
First off I started by installing the database Plug-in.
This was done successfully
Next I tried to connect to the data base (see figure below)
Figure of successful connection
Now The next thing I want to do is use a MySQL connector.
Therefore I've installed MySQL onto the r-PI and used following code to implement it.
import mysql.connector
print("Step 1")
cnx = mysql.connector.connect(user='a21ib2a01',
password='secret',
host='mysql.studev.groept.be',
database='a21ib2a01')
Print("Step 2")
When now I run my code the terminal will output:
Step1
For some reason I don't know; the connect function always times my program out with the next occurring errors:
mysql.connector.errors.InterfaceError: 2003: Can't connect to MySQL server on 'mysql.studev.groept.be:3306' (110 Connection timed out)
So does anyone know why my connection is successful but I can't connect to it?
Long story short what am I doing wrong and how do I fix this?
Thanks in advance!
Your timeout means the network on your rPi cannot reach -- cannot find a route to -- your MySQL host mysql.studev.groept.be.
If you do traceroute mysql.studev.groept.be in a shell in the rPi you may see what's wrong.
When in a shell on your rPi, can you ssh to any machine in your uni's network? If so, you might be able to use ssh port-forwarding to get a route to the database server.
Do you run intelliJ on the rPi directly, or on your laptop? If you run it on the laptop, it looks like the laptop can find a route to your server but the rPi cannot.
(If this were my project, I'd install a MySQL server on my laptop to reduce the network-engineering hassles of connecting through multiple hops involving a VPN.)
I am having an issue connecting to on-prem Oracle databases from AWS EKS. The error I get is:
"ORA-12545: Connect failed because target host or object does not exist".
The the exact same script, from the exact same docker image, works fine when run from any other platform (not EKS).
Verified the firewall rules are in place by obtaining a shell on the pod and using telnet to the target. We do get a basic TCP connection, however any attempts to connect with the Cx oracle client fail with the above error.
Below is my cx oracle code, I don't feel there is a issue with how the connection string is passed, because the code works fine on different platform. Just that EKS container seems to have problems making a connection.
dsn_tns = cx_Oracle.makedsn(host,port,service_name=service_name)
connection = cx_Oracle.connect(user,password,dsn_tns,encoding = 'utf-8')
I am scraping data using a Windows VPS. I am using Python MySQL-Connector to upload this scraped data to a Linux dedicated server.
import mysql.connector
mydb = mysql.connector.connect(
host="...",
user="...",
password="...",
database="...",
connection_timeout=60
)
mycursor = mydb.cursor()
About 25% of the time this fails, or takes more than twenty seconds. When it fails I get a 10060 error. Is the problem more likely to be with my Linux server or more likely my Windows VPS? Help sincerely appreciated. Here is the error message:
2055: Lost connection to MySQL server at '[IP]:3306', system error: 10060 A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond.
This error is likely ralated to slow mysql database server, so you could do following:
Increase your connection_timeout to higher value i.e. 600.
Configure settings that are responsible for database server perormance (i.e. see this presentation - https://www.oracle.com/technetwork/community/developer-day/mysql-performance-tuning-403029.pdf) especially set mysql RAM usage server (1/2 of total system RAM - variable innodb_buffer_pool_size (set it in my.cnf))
Ensure that your closes cursor after your finished to use it, call mycursor.close.
Increase if possible resources (CPU, RAM, more faster disk drive) of a system with your database server.
Looking at the error message, I suspect this is caused by a port being blocked (the default TCP/IP port of the MySQL server is 3306).
Check if the port 3306 is busy:
netstat -tulpn | grep 3306
then try to free that port before running your app again.
I am trying to build a data processing pipline. In a nutshell the pipline includes the follwing tasks: connect to a remote SQL db, extract data, process it and output files. I've built all these steps in python. Since the db is on a server I can only run the pipline only from a private server (with a static IP address) - for security reasons. So I ran all my py scripts from my own server (ubuntu). Now, to stitch the components of the pipilne I want to use apache-airflow. I followed a couple of instructions from different sources and failed to make it work. Everything seems to run but when I run airflow webserver the local host refuses the connection (for some reason). My question is rather general. I want to know if there is a good step by step instruction on how to install and use apache airflow specifically on remote server (ubuntu).
I have tried the following:
https://medium.com/#taufiq_ibrahim/apache-airflow-installation-on-ubuntu-ddc087482c14
https://vujade.co/install-apache-airflow-ubuntu-18-04/
https://towardsdatascience.com/getting-started-with-apache-airflow-df1aa77d7b1b
plus the same with docker.
Here's the log of the error I get: (connection refused)
Here's the the screen shot of when I run airflow webserver:
I wish to connect to a database server through my local machine at work, but I do not have direct access to the database server (due to security reasons). The database server is accessible through another intermediary server which I can connect to.
I understand I can connect to the database if I run my script on the intermediary server, but is there any way through which I can connect to the database server directly through my local machine?
I am trying to do this in a Python script as I wish to read the data into a pandas dataframe (I can do this part once I can set up the connection).
If you have SSH access to that intermediary server you can connect via an SSH tunnel. This post describes how to do that: Enable Python to Connect to MySQL via SSH Tunnelling