Related
I'm looking to connect to a database via python.
The database is on phpmyadmin.
I read the section "variable" on phpmyadmin and I saw that the hostname is "atexo-sourcing-prod-01.local-trust.com".
Unfortunately, when I try to connect with this python code and with the same login and password that I use to connect to phpmyadmnin:
try:
conn = mariadb.connect(
user="mylogin",
password="my_password",
host="atexo-sourcing-prod-01.local-trust.com",
port=3306,
database="atexo_prod_sourcing_db")
except mariadb.Error as e:
print(f"Error connecting to MariaDB Platform: {e}")
I have an error, saying
Can't connect to server on 'atexo-sourcing-prod-01.local-trust.com'
(10061)
(I think it's not a problem of hostname, because when I put a random name, I have an other error).
Any idea to fix this?
When I started hosting my website, I received a phpMyAdmin thing where I could manage the databases for the website. How can I access those databases from my remote PC?
I tried connecting to one using Python but I keep getting the error (10061 No connection could be made because the target machine actively refused it) and I need help because I don't know how to solve it.
Here is the code I wrote:
import mysql.connector
try:
mydb = mysql.connector.connect(
host='localhost',
user='someuser',
password='somepass',
database='database'
)
except Exception as e:
print(type(e))
print(e)
You should not be using localhost if you are connecting to a remote database which is on another server machine. Instead use the database server IP address:
import mysql.connector
try:
mydb = mysql.connector.connect(
host='11.162.174.78',
port='3306',
user='user_id',
password='user_password',
database='name_of_database'
)
except Exception as e:
print(type(e))
print(e)
Having said this, in your question above you mentioned 'managed the databases for the website', to which I would suggest that you use a proper database management tool such as MySQL Workbench or HeidiSQL.
I have a MySQL server installed locally and I have Python code that accesses MySQL Database and executes a simple query:
from mysql.connector import connect
from mysql.connector import ProgrammingError
DB = {
'user':'andrei',
'password':'qwertttyy',
'host':'localhost',
'port':'3306',
'db':'my_database'
}
class Connection:
instance = None
def __new__(cls):
if not cls.instance:
try:
cls.instance = connect(**DB)
except:
raise
return cls.instance
def excuteDQL(query):
cnx = Connection()
cursor = cnx.cursor()
try:
cursor.execute(query)
return cursor.fetchall()
except ProgragrammingError as err:
print('You have an error in your MySQL syntax. Please check and retry')
return []
if __name__ == '__main__':
while True:
query = input('Enter a SQL query: ')
for tuple in executeDQL(query):
print(tuple)
If I go out there and find a cloud MySQL hosting service and pay for it, the access would be as easy as changing the DB mapping with different info?
I think it should be because the connection would still be over standard TCP/IP, except, in this case, it happens to come back the same machine that is emitting. I guess, under the hood, data is packed following TCP/IP rules up to the IP layer, and then these are transferred as IP Packets from the Python process through the OS Networking API to the MySQL Server listening to the port, without further down processing into the Access Layer since the packets never leave the machine, which I understand is the purpose of the Access Layer of the TCP/IP stack, that is, to abstract the physical road the data takes.
Did I say something coherent in my guessing?
If I'm wrong, How can I put a MySQL Server in the cloud?
Yes how you connect to the database would not change. It will be as simple as changing the host name and providing whatever credentials you need ( Access Token , User info, etc). The way you insert data doesn't change once you make a connection to the DB.
Here is a good script which should provide some info: https://gist.github.com/kirang89/7161185
Original: I have recently started getting MySQL OperationalErrors from some of my old code and cannot seem to trace back the problem. Since it was working before, I thought it may have been a software update that broke something. I am using python 2.7 with django runfcgi with nginx. Here is my original code:
views.py
DBNAME = "test"
DBIP = "localhost"
DBUSER = "django"
DBPASS = "password"
db = MySQLdb.connect(DBIP,DBUSER,DBPASS,DBNAME)
cursor = db.cursor()
def list(request):
statement = "SELECT item from table where selected = 1"
cursor.execute(statement)
results = cursor.fetchall()
I have tried the following, but it still does not work:
views.py
class DB:
conn = None
DBNAME = "test"
DBIP = "localhost"
DBUSER = "django"
DBPASS = "password"
def connect(self):
self.conn = MySQLdb.connect(DBIP,DBUSER,DBPASS,DBNAME)
def cursor(self):
try:
return self.conn.cursor()
except (AttributeError, MySQLdb.OperationalError):
self.connect()
return self.conn.cursor()
db = DB()
cursor = db.cursor()
def list(request):
cursor = db.cursor()
statement = "SELECT item from table where selected = 1"
cursor.execute(statement)
results = cursor.fetchall()
Currently, my only workaround is to do MySQLdb.connect() in each function that uses mysql. Also I noticed that when using django's manage.py runserver, I would not have this problem while nginx would throw these errors. I doubt that I am timing out with the connection because list() is being called within seconds of starting the server up. Were there any updates to the software I am using that would cause this to break/is there any fix for this?
Edit: I realized that I recently wrote a piece of middle-ware to daemonize a function and this was the cause of the problem. However, I cannot figure out why. Here is the code for the middle-ware
def process_request_handler(sender, **kwargs):
t = threading.Thread(target=dispatch.execute,
args=[kwargs['nodes'],kwargs['callback']],
kwargs={})
t.setDaemon(True)
t.start()
return
process_request.connect(process_request_handler)
Sometimes if you see "OperationalError: (2006, 'MySQL server has gone away')", it is because you are issuing a query that is too large. This can happen, for instance, if you're storing your sessions in MySQL, and you're trying to put something really big in the session. To fix the problem, you need to increase the value of the max_allowed_packet setting in MySQL.
The default value is 1048576.
So see the current value for the default, run the following SQL:
select ##max_allowed_packet;
To temporarily set a new value, run the following SQL:
set global max_allowed_packet=10485760;
To fix the problem more permanently, create a /etc/my.cnf file with at least the following:
[mysqld]
max_allowed_packet = 16M
After editing /etc/my.cnf, you'll need to restart MySQL or restart your machine if you don't know how.
As per the MySQL documentation, your error message is raised when the client can't send a question to the server, most likely because the server itself has closed the connection. In the most common case the server will close an idle connection after a (default) of 8 hours. This is configurable on the server side.
The MySQL documentation gives a number of other possible causes which might be worth looking into to see if they fit your situation.
An alternative to calling connect() in every function (which might end up needlessly creating new connections) would be to investigate using the ping() method on the connection object; this tests the connection with the option of attempting an automatic reconnect. I struggled to find some decent documentation for the ping() method online, but the answer to this question might help.
Note, automatically reconnecting can be dangerous when handling transactions as it appears the reconnect causes an implicit rollback (and appears to be the main reason why autoreconnect is not a feature of the MySQLdb implementation).
This might be due to DB connections getting copied in your child threads from the main thread. I faced the same error when using python's multiprocessing library to spawn different processes. The connection objects are copied between processes during forking and it leads to MySQL OperationalErrors when making DB calls in the child thread.
Here's a good reference to solve this: Django multiprocessing and database connections
For me this was happening in debug mode.
So I tried Persistent connections in debug mode, checkout the link: Django - Documentation - Databases - Persistent connections.
In settings:
'default': {
'ENGINE': 'django.db.backends.mysql',
'NAME': 'dbname',
'USER': 'root',
'PASSWORD': 'root',
'HOST': 'localhost',
'PORT': '3306',
'CONN_MAX_AGE': None
},
Check if you are allowed to create mysql connection object in one thread and then use it in another.
If it's forbidden, use threading.Local for per-thread connections:
class Db(threading.local):
""" thread-local db object """
con = None
def __init__(self, ...options...):
super(Db, self).__init__()
self.con = MySQLdb.connect(...options...)
db1 = Db(...)
def test():
"""safe to run from any thread"""
cursor = db.con.cursor()
cursor.execute(...)
This error is mysterious because MySQL doesn't report why it disconnects, it just goes away.
It seems there are many causes of this kind of disconnection. One I just found is, if the query string too large, the server will disconnect. This probably relates to the max_allowed_packets setting.
I've been struggling with this issue too. I don't like the idea of increasing timeout on mysqlserver. Autoreconnect with CONNECTION_MAX_AGE doesn't work either as it was mentioned. Unfortunately I ended up with wrapping every method that queries the database like this
def do_db( callback, *arg, **args):
try:
return callback(*arg, **args)
except (OperationalError, InterfaceError) as e: # Connection has gone away, fiter it with message or error code if you could catch another errors
connection.close()
return callback(*arg, **args)
do_db(User.objects.get, id=123) # instead of User.objects.get(id=123)
As you can see I rather prefer catching the exception than pinging the database every time before querying it. Because catching an exception is a rare case. I would expect django to reconnect automatically but they seemed to refused that issue.
This error may occur when you try to use the connection after a time-consuming operation that doesn't go to the database. Since the connection is not used for some time, MySQL timeout is hit and the connection is silently dropped.
You can try calling close_old_connections() after the time-consuming non-DB operation so that a new connection is opened if the connection is unusable. Beware, do not use close_old_connections() if you have a transaction.
The most common issue regarding such warning, is the fact that your application has reached the wait_timeout value of MySQL.
I had the same problem with a Flask app.
Here's how I solved:
$ grep timeout /etc/mysql/mysql.conf.d/mysqld.cnf
# https://support.rackspace.com/how-to/how-to-change-the-mysql-timeout-on-a-server/
# wait = timeout for application session (tdm)
# inteactive = timeout for keyboard session (terminal)
# 7 days = 604800s / 4 hours = 14400s
wait_timeout = 604800
interactive_timeout = 14400
Observation: if you search for the variables via MySQL batch mode, the values will appear as it is. But If you perform SHOW VARIABLES LIKE 'wait%'; or SHOW VARIABLES LIKE 'interactive%';, the value configured for interactive_timeout, will appear to both variables, and I don't know why, but the fact is, that the values configured for each variable at '/etc/mysql/mysql.conf.d/mysqld.cnf', will be respected by MySQL.
How old is this code? Django has had databases defined in settings since at least .96. Only other thing I can think of is multi-db support, which changed things a bit, but even that was 1.1 or 1.2.
Even if you need a special DB for certain views, I think you'd probably be better off defining it in settings.
SQLAlchemy now has a great write-up on how you can use pinging to be pessimistic about your connection's freshness:
http://docs.sqlalchemy.org/en/latest/core/pooling.html#disconnect-handling-pessimistic
From there,
from sqlalchemy import exc
from sqlalchemy import event
from sqlalchemy.pool import Pool
#event.listens_for(Pool, "checkout")
def ping_connection(dbapi_connection, connection_record, connection_proxy):
cursor = dbapi_connection.cursor()
try:
cursor.execute("SELECT 1")
except:
# optional - dispose the whole pool
# instead of invalidating one at a time
# connection_proxy._pool.dispose()
# raise DisconnectionError - pool will try
# connecting again up to three times before raising.
raise exc.DisconnectionError()
cursor.close()
And a test to make sure the above works:
from sqlalchemy import create_engine
e = create_engine("mysql://scott:tiger#localhost/test", echo_pool=True)
c1 = e.connect()
c2 = e.connect()
c3 = e.connect()
c1.close()
c2.close()
c3.close()
# pool size is now three.
print "Restart the server"
raw_input()
for i in xrange(10):
c = e.connect()
print c.execute("select 1").fetchall()
c.close()
I had this problem and did not have the option to change my configuration. I finally figured out that the problem was occurring 49500 records in to my 50000-record loop, because that was the about the time I was trying again (after having tried a long time ago) to hit my second database.
So I changed my code so that every few thousand records, I touched the second database again (with a count() of a very small table), and that fixed it. No doubt "ping" or some other means of touching the database would work, as well.
Firstly, You should make sure the MySQL session & global enviroments wait_timeout and interactive_timeout values. And secondly Your client should try to reconnect to the server below those enviroments values.
I'm trying to connect to MySQL on localhost using PyMySQL:
import pymysql
conn = pymysql.connect(db='base', user='root', passwd='pwd', host='localhost')
but (both on Python 2.7 and Python 3.2) I get the error:
socket.error: [Errno 111] Connection refused
pymysql.err.OperationalError: (2003, "Can't connect to MySQL server on 'localhost' (111)")
I'm sure mysqld is running because I can connect using mysql command or phpMyAdmin. Moreover, I can connect using MySQLdb on Python 2 with nearly the same code:
import MySQLdb
conn = MySQLdb.connect(db='base', user='root', passwd='pwd', host='localhost')
It seems that the problem is on PyMySQL side rather than MySQL but I have no idea how to solve it.
Two guesses:
Run mysqladmin variables | grep socket to get where the socket is located, and try setting up a connection like so:
pymysql.connect(db='base', user='root', passwd='pwd', unix_socket="/tmp/mysql.sock")
Run mysqladmin variables | grep port and verify that the port is 3306. If not, you can set the port manually like so:
pymysql.connect(db='base', user='root', passwd='pwd', host='localhost', port=XXXX)
Seems like changing localhost to 127.0.0.1 fixes the error, at least in my configuration.
If it doesn't, I would look for errors in tcp sockets connection and, of course, post it as a bug in pymysql bugtrack.
I solved the issue by replacing localhost with 127.0.0.1 and changing the password to my MYSQL database password as shown below;
conn = pymysql.connect(
host = '127.0.0.1',
port = 3306,
user = 'root',
passwd = 'XXXXXXXXX',
db = 'mysql'
)
I met the same question and my solution is as follows:
Run ssh -fN -L 3307:mysql_host:3306 ssh_user#ssh_host in my terminal.
Then input your ssh password
conn = pymysql.connect(db='base', user='root', passwd='pwd', host='localhost')
This error occurs because database does not support link directly.
I asked why socket worked but not TCP and the answer was that bind-address in /etc/my.cnf was not set correctly. This could be your problem too since the socket methods works just fine but the TCP one does not.
Those who are strugging to connect localhost MySQL from dockerised flask-sqlalchemy or using pymysql, pls look into this thread, very usefull How to connect locally hosted MySQL database with the docker container
This worked for me:
import pymysql
db = pymysql.connect(host="localhost",port=8889,user="root",passwd="root")
cursor=db.cursor()
cursor.execute("SHOW DATABASES")
results=cursor.fetchall()
for result in results:
print (result)
if you want to find the port # go to mysql in terminal, and type:
SHOW VARIABLES WHERE Variable_name = 'hostname';
SHOW VARIABLES WHERE Variable_name = 'port';
I had this same problem on AWS - and turns out that my security group was blocking the connection. I temporarily opened up all connections and voila! It connected!
Do you have any type of FW or host-based FW that could be blocking the connection? I thought it was my code and all was fine. Also check the port you are connecting on.
If you are using Docker, you might need to use host.docker.internal instead of localhost.
I managed to solve my issue by using the port without any quotation like so:
port = 3306,
You need to add the port to the connection as well. Try this and it works fine.
pymysql(Module Name).connect(host="localhost", user="root", passwd="root", port=8889, db="db_name")