I was writing a proprietary script that queries a company database to pull certain information. I was using Psycopg2. At this point the lines I was using were like:
conn = psycopg2.connect("dbname='somedb' user='usr' host='something.azure.com' password='pswd' port='5432'")
cur = conn.cursor()
query = "something"
cur.execute(query)
results = cur.fetchall()
The script was running fine until a few days ago, after quick successions of runs during debugging, I started getting:
psycopg2.OperationalError: server closed the connection unexpectedly
This probably means the server terminated abnormally
before or while processing the request.
A closer look that I realized I did not properly close the connection. So I modified it to:
with psycopg2.connect("dbname='somedb' user='usr' host='something.azure.com' password='pswd' port='5432'") as conn:
cur = conn.cursor()
query = "something"
cur.execute(query)
results = cur.fetchall()
The error persisted even when no additional connections were present on the server end, so hitting max_connection is unlikely the reason here.
Strangely, I can access the same server through PGAdmin or use sqlalchemy.create_engine with pandas.read_sql on the same machine. Additionally, the script runs fine on another colleague's machine while mimicking my IP address through VPN.
Edit: sqlalchemy worked for my machine for exactly once, after which I started to get the same error.
My sqlalchemy code:
engine = create_engine('postgresql://{0}:{1}#{2}:5432/db'.format(USR, PSWD, HOST))
query = "something"
df = pd.read_sql(sql=query, con=engine)
I am trying to get Python to run a stored procedure in my SQL Server which kicks off a series of procedures which involves importing a file processing it and outputting a couple of files.
So far I have got my code so that it accepts an input to a table but then the whole thing hangs when it calls the stored procedure.
Checking Who2 on the server it is waiting on the preemptive_OS_Pipeops which searching has revealed it is waiting on something outside of SQL Server to finish before proceeding.
Is someone able to shed some light if it is possible to use pyodbc to blind activate a stored procedure then close the connection?
My belief is by just telling the procedure to run then closing out should fix the issue but I am having issues finding the code for this
Python code:
connection2 = pypyodbc.connect('Driver={SQL Server}; Server=server;Database=db', timeout=1)
cursor2 = connection2.cursor()
cursor2.execute("{CALL uspGoBabyGo}")
connection2.close()
return 'file uploaded successfully'
Stored procedure:
BEGIN
SET NOCOUNT ON;
EXECUTE [dbo].[uspCableMapImport]
END
After searching and the script stopped posting the record to the table I found the solution to the issue. I needed to add in the autocommit=True line to the script, now the code is as follows;
connection = pyodbc.connect('Driver={SQL Server};
Server='Server';Database='DB';Trusted_Connection=yes')
connection.autocommit=True
cursor = connection.cursor()
referee = file.filename.rsplit('.')[0]
SQLCommand = ("INSERT INTO RequestTable(Reference, Requested) VALUES ('"+ str(referee) +"', " + datetime.now().strftime('%Y-%m-%d') + ")")
cursor.execute(SQLCommand)
connection.commit
SQLCommand2 = ("{CALL uspGoBabyGo}")
cursor.execute(SQLCommand2)
connection.commit
connection.close()
I am trying to run MySQL commands through Python source code and have it display in my web browser.
When I compile the code in an editor (Komodo Edit) it complies and returns the expected set of rows, but when I try to view in a browser the source code is displayed, not the query result I seek...
Here's what I have...
import mysql.connector
conn = None
try:
conn = mysql.connector.Connect(host="localhost", user="jump", password="secret")
print('Connected to MySQL!')
except Exception as ex:
print('cannot connect to MySQL : exception : ' + str(ex))
cursor = conn.cursor()
cursor.close()
print('terminating connection with MySQL')
conn.close()
...I'm pretty sure the source is good, and that somewhere I missed a configuration step or some such.
I can get my command prompt to open the source file and display the query results, so I've got a connection between Python and MySQL server, which I verified through the MySQL workbench also.
Thanks for any help y'all can give!
-M
I have case :
import pymysql
conn = pymysql.connect(host='127.0.0.1', unix_socket='/opt/lampp/var/mysql/mysql.sock', user='root', passwd=None, db='test')
cur = conn.cursor()
cur.execute("test < /mypath/test.sql")
cur.close()
conn.close()
I always get error :
1064 , "You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near 'test < /mypath/test.sql' at line 1"
I tried to use source and it still failed. Did you know why?
Thank you.
Your error message says that the MySQL server can't understand
test < /mypath/test.sql' at line 1
If you're a long time *nix user, it seems intuitive that you should be able to use commands like this to pass various sorts of data streams to various programs. But that's not the way the Python sql API (or most language-specific) sql APIs works.
You need to pass a valid SQL query to the execute() method in the API, so the API can pass it to the database server. A vaild query will be something like INSERT or CREATE TABLE.
Look, the server might be on a different host machine, so telling the server to read from /mypath/test.sql is very likely a meaningless instruction to that server. Even if it did understand it, it might say File test.sql not found.
The mysql(1) command line client software package can read commands from files. Is that what you want?
>>> import MySQLdb
>>> db = MySQLdb.connect(host = 'demodb', user = 'root', passwd = 'root', db = 'mydb')
>>> cur = db.cursor()
>>> cur.execute('select * from mytable')
>>> rows = cur.fetchall()
Install MySQL-Python package to use MySQLdb.
I have built a site using Django and I am receiving this annoying error when I am trying to execute a query.
If I restart the Apache server, the error will go away for a short time.
Traceback:
File "/usr/local/lib/python2.7/site-packages/django/core/handlers/base.py" in get_response
100. response = callback(request, *callback_args, **callback_kwargs)
File "/home/fran/cron/views/set_caches.py" in set_caches
24. cursor.execute(query, [category['id']])
File "/usr/local/lib/python2.7/site-packages/django/db/backends/util.py" in execute
15. return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/site-packages/django/db/backends/mysql/base.py" in execute
86. return self.cursor.execute(query, args)
File "build/bdist.linux-i686/egg/MySQLdb/cursors.py" in execute
155. charset = db.character_set_name()
Exception Type: InterfaceError at /blablabla/
Exception Value: (0, '')
This is caused by a global cursor. Try creating and closing the cursor within each method a raw query is needed.
cursor = connection.cursor()
cursor.execute(query)
cursor.close()
You get this error when you have a db.close() call and later try to access the database without creating a new connection. Try to find if you close the connection to the database when you don't mean to.
I agreed with Moberg. This error is caused when we try to access the database after we have closed the connection. This could be caused by some wrong indentation in the code. Below is my code.
conn = connect()
cur = conn.cursor()
tk = get_tickers(cur)
for t in tk:
prices = read_price(t, cur)
if prices != None:
update_price(t, cur)
print 'Price after update of ticker ', t, ':'
p_open, p_high, p_low, p_close = read_price(t, cur)
print p_open, p_high, p_low, p_close
else:
print 'Price for ', t, ' is not available'
conn.close()
I got the same error as reported by Marian. After dedenting conn.close(), everything worked well. Confirmed that global conn is not an issue.
I had the same problem as for April of 2019 using python 3.7 and Mysql 2.7.
At intermittent intervals, the string (0, '') would be added at random to my SQL statements causing errors. I have solved the issue by commenting on the closing of the database connection and just leaving the closing of the cursors across my code.
def set_db():
db = pymysql.connect(host='localhost',
user="root",
passwd="root",
db="DATABASE")
return db
def execute_sql(cnx, sql_clause, fetch_all):
if sql_clause and sql_clause is not None:
try:
cnx.execute(sql_clause)
except Exception as e:
print("Error in sql: " + sql_clause + str(e))
return 0
pass
if fetch_all:
result = cnx.fetchall()
else:
result = cnx.fetchone()
return result
else:
print("Empty sql.")
return 0
db = set_db()
cnx = db.cursor()
sql = "SELECT * FROM TABLE"
result = execute_sql(cnx, sql, 1)
cnx.close() #close the cursor
#db.close #do not close the db connection
...
I had the same issue using threading with Python3 and Pymysql. I was getting deadlocks and then I would get hit with InterfaceError (0, '').
My issue was that I was trying to do a rollback on exception of the query- I believe this rollback was trying to use a connection that no longer existed and it was giving me the interface error. I took this rollback out (because I am OK with not doing rollback for this query) and I just let things go. This fixed my issue.
def delete_q_msg(self, assetid, queuemsgtypeid, msgid):
"""
Given the paramerts below remove items from the msg queue equal to or older than this.
If appropriate send them into a history table to be processed later
:param assetid:
:param queuemsgtypeid:
:param msgid:
:return:
"""
params = (assetid, queuemsgtypeid, msgid,)
db_connection = self._connect_to_db()
sp_sql = "{db}.ps_delete_q_msg".format(db=self._db_settings["database"])
return_value = []
try:
with db_connection.cursor() as cursor:
cursor.callproc(sp_sql, params)
return_value = cursor.fetchall()
db_connection.commit()
except Exception as ex:
# i think we dont want rollback here
# db_connection.rollback()
raise Exception(ex)
finally:
db_connection.close()
return return_value
I can confirm this is caused by a global cursor which is then later used in some functions. My symptoms were the exact same: intermittent interface errors that would temporarily be cleared up by an apache restart.
from django.db import connection
cursor = connection.cursor() # BAD
def foo():
cursor.execute('select * from bar')
But, I am using Django on top of Oracle 11.2 so I do not believe this is a bug in the MySQL/python driver. This is probably due to the caching done by apache/mod_wsgi.
I had the same issue with Flask+pymysql, I was getting an empty tuple as a result in the except: block, something like this "(\"(0, '')\",)" to be specific.
It turned out that the connection was getting closed and later the code tried accessing it which resulted into this error.
So I solved it by referring to above solutions and used a function for connection which assured me a conn every time I had to access the db.
You can recreate this issue by inserting conn.close() just before accessing the cursor.
For reference I used this site which helped me solve this issue.
https://hackersandslackers.com/python-mysql-pymysql/
For me, removing the conn.close() from my function worked. I was trying to access the database again after closing.
I am using flask with AWS.
Also you can try to restart your flask application if it has been running for a long time & if you are also using AWS RDS with MYSQL workbench like in my case, then just check whether your session is expired or not and update the access key and id.
Hope this helps.
I had this same problem and what worked for me in Django was what is described in this answer, which consists of:
Replacing
'ENGINE': 'django.db.backends.mysql'
with
'ENGINE': 'mysql_server_has_gone_away'
on
settings.DATABASES['ENGINE']
and installing with pip the package below:
mysql_server_has_gone_away==1.0.0
with connections.cursor() as cursor:
res=cursor.execute(sql)