I followed the answer here to create a connection using psycopg2. It works on the first call on the endpoint. The second try gives this error psycopg2.InterfaceError: connection already closed. Below is a snippet of my code:
from config import conn
with conn:
with conn.cursor() as cursor:
cursor.execute("""
select ...
"""
)
pos = cursor.fetchone()
cursor.execute("""
select ...'
"""
)
neg = cursor.fetchone()
conn.close()
Since you're closing the connection in the last line of your code, it cannot be used further (when calling the endpoint for the second time) without reconnecting.
You can either delete the last line (which would result in never closing connection) and move conn.close() to app shutdown event or perform psycopg2.connect each time you need it.
Related
I am trying to execute stored procedure by using pyodbc in databricks, after executing SP I tried to commit the connection but, commit is not happening. Here I am giving my code, please help me out from this issue.
import pyodbc
#### Connecting Azure SQL
def db_connection():
try:
username = "starsusername"
password = "password-db"
server = "server-name"
database_name = "db-name2"
port = "db-port"
conn=pyodbc.connect('Driver={ODBC Driver 17 for SQL server};SERVER=tcp:'+server+','+port+';DATABASE='+ database_name +';UID='+ username +';PWD='+ password)
cursor=conn.cursor()
return cursor, conn
except Exception as e:
print("Faild to Connect AZURE SQL: \n"+str(e))
cursor, conn = db_connection()
# conn1.autocommit=True
cursor.execute("delete from db.table_name")
cursor.execute("insert into db.table_name(BUSINESS_DATE) values('2021-10-02')")
cursor.execute("exec db.SP_NAME '20211023'")
conn.commit()
conn.close()
here I am commiting connection after SP excution. deletion and insertion is not happening at all. and I tried with cursor.execute("SET NOCOUNT ON; exec db.SP_NAME '20211023'") but it's also not working.
Thanks in Advance
If you check this document on pyodbc, you will find that -
To call a stored procedure right now, pass the call to the execute method using either a format your database recognizes or using the ODBC call escape format. The ODBC driver will then reformat the call for you to match the given database.
Note that after connection is set up or done, try doing conn.autocommit = True before calling your SP and it will help. By default it is false.
Executing the Stored Procedure.
You will be able to execute your stored procedure if you follow the below code snippet.
cursor = conn.cursor()
conn.autocommit = True
executesp = """EXEC yourstoredprocedure """
cursor.execute(executesp)
conn.commit()
Delete the Records in SQL Server
You can delete record as shown in the below example.
...#just an example
cursor.execute('''
DELETE FROM product
WHERE product_id in (5,6)
''')
conn.commit()
Don’t forget to add conn.commit() at the end of the code, to ensure that the command would get executed.
Insert record in SQL Server
The below snippet show how we can do the same.
...#just an example
cursor.execute("INSERT INTO EMP (EMPNO, ENAME, JOB, MGR) VALUES (535, 'Scott', 'Manager', 545)")
conn.commit()
I will suggest you to read the for following document for more information.
Delete Record Documentation.
Insert Record Document
I write a program that, with the help of pyodbc, connects to the base several times and performs selects.
Unfortunately, I have to reestablish the connection before calling any of my methods.
Why doesn't a single connection in each method work?
# create object (connect to DB)
conn = db.db_connect()
# Call method with my select
weak_password_list = db.Find_LoginsWithWeakPassword(conn)
# I need to connect again
conn = db.db_connect()
# Call method with my select
logins_with_expired_password = db.LoginsWithExpiredPassword(conn)
# And again...
conn = db.db_connect()
# Call method with my select
logins_with_expiring_password = db.Find_LoginsWithExpiringPassword(conn)
######################################################
def db_connect(self):
try:
conn = pyodbc.connect('Driver={SQL Server};'
'Server='+self.server_name+''
'Database='+self.database_name+';'
'Trusted_Connection='+self.trusted_connection+'')
except Exception as e:
conn = ""
self.print_error("Failed to connect to the database.", e)
return conn
############################
def Find_LoginsWithWeakPassword(self, conn):
try:
cursor = conn.cursor()
query_result = cursor.execute('''SELECT * FROM table_name''')
except Exception as e:
query_result=""
self.print_error("Select failed in Find_LoginsWithWeakPassword", e)
return query_result
If I only connect once, the second and subsequent methods with select has no effect.
Why?
When you call
weak_password_list = db.Find_LoginsWithWeakPassword(conn)
the function returns the pyodbc Cursor object returned by .execute():
<pyodbc.Cursor object at 0x012F4F60>
You are not calling .fetchall() (or similar) on it, so the connection has an open cursor with unconsumed results. If you do your next call
logins_with_expired_password = db.LoginsWithExpiredPassword(conn)
without first (implicitly) closing the existing connection by clobbering it, then .execute() will fail with
('HY000', '[HY000] [Microsoft][ODBC SQL Server Driver]Connection is busy with results for another hstmt (0) (SQLExecDirectW)')
TL;DR: Consume you result sets before calling another function, either by having the functions themselves call .fetchall() or by calling .fetchall() on the cursor objects that they return.
I would like to run below postgresql function queries in python but I keep getting error message says
cursor = connection.cursor()
cursor.execute("SELECT
ST_Distance_sphere(st_makepoint(32.836956,39.925018)
,st_makepoint(28.990174,41.036857))")
df = cursor.fetchall()
df
InFailedSqlTransaction: current transaction is aborted, commands ignored until end of transaction block
How can I fix this? Thank you.
According to psycopg docs, you probably have an error in your command (SQL).
There was a problem in the previous command to the database, which
resulted in an error. The database will not recover automatically from
this condition: you must run a rollback() before sending new commands
to the session (if this seems too harsh, remember that PostgreSQL
supports nested transactions using the SAVEPOINT command).
I highly recommend using try/except/finally clause in your database connections. Or use with statement.
Here is an example from http://www.postgresqltutorial.com/postgresql-python/transaction/:
conn = psycopg2.connect(dsn)
# transaction 1
with conn:
with conn.cursor() as cur:
cur.execute(sql)
# transaction 2
with conn:
with conn.cursor() as cur:
cur.execute(sql)
conn.close()
I am using psycopg2 library to handle connection with Postgress database.
Are the following two approaches to handling db connection comparable?
Snipet 1:
cnx = connect(user=<...>, password=<...>, host=<...>, database=<...>)
cursor = cnx.cursor()
cursor.execute(sql)
cnx.close()
Snipet 2:
with psycopg2.connect(user=<...>, password=<...>, host=<...>, database=<...>) as cnx:
cursor = cnx.cursor()
cursor.execute(sql)
I am especialy interested if using WITH automatically closes connection? I was informed that second approach is preferable because it does connection close automatically. Yet when i test it using cnx.closed it shows open connection.
The with block tries on exit to close (commit) a transaction, not a connection. For the documentation:
Note that, unlike file objects or other resources, exiting the connection’s with block doesn’t close the connection but only the transaction associated with it [...]
I am trying to create a login function. But it only works ones. Ex- When i give a wrong userid and password I got correct error massage that "Could't login" after canceling that message and giving correct userid and password then I get "pymysql.err.Error: Already closed" below are the sample code.
import pymysql
# Connect to the database
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='python_code',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
class LoginModel:
def check_user(self, data):
try:
with connection.cursor() as cursor:
# Read a single record
sql = "SELECT `username` FROM `users` WHERE `username`=%s"
cursor.execute(sql, (data.username))
user = cursor.fetchone()
print(user)
if user:
if (user, data.password):
return user
else:
return False
else:
return False
finally:
connection.close()
You have a mismatch with respect to the number of times you're creating the connection (once) and the number of times you're closing the connection (once per login attempt).
One fix would be to move your:
connection = pymysql.connect(host='localhost',
user='root',
password='',
db='python_code',
charset='utf8mb4',
cursorclass=pymysql.cursors.DictCursor)
into your def check__user(). It would work because you'd create and close the connection on each invocation (as others have pointed out, the finally clause always gets executed.)
That's not a great design because getting database connections tends to be relatively expensive. So keeping the connection creating outside of the method is preferred.... which means you must remove the connection.close() within the method.
I think you're mixing up connection.close() with cursor.close(). You want to do the latter, not the former. In your example you don't have to explicitly close the cursor because that happens automatically with your with connection.cursor() as cursor: line.
Change finally to except, or remove the try block completely.
This is the culprit code:
finally:
connection.close()
Per the docs:
"A finally clause is always executed before leaving the try statement, whether an exception has occurred or not"
From: https://docs.python.org/2/tutorial/errors.html
You didn't describe alternative behavior for what you would like to see happen instead of this, but my answer addresses the crux of your question.
Had the same issue. The "Finally clause is needed for Postgres with the psycopg2 driver, if used with context manager (with clause), it close the cursor but not the connection. The same does not apply with Pymysql.