Asnycpg PostgreSQL connection issues - python

I'm using Python with asnycpg to interact with my PostgreSQL database.
After some time, if I don't interact with it and then try to do so, I get an connection is closed error. Is this a server side config or client side config issue?
How to solve it?

The database automatically closes the connection due to security reasons.
So I suggest you to open the connection to db just before running queries with asyncpg, and then to reclose the connection right afterwards.
Furthermore, you can manage the possible errors that you get when the connection is closed by properly rising exceptions.
Take a look at this example:
Import asyncpg
print("I'm going to run a query with asyncpg")
# if the connection to db is not opened, then open it
if not connection:
# try to open the connection to db
try:
connection = await asyncpg.connect(
host=YOUR_DATABASE_HOST,
user=YOUR_DATABASE_USER,
password=YOUR_DATABASE_PASS,
database=YOUR_DATABASE_DB_NAME
)
except (Exception, asyncpg.ConnectionFailureError) as error:
Print("Error while connecting to db: {}".format(error))
else:
#connection already up and running
pass
QUERY_STRING = """
INSERT INTO my_table(field_1, field_2)
VALUES ($1, $2);
"""
try:
await connection.execute(QUERY_STRING, value_to_assign_to_field_1, value_to_assign_to_field_2)
return None
# except (Exception, asyncpg.UniqueViolationError) as integrError:
# print("You are violating unique constraint.")
except (Exception, asyncpg.ConnectionFailureError) as error:
print("Connection to db has failed. Cannot add data.")
return "{}".format(error)
finally:
if (connection):
await Utils.close_connection(connection)
print("data has been added. closing the connection.")

Related

Postgresql - How to open an Autocommit Connection to handle a Vacuum Full

I am trying to achieve the same thing as in earlier question psycopg2: How to execute vacuum postgresql query in python script; however, the recommendation to open an autocommit connection includes a link which is broken.
The below code runs without error BUT the table is not vacuumed.
How does this need to be written to call the Vacuum Full correctly?
#!/usr/bin/python
import psycopg2
from config import config
def connect():
""" Connect to the PostgreSQL database server """
conn = None
try:
# read connection parameters
params = config()
# connect to the PostgreSQL server
conn = psycopg2.connect(**params)
conn.autocommit=1
# create a cursor
cur = conn.cursor()
# execute Vacuum Full
cur.execute('Vacuum Full netsuite_display')
# close the communication with the PostgreSQL
cur.close()
except (Exception, psycopg2.DatabaseError) as error:
print(error)
finally:
if conn is not None:
conn.close()
print('Database connection closed.')
if __name__ == '__main__':
connect()

SQL do not commit when hosted by server?

I made a website with flask which connect to a db file with sqlite3. When i run it locally it functions, but when I host it on vercel, the log shows no error but the db never commit and save any changes.
#example of a update
with sqlite3.connect(db) as connection:
try:
connection.execute("UPDATE Survey SET Responders = Survey.Responders+1 WHERE SurveyID = ?",(data["id"],))
connection.commit()
print("response added")
except Error as e:
print(e)
The database never save any changes, although i can select from the connection normally

Implement connection pooling with Python for connecting to SQL Server on Windows

I am writing a Python script that will read data from a SQL Server database. For this I have used pyodbc to connect to SQL Server on Windows (my driver is ODBC Driver 17 for SQL Server).
My script works fine, but I need to use a connection pool instead of a single connection to manage resources more effectively. However the documentation for pyodbc only mentions pooling without providing examples of how connection pooling can be implemented. Any ideas of how this can be done using Python while connecting to an SQL Server? I only found solutions for PostgreSQL that use psycopg2, but this does not work for me obviously.
At the moment my code looks like this (please disregard the missing indentation which happened when copying the file from my IDE):
def get_limited_rows(size):
try:
server = 'here-is-IP-address-of-servier'
database = 'here-is-my-db-name'
username = 'here-is-my-username'
password = 'here-is-my-password'
conn = pyodbc.connect('DRIVER={ODBC Driver 17 for SQL Server};SERVER='+server+';DATABASE='+database+';UID='+username+';PWD='+password)
cursor = conn.cursor()
print('Connected to database')
select_query = 'select APPN, APPD from MAIN'
cursor.execute(select_query)
while True:
records = cursor.fetchmany(size)
if not records:
cursor.close()
sys.exit("Completed")
else:
for record in records:
print(record)
time.sleep(10)
except pyodbc.Error as error:
print('Error reading data from table', error)
finally:
if (conn):
conn.close()
print('Data base connection closed')

Communication link failure with pyodbc

import pyodbc
import time
connection = pyodbc.connect(..............)
cursor = connection.cursor()
while True:
time.sleep(1)
cursor.execute(INSERT_QUERY)
cursor.commit()
that works. but suddenly I got an exception pyodbc.Error: ('08S01', '[08S01] [Microsoft][ODBC SQL Server Driver]Communication link failure (0) (SQLExecDirectW)')
Why is that ?Why the link suddenly disconnect ? How can I handle that exeption and re-connect? how can I fix that?
By googling the error code, it means the connection failed for some reason for another.
You might want to add retry/reconnection logic for that case; crudely something like this.
import pyodbc
import time
connection = None
while True:
time.sleep(1)
if not connection: # No connection yet? Connect.
connection = pyodbc.connect("..............")
cursor = connection.cursor()
try:
cursor.execute(INSERT_QUERY)
cursor.commit()
except pyodbc.Error as pe:
print("Error:", pe)
if pe.args[0] == "08S01": # Communication error.
# Nuke the connection and retry.
try:
connection.close()
except:
pass
connection = None
continue
raise # Re-raise any other exception

MySQL Connector in Python

Trying to connect to my MySQL DB on my VPS. I'm getting the following error, when using the below code;
(I've stripped out my credentials - however I know they work, as I use them for my PHP dev as well.
CODE
import mysql.connector as mysql
from mysql.connector import errorcode
try:
conn = mysql.connect( user= %USER%
,password=%PW%
,host = %HOST%
,database = %DB%
)
except mysql.Error as err:
if err.errno == errorcode.ER_ACCESS_DENIED_ERROR:
print("Something is wrong with your user name or password")
elif err.errno == errorcode.ER_BAD_DB_ERROR:
print("Database does not exists")
else:
print(err)
else:
conn.close()
ERROR MESSAGE
2003: Can't connect to MySQL server on '%HOST%:3306' (10060 A connection attempt failed because the connected party did not properly respond after a period of time, or established connection failed because connected host has failed to respond)
Connector That I'm using
http://cdn.mysql.com/Downloads/Connector-Python/mysql-connector-python-2.0.1-py3.4.msi
Any idea what I'm doing wrong?
Check the GRANT on your MySQL database. It might be that you haven't been GRANTed permission to connect from that host with the given username and password.
The MySQL error number is 2003. I usually find it helpful to do a Google search to see if others have had the problem:
http://dev.mysql.com/doc/refman/5.0/en/access-denied.html
Is there a firewall between the client machine and the database server? I would hope that port 3306 would be blocked by default. If that's true, you'll need to set up a firewall rule to allow access from the client IP to the database IP via port 3306.

Categories

Resources