Develop Reference

FastAPI db query stuck. Kubernetes

I have one small app that use fastapi. The problem is when I deploy it to my server and trying to make a post request to route which contains some database operations, it just stuck and gives me 504 error. But on my local machine it working well.
Here is how my db connecting:
app.add_event_handler("startup", tasks.create_start_app_handler(app))
app.add_event_handler("shutdown", tasks.create_stop_app_handler(app))
I tried to revert db connection from startup application to creation of this connection with closing it in different route to test and its worked. Like:
#app.get("/")
async def create_item():
engine = create_engine(
DB_URL
)
SessionLocal = sessionmaker(autocommit=False, autoflush=False, bind=engine)
Base = declarative_base()
t = engine.execute('SELECT * FROM auth_user').fetchone()
engine.dispose()
return t
How it's depend on events? Versions of postgresql are different, but I don't think it's because of it.
Currently I have deployment with 2 pods running in it. When I use psql command I can connect normally. So it only stuck in application, not it pod.

If somebody finds same, I fixed it by updating pgpoll from 4.2.2 to latest.

Django with Peewee Connection Pooling MySQL disconnect

I'm running a Django project with Peewee in Python 3.6 and trying to track down what's wrong with the connection pooling. I keep getting the following error on the development server (for some reason I never experience this issue on my local machine):
Lost connection to MySQL server during query
The repro steps are reliable and are:
Restart Apache on the instance.
Go to my Django page and press a button which triggers a DB operation.
Works fine.
Wait exactly 10 minutes (I've tested enough to get the exact number).
Press another button to trigger another DB operation.
Get the lost connection error above.
The code is structured such that I have all the DB operations inside an independent Python module which is imported into the Django module.
In the main class constructor I'm setting up the DB as such:
from playhouse.pool import PooledMySQLDatabase
def __init__(self, host, database, user, password, stale_timeout=300):
self.mysql_db = PooledMySQLDatabase(host=host, database=database, user=user, password=password, stale_timeout=stale_timeout)
db_proxy.initialize(self.mysql_db)
Every call which needs to make calls out to the DB are done like this:
def get_user_by_id(self, user_id):
db_proxy.connect(reuse_if_open=True)
user = (User.get(User.user_id == user_id))
db_proxy.close()
return {'id': user.user_id, 'first_name': user.first_name, 'last_name': user.last_name, 'email': user.email }
I looked at the wait_timeout value on the MySQL instance and its value is 3600 so that doesn't seem to be the issue (and I tried changing it anyway just to see).
Any ideas on what I could be doing wrong here?
Update:
I found that the /etc/my.cnf configuration file for MySQL has the wait-timeout value set to 600, which matches what I'm experiencing. I don't know why this value doesn't show when I runSHOW VARIABLES LIKE 'wait_timeout'; on the MySQL DB (that returns 3600) but it does seem likely the issue is coming from the wait timeout.
Given this I tried setting the stale timeout to 60, assuming that if it's less than the wait timeout it might fix the issue but it didn't make a difference.

You need to be sure you're recycling the connections properly -- that means that when a request begins you open a connection and when the response is delivered you close the connection. The pool is not recycling the conn most likely because you're never putting it back in the pool, so it looks like its still "in use". This can easily be done with middleware and is described here:
http://docs.peewee-orm.com/en/latest/peewee/database.html#django

I finally came up with a fix which works for my case, after trying numerous ideas. It's not ideal but it works. This post on Connection pooling pointed me in the right direction.
I created a Django middleware class and configured it to be the first in the list of Django middleware.
from peewee import OperationalError
from playhouse.pool import PooledMySQLDatabase
database = PooledMySQLDatabase(None)
class PeeweeConnectionMiddleware(object):
CONN_FAILURE_CODES = [ 2006, 2013, ]
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
if database.database: # Is DB initialized?
response = None
try:
database.connect(reuse_if_open=True)
with database.atomic() as transaction:
try:
response = self.get_response(request)
except:
transaction.rollback()
raise
except OperationalError as exception:
if exception.args[0] in self.CONN_FAILURE_CODES:
database.close_all()
database.connect()
response = None
with database.atomic() as transaction:
try:
response = self.get_response(request)
except:
transaction.rollback()
raise
else:
raise
finally:
if not database.is_closed():
database.close()
return response
else:
return self.get_response(request)

QueuePool limit reached for Flask application

I am running an application with Flask and Flask-SQLAlchemy.
from config import FlaskDatabaseConfig
from flask import Flask
from flask import request
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy
application = Flask(__name__)
application.config.from_object(FlaskDatabaseConfig())
db = SQLAlchemy(application)
#application.route("/queue/request", methods=["POST"])
def handle_queued_request():
stuff()
return ""
def stuff():
# Includes database queries and updates and a call to db.session.commit()
# db.session.begin() and db.session.close() are not called
pass
if __name__ == "__main__":
application.run(debug=False, port=5001)
Now, from my understanding, by using Flask-SQLAlchemy I do not need to manage sessions on my own. So why am I getting the following error if I run several requests turn by turn to my endpoint?
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/3o7r)
I've tried using db.session.close() but then, instead of this error, my database updates are not committed properly. What am I doing incorrectly? Do I need to manually close connections with the database once a request has been handled?

I have found a solution to this. The issue was that I had a lot of processes that were "idle in transaction" because I did not call db.session.commit() after making certain database SELECT statements using Query.first()
To investigate this, I queried my (development) PostgreSQL database directly using:
SELECT * FROM pg_stat_activity

Just remove connection every time you make a query session to db.
products = db.session.query(Product).limit(20).all()
db.session.remove()

How to completely remove database connection in sqlalchemy? Giving max_user_limit on python anywhere

I am hosting a flask application on Pythonanywhere. Where I have to make few queries from the database. While using MySQLdb I am able to close all the connection to database and don't get any error. But while using sqlalchemy some how connections to the database do not get closed.
This is my connection manager class which has a method defined to close the database connection.
class ConnectionManager:
def __init__(self):
self.base = declarative_base()
def get_db_session(self):
self.engine = create_engine(get_db_path())
self.base.metadata.bind = self.engine
self.session_maker = sessionmaker(bind = self.engine)
self.session = self.session_maker()
return self.session
def persist_in_db(self,record):
session = self.get_db_session()
session.add(record)
session.commit()
session.close()
def close_session(self):
self.session.close()
self.session_maker.close_all()
del self.session
del self.session_maker
del self.engine
#self.engine.dispose()
Before returning the response form the app I call close_session method.
So my question basically is where I am conceptually getting wrong and how can I completely remove database connection.

This is caused by connection pooling. You can disable connection pooling by using NullPool.
self.engine = create_engine(get_db_path(), poolclass=NullPool)
Be careful though, this may not be a good idea in a web app if each web request needs a DB connection.

Flask unit tests with SQLAlchemy and PostgreSQL exhausts db connections

I'm running a suite of fairly straightforward test cases using Flask, SQLAlchemy, and PostgreSQL. Using an application factory, I've defined a base unit test class like so:
class BaseTestCase(unittest.TestCase):
def setUp(self):
self.app = create_app()
self.app.config.from_object('app.config.test')
self.api_base = '/api/v1'
self.ctx = self.app.test_request_context()
self.ctx.push()
self.client = self.app.test_client()
db.create_all()
def tearDown(self):
db.session.remove()
db.drop_all(app=self.app)
print db.engine.pool.status()
if self.ctx is not None:
self.ctx.pop()
All goes well for a few unit tests, up until:
OperationalError: (OperationalError) FATAL: remaining connection slots are reserved for non-replication superuser connections
It seems that there is only 1 connection in the pool (db.engine.pool.status() for each test shows: Pool size: 5 Connections in pool: 1 Current Overflow: -4 Current Checked out connections: 0), but somehow the app never disconnects. Clearly a new app instance is created for each test case, which seems fine according to the documentation. If I move the app creation to the module level it works fine.
Does anyone know why this is happening?
Thanks

Add db.get_engine(self.app).dispose() after db.drop_all()