Develop Reference

make flask manage.py multithreaded

I'm using python, flask and and manage.py to create a backend-website with different routes, like /api/foo and /api/bar.
When accessing /api/foo the code should request data from /api/bar and process those data. But this (post-) request hangs endless, probably because the flask server is single threaded.
Currently the web-endpoint for /api/foo and /api/bar is started that way:
from flask_script import Manager
app = create_app(os.getenv('FLASK_CONFIG') or 'default')
manager = Manager(app)
if __name__ == '__main__':
manager.run()
So no app.run() is needed. This means, it's not possible to call something like app.run(threaded=True). But manager.run() does not support the threaded=True parameter.
How can I made the manage.py multi threaded?
btw: This is currently for development only. In production this app will use gunicorn or uWSGI later.

Flask: db is not being referenced properly when importing into my database seeder script

# project\__init__.py
from flask import Flask
from flask_mysqldb import MySQL
from .config import app_config
db = MySQL()
def create_app(config_name):
app = Flask(__name__,
instance_path=os.path.join(os.path.dirname(__file__), 'instance'),
instance_relative_config=True)
app.config.from_object(app_config[config_name])
db.init_app(app)
print(db) # prints <flask_mysqldb.MySQL object at 0x000002A13710FC10>
# project/main.py
from . import db
#main.route('/foobar')
def foobar():
print(db) # prints <flask_mysqldb.MySQL object at 0x000002A13710FC10>
# project/database/seed_shipment.py
from project import create_app, db
def foo():
.
.
.
def goo()
.
.
.
if __name__ == '__main__':
config_name = os.getenv('FLASK_ENV')
app = create_app(config_name)
cursor = db.connection.cursor()
print(db) # prints <flask_mysqldb.MySQL object at 0x000002056B4EFD60>
print(db.connection) # returns None
for x in range(20):
# code which generates dummy data using foo() and goo()
cursor.execute("INSERT INTO shipment (column1, column2) VALUES (%s)", (var1, var2))
db.connection.commit()
My database connection works fine when I host the app and carry out CRUD operations using the interface in my browser. Such as login, sign up, create a shipment.
Note that I am not executing flask run from my terminal but instead python -m project.run, here is the code of this script:
# project/run.py
import os
from . import create_app
config_name = os.getenv('FLASK_ENV')
app = create_app(config_name)
if __name__ == '__main__':
app.run()
However, when I run python -m project.database.seed_shipment db seems to reference a different MySQL instance which has no connection. See the print results in the comments in my code.

My database connection works fine when I host the app and carry out CRUD operations using the interface in my browser. Such as login, sign up, create a shipment.
This indicates to me that you are correctly connecting to your db for each request.
However, when I run python -m project.database.seed_shipment db seems to reference a different MySQL instance which has no connection. See the print results in the comments in my code.
This, on the other hand, indicates to me that you are not connecting to your db when you directly execute seed_shipment - in fact, you seem to reference something different (as you write). In other words, db = MySQL() is not called.
I recommend that you try the following.
# project\__init__.py
from flask import Flask, g
from flask_mysqldb import MySQL
from .config import app_config
def create_app(config_name):
app = Flask(__name__,
instance_path=os.path.join(os.path.dirname(__file__), 'instance'),
instance_relative_config=True)
app.config.from_object(app_config[config_name])
# this here is just to register the teardown context and some variables ....
MySQL(app)
return app
def get_db():
if 'db' not in g:
g.db = MySQL().connection.cursor()
return g.db
That way, MySQL(app) is always executed when you call create_app. Now, you should be able to simply call get_db each time you want your db.

How to emit data from background process to Flask-SocketIO

emitted data from different process to socketio but not working
I created Flask App in which I am using Flask-SocketIO framework. The code for flask app is below:
from web import create_app, socketio
app = create_app()
if __name__ == '__main__':
socketio.run()
I am running this using flask run command.
But I have another python script in which I am importing socketio and want to emit data to client's browser.
# cli-script.py
import time
from web import socketio
def demo():
while 1:
socketio.emit('my-event', ("My Data"))
time.sleep(10)
demo()
My flask application folder structure looks like this:
/-
web
__init__.py
code.py
web-script.py
cli-script.py
and I am running two python processes:
flask run
python cli-script.py
Why this doesn't work ?

QueuePool limit reached for Flask application

I am running an application with Flask and Flask-SQLAlchemy.
from config import FlaskDatabaseConfig
from flask import Flask
from flask import request
from flask_migrate import Migrate
from flask_sqlalchemy import SQLAlchemy
application = Flask(__name__)
application.config.from_object(FlaskDatabaseConfig())
db = SQLAlchemy(application)
#application.route("/queue/request", methods=["POST"])
def handle_queued_request():
stuff()
return ""
def stuff():
# Includes database queries and updates and a call to db.session.commit()
# db.session.begin() and db.session.close() are not called
pass
if __name__ == "__main__":
application.run(debug=False, port=5001)
Now, from my understanding, by using Flask-SQLAlchemy I do not need to manage sessions on my own. So why am I getting the following error if I run several requests turn by turn to my endpoint?
sqlalchemy.exc.TimeoutError: QueuePool limit of size 5 overflow 10 reached, connection timed out, timeout 30 (Background on this error at: http://sqlalche.me/e/3o7r)
I've tried using db.session.close() but then, instead of this error, my database updates are not committed properly. What am I doing incorrectly? Do I need to manually close connections with the database once a request has been handled?

I have found a solution to this. The issue was that I had a lot of processes that were "idle in transaction" because I did not call db.session.commit() after making certain database SELECT statements using Query.first()
To investigate this, I queried my (development) PostgreSQL database directly using:
SELECT * FROM pg_stat_activity

Just remove connection every time you make a query session to db.
products = db.session.query(Product).limit(20).all()
db.session.remove()

Can't connect to mongolab through flask pymongo on heroku

The code below works on localhost. I get the message '50' which is the number of rows I have. But it doesn't work when I push it to heroku. I get the "Application error message" and the heroku logs just say app crashed.
And yes, the heroku app has the mongolab add-on connected.
What am I doing incorrectly?
import os
from flask import Flask
from flask.ext.pymongo import PyMongo
app = Flask(__name__)
app.debug = True
app.config['MONGO_URI'] = os.environ['MONGOLAB_URI']
mongo = PyMongo(app, config_prefix='MONGO')
#app.route("/")
def hello():
num = mongo.db.test.count()
return '%s' % num
if __name__ == "__main__":
app.run()

On heroku (unless you've changed it) mongolab stores its URI at MONGOLAB_URI (per these docs).
The crash is probably PyMongo saying it can't connect to NULL, but you can check that with heroku logs on the cli.

Check that you PyMongo version is compatible with Mongo 3.0. MongoLab has switched to it recently: http://blog.mongolab.com/2015/07/mongodb-version-3-0-now-ga-on-mongolab/