I'm making a simple Flask web application for fun and I wanted to use nosetests. I'm stuck at how to use Flask-SQLAlchemy to connect to an in-memory test database in my tests file. When I run my tests - Flask connects to my main app's database and what is more, fails to clean it up after each test. Here's my tests code:
import nose
from nose.tools import *
from pyquery import PyQuery as pq
from flask.ext.sqlalchemy import SQLAlchemy
from app import site, db
from app.models import Post
class TestApp(object):
def setUp(self):
site.config['TESTING'] = True
site.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://'
self.test_app = site.test_client()
db.create_all()
def tearDown(self):
# db.session.remove()
db.drop_all()
def test_posts_index(self):
db.session.add(Post('title', 'body'))
db.session.add(Post('title2', 'body'))
db.session.commit() # this writes to production db ie app.db file
# instead of sqlite://
rv = self.test_app.get('/posts')
d = pq(rv.data)
print len(d('h1'))
assert len(d('h1')) == 2
And here's my app/__init__.py code:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
from app import config
site = Flask(__name__)
site.config['SQLALCHEMY_DATABASE_URI'] = config.db_uri
db = SQLAlchemy(site)
site.secret_key = 'A0Zr98j/3yX R~XHH!jmN]LWX/,?RT'
from app import db_setup
db_setup.create_db()
import controllers, models
The db_setup.create_db() in app/__init__.py function looks simply like this:
from app import db
from app.models import Post
def create_db():
db.create_all()
db.session.commit()
I tried instantiating the application and database in the tests file, but then my models don't work because they from app import db, where db is the production db object. I also sprinkled a few print statements in the test case like print db and they print out something like <SQLAlchemy engine sqlite://>, but it still writes to the production db anyways.
I'd really appreciate any tips on how to get around this. Thanks!
Why don't you use something about the environment to determine whether the app starts in a testing or live mode?
if 'testing' in os.environ:
site.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite://'
else:
site.config['SQLALCHEMY_DATABASE_URI'] = config.db_uri
There are so many ways to skin this particular cat. If you don't like the idea of having if blocks littering your code you can import your settings from an entirely separate module based on whether the app is started in testing or live mode.
I was able to figure out the problem, it's related to me initiating a connection to the database in my __init__.py file, which I shouldn't do.
The culprit was the
from app import db_setup
db_setup.create_db()
code. Essentially, every time I did an from app import db, I think that app gets instantiated, it calls db_setup.create_db(), which creates the tables using the production config. From there on, despite trying to set the flask app config SQLALCHEMY_DATABASE_URI to an in memory database, the db object would continue to use the database instantiated in the __init__.py file.
All I have to do is call create_all() from the environment my will run in at that time. Hope this helps anyone how might run into something similar.
I had the same problem, but I didn't use a db.create_all() type statement in my init.py file at all.
In the end, the only way I could around the issue was to use
def setUp(self):
with app.app_context():
db.create_all()
Related
In development (so sqlite3) I'm getting this error on any database access:
sqlalchemy.exc.OperationalError: (sqlite3.OperationalError) no such table: ujs ...
I got here by saying
export FLASK_ENV=development
export FLASK_APP=my_app.py
flask db init
flask db migrate
flask db upgrade
flask run
and then doing an HTTP GET against that dev server.
I believe the migration workflow succeeded, because when I use the sqlite3 commandline client, I can see the (empty) table with a believably correct schema.
╭╴ (get-db-working *%=)╶╮
╰ jeff#starshine:TN_flask_web $ sqlite3 dev.db
SQLite version 3.27.2 2019-02-25 16:06:06
Enter ".help" for usage hints.
sqlite> .table
alembic_version ujs
sqlite> .quit
╭╴ (get-db-working *%=)╶╮
╰ jeff#starshine:TN_flask_web $
I therefore believe I've made a coding error. But I'm not seeing it.
I have this code (pared down to what I believe is the essential bits):
my_app.py:
from app import create_app, db, cli
from app.models import UJS
app = create_app()
cli.register(app)
#app.shell_context_processor
def make_shell_context():
return {'db': db,
'UJS': UJS}
app/models.py:
from app import db
import time
def now_in_microseconds():
"""Return the current time in microseconds since the epoch.
"""
return time.time() * 1000 * 1000
class UJS(db.Model):
id = db.Column(db.Integer, primary_key=True)
timestamp_microseconds = db.Column(db.BigInteger, default=now_in_microseconds)
ip_hash = db.column(db.String(40))
# And then some more columns, all quite boring.
def __repr__(self):
return '<[{tag}]/[{ip}] {microsec}/{city}>'.format(
tag=self.tag, ip=self.ip_hash,
microsec=self.timestamp_microseconds, city=self.city)
app/__init__.py:
from flask import Flask, request, current_app
from flask_sqlalchemy import SQLAlchemy
from flask_migrate import Migrate
from config import Config
db = SQLAlchemy()
migrate = Migrate()
def create_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(config_class)
try:
app.config.from_pyfile("../config_local.py")
except FileNotFoundError:
print('No local config found.')
except:
print('Unexpected error on app.config.from_pyfile()')
db.init_app(app)
migrate.init_app(app, db)
...
return app
from app import models
and app/main/routes.py:
from flask import request, g, current_app, session
from app import db
from app.main import bp
from app.models import UJS
#bp.before_app_request
def before_request():
if 'static' == request.endpoint:
# This should only happen in dev. Otherwise, nginx handles static routes directly.
return
# I expect this to return an empty list, but it throws a 500.
print(UJS.query.all())
Any suggestions what I'm missing?
For anyone who might find this question later on: the problem was about having the right absolute path to your DB in your SQLALCHEMY_DATABASE_URI config value.
Also (this wasnt the case here, but it might possibly gotcha with the same symptoms) - if you omit __tablename__ on Model declaration, SQLAlchemy might autogenerate something you wont expect. Just a thing to keep in mind, if you're working with an existing DB with some schema already in place.
I want to add a new table or add data by calling db, but i got some problem when i try to import db
db return like this <SQLAlchemy engine=None>
which mean i didnt already doing this db.init_app(app)
this is my file struckture
Root
run.py
------>server/__init__.py
Config.py
import os
class Config:
SECRET_KEY = os.environ.get('SECRET_KEY')
SQLALCHEMY_DATABASE_URI = 'mysql+pymysql://root:#localhost/flask_py'
MAIL_SERVER = 'smtp.googlemail.com'
MAIL_PORT = 587
MAIL_USE_TLS = True
MAIL_USERNAME = os.environ.get('EMAIL_USER')
MAIL_PASSWORD = os.environ.get('EMAIL_PASS')
Run.py
from server import server_app, db
app = server_app()
if __name__ == '__main__':
app.run(debug=True)
__init__.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
from server.config import Config
db = SQLAlchemy()
def server_app(config_class=Config):
app = Flask(__name__)
app.config.from_object(Config)
db.init_app(app)
from server.users.routes import users
app.register_blueprint(users)
return app
in my command line using windows i want to import db, i try like this :
D:\PYTHON\root>python
from run import db
db.create_all()
but when i check is :
<SQLAlchemy engine=None>
I have a full working solution here very similar to what you're doing. Check my layout and then look at the create_db.py.
https://github.com/researcher2/stackoverflow_56885380
It appears in your case the "server_app" is not being executed in your interactive shell. Assuming you are running interactive shell in root directory, you would want to do the following:
from server import db, create_app
app = server_app()
with app.app_context():
db.create_all()
The annoying thing about flask-sqlalchemy as opposed to plain sqlalchemy is the db is coupled to a flask app. The db config comes from the flask config and the initiation is done during db_init or just SqlAlchemy(db) if you want the simpler version for single app setup.
Your models would also need to be setup properly. In my example above I just had them in the create_db script.
This post may help you as well regarding factories and blueprints. I created the above github to answer it.
Reflecting different databases in Flask factory setup
I'm writing a Flask app and I instantiate a mongo database in main.py.
I've got a submodule called user.py that holds class User. main.py takes login credentials and sends it to the class User which handles the rest.
How can I cleanly pass my mongo instance to the User class? I've tried static variables in a config.py file but they don't work because the variables are always None when user.py tries to use them.
Right now I'm resorting to passing in mongo as a parameter, but this seems like a dirty way to do it considering there will be many modules. Here's my code;
# Setup app and database
app = Flask(__name__)
app.config['MONGO_URI'] = 'mongodb://'
mongo = PyMongo(app)
You can import mongo directly into your user.py module.
To avoid circular import error, you just have to move the imports to the bottom of the file. As long as you import User into main after app is defined, it should resolve the circular import:
user.py
from .main import mongo
# class User():
# ... Your Code
main.py
from flask import Flask
from flask_pymongo import PyMongo
app = Flask(__name__)
mongo = PyMongo(app)
from .user import User
Moving imports to the bottom are normally not a good idea, but in Flask is quite common. Below is an example of a similar scenario from the official flask documentation:
# app.py
from flask import Flask
app = Flask(__name__)
import app.views
# views.py
from app import app
#app.route('/')
def index():
return 'Hello World!'
http://flask-.readthedocs.io/en/0.6/patterns/packages/#simple-packages
Do the importing at the bottom of the file [...]
Every Python programmer hates them, and yet we just added some: circular imports (That’s when two modules depend on each other. In this case views.py depends on init.py). Be advised that this is a bad idea in general but here it is actually fine. The reason for this is that we are not actually using the views in init.py and just ensuring the module is imported and we are doing that at the bottom of the file.
I have a project and I've defined my db.py module as:
app = get_global_flask_app()
app.config['SQLALCHEMY_DATABASE_URI'] = "postgresql://foo:bar#127.0.0.1:5432/test"
app.config["SQLALCHEMY_TRACK_MODIFICATIONS"] = False
db = SQLAlchemy(app)
db.create_all()
Then I import db.db into modules that need to query the database and insert data (db.session.query()).
However, this means that when I write test code (pytest) to test any module that imports db.py, I will need to define "SQLALCHEMY_DATABASE_URI". One solution is to have db be a lazy attribute so that the code above is executed in tests only if the database is used/tested. Is there a common design pattern for Flask() + SQLA + SQLALCHEMY_DATABASE_URI out there that I'm missing? How would you solve this problem? Flask-config?
The way we normally solve this problem is with an application factory and a config.
This means you have a function somewhere in your project that looks something like this (taken from the documentation with modifications):
def create_app(config_filename, settings_override=None):
app = Flask(__name__)
app.config.from_pyfile(config_filename)
app.config.from_object(settings_override)
from yourapplication.model import db
db.init_app(app)
from yourapplication.views.admin import admin
from yourapplication.views.frontend import frontend
app.register_blueprint(admin)
app.register_blueprint(frontend)
return app
Then (and hopefully you're using pytest) in your root test directory, you have a conftest file which automatically prepares your test environment something like this:
import pytest
from your_project import create_app
class TestConfig:
SQLALCHEMY_DATABASE_URI = "postgresql://foo:bar#127.0.0.1:5432/test"
SQLALCHEMY_TRACK_MODIFICATIONS = False
ANY OTHER SETTINGS...
#pytest.fixture(autouse=True)
def app(request):
app = create_app(settings_override=TestConfig)
ctx = app.app_context()
ctx.push()
def teardown():
ctx.pop()
request.addfinalizer(teardown)
return app
Typically, we create another fixture which is also autouse=True that handles that DB set-up, flush, and possibly loading fixtures, and only use that fixture in tests which need to access the DB (integration or functional tests), which means simply that we include in a conftest file in the same directory as our integration tests.
I am following the Flask SQLalchemy Quickstart which has all of the code in a single file:
Here is my initial index.py:
from flask import Flask
from flask.ext.sqlalchemy import SQLAlchemy
app = Flask(__name__)
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
# [snip] - some classes related to SQLAlchemy are here
if __name__ == '__main__':
app.run(host='0.0.0.0')
I want to split the code up a bit, so I created a separate file called database.py which will contain all of the database related code, and be imported as a module.
I modified my index.py to look like this:
from flask import Flask
# Import my database module
import database
app = Flask(__name__)
if __name__ == '__main__':
app.run(host='0.0.0.0')
And the file database.py:
from flask.ext.sqlalchemy import SQLAlchemy
app.config['SQLALCHEMY_DATABASE_URI]'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
# [snip] - some classes related to SQLAlchemy are here
Obviously when I try to run this code I get the following error:
File "database.py", line 5, in <module>
app.config['SQLALCHEMY_DATABASE_URI]'] = 'sqlite:////tmp/test.db'
NameError: name 'app' is not defined
I can see that this is because the app object only exists within the parent module.
I could put the following lines back into index.py:
app.config['SQLALCHEMY_DATABASE_URI'] = 'sqlite:////tmp/test.db'
db = SQLAlchemy(app)
But this creates a similar problem, whereby db is not available within the database.py file.
What is the correct way to code this?
You can import the object app into database.py by putting:
from index import app
in database.py.
Edited answer after comment:
Simply use
from index import app
in database.py.
Alternatively, with the
import index
statement, use index.app instead of app only.
This should help you get out of python's import hell.
Btw: Not sure which IDE you are using. I like pycharm a lot. Using it you can refactor code and issues such as above are prevented automagically.