Develop Reference

how to load pre-existing data flask-sqlalchemy

I am writing a REST API using flask_restful and managing the mysql db using flask-sqlalchemy. I would like to know what the best practice for loading existing data into a table when the app starts is.
I am currently calling the db.create_all() method withing an endpoint with the #app.before_first_request decorator. I would like to then fill in one of the tables created with existing data from a csv file. Should the code to push the data in a separate script or within the function?
Thanks!

I would separate loading initial database data from application initialization, because probably initial data from my experience would not be changed often and can take some time if file is bigger, and usually you don't need to reload it in the database each time application loads.
I think you will most certainly need database migrations at some point in your application development, so I would suggest setting up Flask-Migrate to handle that, and running its upgrade method on application creation (create_app method if you are using Flask application factories pattern) which will handle database migrations. I am saying this since it will save you some headache when you are introducing it later on database already populated with actual data which is initialized with db.create_all().
And for populating database with seed data I would go with Flask CLI or Flask-Script. In one of my recent projects I used Flask-Script for this, and created separate manage.py file which amongst other application management methods contained initial data seeding method which looked something like this:
#manager.command
def seed():
"Load initial data into database."
db.session.add(...)
db.session.commit()
And it was run on demand by following command:
python manage.py seed

Stuck in a django south migration - TransactionManagement error

I am having a trouble when applying a django south migration:
As always, I executed the migrate command after a successful schemamigration
python manage.py migrate webapp
The log console:
Running migrations for webapp:
- Migrating forwards to 0020_auto__add_example.
> webapp:0020_auto__add_example
TransactionManagementError: Transaction managed block ended with pending COMMIT/ROLLBACK
The error is not related with the specific migration as if I move backwards and try another it shows the same message.
Edit. This is the log of the query:
(0.005) SELECT `south_migrationhistory`.`id`, `south_migrationhistory`.`app_name`, `south_migrationhistory`.`migration`, `south_migrationhistory`.`applied` FROM `south_migrationhistory` WHERE `south_migrationhistory`.`applied` IS NOT NULL ORDER BY `south_migrationhistory`.`applied` ASC; args=()
Running migrations for webapp:
- Migrating forwards to 0020_auto__add_example.
> webapp:0020_auto__add_example
(0.002) CREATE TABLE ROLLBACK_TEST (X INT); args=()
TransactionManagementError: Transaction managed block ended with pending COMMIT/ROLLBACK

I just ran into a similar issue.
MySQL 5.6.13 (on Amazon RDS)
Django==1.5.4
MySQL-python==1.2.4
South==0.8.2
I went through almost every possible fix here and through countless Google searches with zero luck.
I looked at the database schema and a table I had not created named 'ROLLBACK_TEST' was part of the schema.
Once I dropped that mystery table the migration ran flawlessly.
This table could only have originated via Django, South or possibly an internal process at Amazon as nothing else has access.

I had the same problem with Django 1.6 and South 1.0 on a MySQL instance. After turning on the django.db.backends logger I realised the migration was stuck on the following SQL statement:
DEBUG (0.003) CREATE TABLE ROLLBACK_TEST (X INT); args=None
So I checked the database and sure enough found the ROLLBACK_TEST table. Deleting it resolved the problem:
$ manage.py dbshell
mysql> DROP TABLE ROLLBACK_TEST;

I had the same problem and was banging my head for a while.
It turns out my (MySQL) database user didn't have sufficient privileges.
I assigned: ALTER, CREATE, DELETE, DROP, INDEX, INSERT, SELECT, UPDATE to the user and everything worked fine.

I had the same problem and for me the solution was simply to give the proper rights of my sqlite development.db file to the user who was executing the python manage.py migrate webapp command. I had the file owned by www-data and hence couldn't work on the file.

I am writing the answer to the problem I had as it can be useful for somebody.
After some time of debugging I found that the problem was not related with django. It was an issue with the database and the virtual machine that hosts it.
I restarted the database machine and the migrations are now working.

When I came to the same issue my problem was more or less related to django. I explain.
I was working with different tabs in my console. One was used with a django shell to test my models and in another tab I run the migrations. I came to an integrity error in my shell tab. So, until I solved the problem (see this thread) or closed the tab, the error in migration tab persisted. As the former answer pointed out, this was something related to the DB -but not DB's fault.

How to call methods of Model Class in Interactive Console locally?

I have been searching this since yesterday and still nothing. From all that researching this is my understanding so far.
You can access your datastore remotely with remote_api_shell.py
Make sure your path is set correctly in your Environment variable.
And according to my understanding the remote datastore that they were talking about is the datastore in appspot.com and not the local one. I don't want to deploy my app right now and hence I want to just work locally, for now atleast.
I created a model named Usersdb in my app. As someone coming from PHP, MYSQL background I thought GQL would have console environment for us to test the queries. But after some googling I found out that you can manipulate local datastore from the interactive console that's in
http://localhost:8081/_ah/admin/interactive
From the post Google App Engine GQL query on localhost I got the idea of performing GqlQuery in the interactive console while in localhost which goes something like this:-
from google.appengine.ext import db
q = db.GqlQuery("SELECT * FROM Userdb where username = 'random_user'")
print q.get().username
But what I really wanted to do was perform method calls like get_by_id() and get_by_key_name() and such in my interactive console without having to test on my app. Like:-
print Userdb.get_by_id(12)
How can I get those running? Do I have to import my python file to the interactive console? I tried doing that too but it crashed app engine. I'm just starting out on app engine. Forgive me if this is a completely stupid question.

You should import the model class that you wrote into your session in the interactive console. For example, if you have a file named model.py in your application, which contains your Userdb class you could write the following in the interactive console:
import model
print model.Userdb.get_by_id(12)

Programmatically sync the db in Django

I'm trying to sync my db from a view, something like this:
from django import http
from django.core import management
def syncdb(request):
management.call_command('syncdb')
return http.HttpResponse('Database synced.')
The issue is, it will block the dev server by asking for user input from the terminal. How can I pass it the '--noinput' option to prevent asking me anything?
I have other ways of marking users as super-user, so there's no need for the user input, but I really need to call syncdb (and flush) programmatically, without logging on to the server via ssh. Any help is appreciated.

management.call_command('syncdb', interactive=False)

Works like this (at least with Django 1.1.):
from django.core.management.commands import syncdb
syncdb.Command().execute(noinput=True)

Start a "throwaway" MySQL session for testing code?

If I want to be able to test my application against a empty MySQL database each time my application's testsuite is run, how can I start up a server as a non-root user which refers to a empty (not saved anywhere, or in saved to /tmp) MySQL database?
My application is in Python, and I'm using unittest on Ubuntu 9.10.

--datadir for just the data or --basedir

You can try the Blackhole and Memory table types in MySQL.