Develop Reference

Django test data loading best-practices

I'm trying to understand best-practices around Django test data loading. I have a simple API that has some very long lists of returned JSON data, and I want to write a behaviorial test to make sure it works right.
Fixtures, or something like them, seem like they should do the trick - but it looks like Django fixtures are intended for seeding the database for tests, and not providing more general sampled/serialized data - e.g. I haven't been able to find an API that lets me load data from one specific fixture.
What's the generally-accepted way for me to manually store/version a file on disk that contains data that will be loaded and self.assertEqual'ed against in my test cases?
Any advice appreciated - thank you!

you should take a look at factory_boy or django model mommy. They are preferred way to load fixtures as compared to the default fixture loading supported by django. But if you really need to use django default fixture,
class SiteTests(TestCase):
#This is the fixture:
#- fields: {content: lots of stuff, query: test, title:
test, url: 'http://google.com'}
#model: mine.site
#pk: 1
fixtures = ['mine']
where mine is the name of the fixture file. Take a look at http://django-testing-docs.readthedocs.org/en/latest/fixtures.html on how it is done.

What JSON parser does Django Fixture Loading Use

My Django unit tests are quite slow. I'm using in-memory sqlite and keeping fixtures to a minimum. However it still takes about 5 seconds to load a fixture.
I've read json fixtures are faster than yaml ones and am therefore using json.
My suspicion is that the fixtures are using a slow json library (json or simplejson instead of cjson).
My questions are these:
Can anyone confirm the fixture loading is the slow part or have an
idea on how to go about this?
Does anyone know what json library django uses to load fixtures?

Django fixtures are loaded using loaddata management command which uses built-in django deserializers which use json module from the standard library.
Fixtures are slow and difficult to maintain. Consider using model factories instead. Basically, there are two major players out there:
factory_boy
model_mommy
Also see:
On Fixtures and Factories
Factory Boy as an Alternative to Django Testing Fixtures

Python design advice

I have written a generic framework in python for some particular type of task. It is a webserver which serves different requests and operations. This framework can be used by many projects and each one has a different set of validation rules. Right now, I'm just updating my script for each project.
I'm thinking of externalizing this validation part, how do I go about this? The validations are more than mere field content validations; I'm thinking of having a config file which maps incoming request <-> validationModule something like /site1/a/b.xml=validateSite1.py and importing this module in an if condition if the request is for site1. So I'll have generic framework scripts + individual scripts for each site.
Is there a cleaner way to do this?

I think it'd be better to use Python itself as the top-level mapping from URL paths to validation modules. A configuration might look like this:
import site1
import site2
def dispatch(uri):
if uri.startswith('/site1/'):
return site1.validate(uri)
elif uri.startswith('/site2/):
return site2.validate(uri)
This simple example might tempt you to "abstract" it out into a more "generic framework" that turns strings into filenames to use as validation scripts. Here are some advantages of the doing the above instead:
Performance: site modules are imported just once, we don't look up filenames per request.
Flexibility: if you decide later that the dispatching logic is more complicated, you can easily use arbitrary Python code to deal with it. There will never be a need to extend your mapping system itself--only the config files that require more complexity.
Single language.

Fixtures with picklefield library for Django

A straight forward question:
Is there some easy way to write fixtures (i.e. in JSON format, but I don't care really) when using models that incorporate PickleFields?
EDIT:
In the end I think I'll just get rid of fixtures altogether. I'll use named *.py scripts that will create all the objects for me. I've always found fixtures quite cumbersome anyway.

Assuming you are using django-picklefield you can use dumpdata/loaddata just like you would with any other model. Tested it briefly and everything works fine.

Have different initial_data fixtures for different stages (testing v. production)

I have an initial_data fixture that I want to load everytime except for production. I already have different settings file for production and non-production deployments.
Any suggestions on how to accomplish this?
Clarification: I do not want test fixtures. Basically, I just need the fixture to be loaded based on a setting change of some sort. I'll be digging into the Django code to see if I could figure out an elegant way to accomplish this.

You can actually setup different test fixtures for each test if you want:
http://docs.djangoproject.com/en/dev/topics/testing/#topics-testing-fixtures
If you only want to load the fixtures in one time, you can also write a custom TestRunner that will allow you to do that setup at the beginning:
docs.djangoproject.com/en/dev/topics/testing/#using-different-testing-frameworks
Both of those will still load the data from the production fixtures as that is done with syncdb, but you can override the data, or even delete it all. This may not be optimal if you are loading large amounts of data into your production product. If this is the case, I would recommend you adding a custom command like load_production_data that allows you to do it quickly and easily from the command line.

The easiest way is to use manage.py testserver [fixture ...]
If this is a staging (rather than dev) deployment, though, you may not want to use django's builtin server. In that case, a quick (if hacky) way of doing what you're after is to have the fixtures in an app (called, for example, "undeployed") that is only installed in your non-production settings.