Develop Reference

Using a model's sub-classes as choice options for that model raises NameError

We are trying to upgrade a legacy code's django version from 1.8 to 1.9. We have one model that is defined like this:
def _get_descendant_question_classes():
stack = [Question]
while stack:
cls = stack.pop()
stack.extend(cls.__subclasses__())
yield cls
def _get_question_choices():
question_classes = _get_descendant_question_classes()
for cls in question_classes:
yield (cls.slug, cls._meta.verbose_name)
class Question(models.Model):
slug = "Question"
type = models.CharField(max_length=10, choices=_get_question_choices(), default=slug)
class TextQuestion(Question):
slug = "TextQuestion"
class SelectQuestion(Question):
slug = "SelectQuestion"
...
Basically the model wants to use its sub-classes as choice options for one of its fields. It does this with traversing the model in a DFS manner and yielding all the sub-classes.
This code works in django 1.8 but in django 1.9 it gives this error:
Traceback (most recent call last):
File "./manage.py", line 16, in <module>
execute_from_command_line(sys.argv)
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 350, in execute_from_command_line
utility.execute()
File "/usr/local/lib/python2.7/site-packages/django/core/management/__init__.py", line 324, in execute
django.setup()
File "/usr/local/lib/python2.7/site-packages/django/__init__.py", line 18, in setup
apps.populate(settings.INSTALLED_APPS)
File "/usr/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models(all_models)
File "/usr/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/usr/local/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/home/saeed/saeed/survey/models.py", line 85, in <module>
class Question(models.Model):
File "/home/saeed/saeed/survey/models.py", line 99, in Question
type = models.CharField(max_length=10, choices=_get_question_choices(), default=slug)
File "/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 1072, in __init__
super(CharField, self).__init__(*args, **kwargs)
File "/usr/local/lib/python2.7/site-packages/django/db/models/fields/__init__.py", line 161, in __init__
choices = list(choices)
File "/home/saeed/saeed/survey/models.py", line 65, in _get_question_choices
for cls in question_classes:
File "/home/saeed/saeed/survey/models.py", line 54, in _get_descendant_question_classes
stack = [Question]
NameError: global name 'Question' is not defined
I understand the problem what I don't understand is how this works in django 1.8? What has changed in django 1.9 that causes this? What is the best way to fix this?

I can reproduce this; with Django 1.8, python3 -m manage check succeeds, while in Django 1.9 it raises a NameError.
There is nothing particular in the Django 1.9 release notes that addresses this change in behaviour.
I would explain this Django 1.8 behaviour by noting that Django is notorious for doing “magic” on the code in order to allow referencing not-yet-executed parts of the code to define models. It is an anomaly from normal Python behaviour, and AFAICT an undocumented one.
So this would be a behaviour that was only accidental and undocumented (therefore should not be relied on), where in normal Python you would expect a NameError when referencing Question before it is defined.
Django 1.9 evidently made a change that reverts to the expected Python behaviour :-)

The exception is due to, when the method _get_descendant_question_classes is defined, the class Question is not defined yet. So, you are referring something that doesn't exist.
You have a design issue here, note you have a cyclic dependency: _get_descendant_question_classes depend on Question and Question depend on _get_descendant_question_classes.
A quick fix could be to use get_model:
def _get_descendant_question_classes():
stack = [get_model('yourappnamehere', 'Question')]
I'm not sure why you need that type field, but I'm confident that you can solve your original problem (what lead you to add that type field) in another simpler way.
Also, in case you need to know the "type" of a Question instance, you just need to check if the object has the attr textquestion_ptr or selectquestion_ptr or just use isinstance

The problem is that, when the class variable typeis initialized in the Question class, the Question class hasn't been created, and yet _get_question_choices(), which references the Question class, gets evaluated right away in order to assign a value to the type's choices, resulting in a circular reference.
To avoid this issue, instead of evaluating _get_question_choices() right away during class declaration, you can initialize type with empty choices first, and then give it the preferred value in Question's __init__ method, which is called only after Question has been instantiated:
class Question(models.Model):
type = models.CharField(max_length=10, choices=(,), default=slug)
def __init__(self, *args, **kwargs):
super(Question, self).__init__(*args, **kwargs)
self._meta.get_field('type').choices = _get_question_choices()

Exception when migrating custom User in Django

I need to create a custom User for my app and followed exactly the example from the documentation with the AUTH_USER_MODEL = 'core.MyUser' in my settings.py. However, when I make a new database, delete all the migrations folders and run the python manage.py migrate again, it gives me the exception like this
File "manage.py", line 22, in <module>
execute_from_command_line(sys.argv)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/core/management/__init__.py", line 367, in execute_from_command_line
utility.execute()
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/core/management/__init__.py", line 359, in execute
self.fetch_command(subcommand).run_from_argv(self.argv)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/core/management/base.py", line 294, in run_from_argv
self.execute(*args, **cmd_options)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/core/management/base.py", line 345, in execute
output = self.handle(*args, **options)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/core/management/commands/makemigrations.py", line 173, in handle
migration_name=self.migration_name,
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/db/migrations/autodetector.py", line 47, in changes
changes = self._detect_changes(convert_apps, graph)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/db/migrations/autodetector.py", line 132, in _detect_changes
self.old_apps = self.from_state.concrete_apps
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/db/migrations/state.py", line 180, in concrete_apps
self.apps = StateApps(self.real_apps, self.models, ignore_swappable=True)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/db/migrations/state.py", line 242, in __init__
self.render_multiple(list(models.values()) + self.real_models)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/db/migrations/state.py", line 277, in render_multiple
model.render(self)
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/db/migrations/state.py", line 559, in render
body,
File "/Users/bubuzzz/Projects/python/apps/lib/python2.7/site-packages/django/db/models/base.py", line 153, in __new__
raise TypeError("%s cannot proxy the swapped model '%s'." % (name, base_meta.swapped))
TypeError: Customer cannot proxy the swapped model 'core.MyUser'.
I am not sure why there is a migrations script for the customer there, since in my app, I used to have the Customer model as well, though I deleted it already.
Then, I created a new django project to test and try to run the migration. Surprisingly, I also see those customer migration steps, but it run successfully.
Applying auth.0008_alter_user_username_max_length... OK
Applying auth.0009_customer... OK
Applying auth.0010_delete_customer... OK
Applying sessions.0001_initial... OK
In short, how can I create the custom User in Django 1.10 ? Example code can be viewed in here https://github.com/bubuzzz/django-customer-swap-exception

You should not delete your migration folder. If you do that, django won't make migrations for you. Create migrations folder in your core app, create an empty __init__.py file inside it, remove your db.sqlite3 file, run ./manage.py makemigrations, and then migrate should work perfectly.

Mehdi Pourfar's answer is correct.If you want to know more details
By running makemigrations, you’re telling Django that you’ve made some changes to your models (in this case, you’ve made new ones) and that you’d like the changes to be stored as a migration.
Migrations are how Django stores changes to your models (and thus your database schema) - they’re just files on disk. You can read the migration for your new model if you like; it’s the file polls/migrations/0001_initial.py. Don’t worry, you’re not expected to read them every time Django makes one, but they’re designed to be human-editable in case you want to manually tweak how Django changes things.
Tell django which app you want to make migrations all fix your problem.It will auto create a folder call migrations keep your model's record.
python manage.py makemigrations core

Web2py DAL has no attribute?

My web2py application returned me an error today, which is quite odd.
Traceback (most recent call last):
File "/var/www/web2py/gluon/restricted.py", line 212, in restricted
exec ccode in environment
File "/var/www/web2py/applications/1MedCloud/controllers/default.py", line 475, in <module>
File "/var/www/web2py/gluon/globals.py", line 194, in <lambda>
self._caller = lambda f: f()
File "/var/www/web2py/applications/1MedCloud/controllers/default.py", line 63, in patient_register
rows = db(db.patientaccount.email==email).select()
File "/var/www/web2py/gluon/dal.py", line 7837, in __getattr__
return ogetattr(self, key)
AttributeError: 'DAL' object has no attribute 'patientaccount'
I am using Mysql as the database, and the table 'patientaccount' does exist. There is no connection issue as I can create tables but not fetch them from the server.
I have been using the very same code to do the db thing, here is my code
db = DAL('mysql://###:$$$#^^^^^^:3306/account_info', pool_size=0)
rows = db(db.patientaccount.email==email).select()
I did not change any code in my default.py file, but accidentally deleted some files inside "database" folder in my application. But I doubt if that could result the error, since the module is fetching tables on the server rather than using local files.
Please help! Thanks in advance!

The DAL does not inspect the MySQL database to discover its tables and fields. You must define the data models explicitly. So, somewhere in your code, you must do:
db.define_table('patientaccount',
Field('email'),
...)
That will define the db.patientaccount table so the DAL knows it exists and what fields it includes.

Testing Django database manipulation scripts with test database

I am attempting to setup the django testing environment to test a script we use to load data into our data visualization web app. It is "external" to the actual django app, so it doesn't seem appropriate or clean to use the manage.py test facility.
I have followed the instructions found in the django docs to write my own test module, but when I reach a statement that actually tries to access the database I get an error saying that a table does not exist.
Is there something else I need to do to make sure that the test database is created with all of the needed tables?
For reference, here is the code from my test case:
import os
import sys
sys.path.append(os.curdir)
os.environ['DJANGO_SETTINGS_MODULE'] = 'visualization_app.settings'
from django.utils import unittest
from django.test.utils import setup_test_environment
from topic_modeling import settings
# The module to be tested
import analysis_import
setup_test_environment()
class CreateAnalysisTestCase(unittest.TestCase):
def test_analysis_creation(self):
self.assertTrue(analysis_import.create_analysis('a_name', 'a description'))
if __name__ == '__main__':
unittest.main()
The error I get looks like this:
ERROR: test_analysis_creation (__main__.CreateAnalysisTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
File "import_scripts/tests.py", line 20, in test_analysis_creation
self.assertTrue(analysis_import.create_analysis('a_name', 'a description'))
File "/home/dan/programmingProjects/topical_guide/import_scripts/analysis_import.py", line 117, in create_analysis
Analysis.objects.get(name=name, dataset=dataset)
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/models/manager.py", line 132, in get
return self.get_query_set().get(*args, **kwargs)
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/models/query.py", line 344, in get
num = len(clone)
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/models/query.py", line 82, in __len__
self._result_cache = list(self.iterator())
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/models/query.py", line 273, in iterator
for row in compiler.results_iter():
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/models/sql/compiler.py", line 680, in results_iter
for rows in self.execute_sql(MULTI):
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/models/sql/compiler.py", line 735, in execute_sql
cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/backends/util.py", line 34, in execute
return self.cursor.execute(sql, params)
File "/usr/local/lib/python2.7/dist-packages/Django-1.3-py2.7.egg/django/db/backends/sqlite3/base.py", line 234, in execute
return Database.Cursor.execute(self, query, params)
DatabaseError: no such table: visualize_analysis

Strictly speaking, test databases are intended to be created, used to run unit tests and then destroyed. They're not really a place to "test" a data loading script. Why not just use your standard dev database? There shouldn't be anything there that you can't get back, have a backup for, etc., and it's pretty much designed for this sort of destructive behavior.

As far as I can see, the module you are testing still uses django's ORM system to read the database, so there is anything wrong about using manage.py to run your test. You can use
./manage.py test analysis_import
to only run test codes which reside in analysis_import module. In this way, you can still run only the tests for this module, but avoid the trouble of writing your own test module.
Then let us know if you still have the same problem.

changing django default model settings

I'm just starting with the django creating your own app tutorial (creating a Poll) I'm deviating slightly as I'm wanting to create an app using my own database model that already exists.
And in the tutorial it says
Table names are automatically
generated by combining the name of
the app (polls) and the lowercase
name of the model -- poll and choice.
(You can override this behavior.)
Primary keys (IDs) are added
automatically. (You can override
this, too.)
By convention, Django appends
"_id" to the foreign key field
name. Yes, you can override this,
as well.
But I can't see where it mentions how you can override this behaviour? I've defined my model as so
from django.db import models
# Create your models here.
class Channels(models.Model):
channelid = models.IntegerField()
channelid.primary_key = True
channelname = models.CharField(max_length=50)
Now when I go in to the shell this is what I get
>>> from tvlistings.models import *
>>> Channels.objects.all()
Traceback (most recent call last):
File "<console>", line 1, in <module>
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 67, in __
repr__
data = list(self[:REPR_OUTPUT_SIZE + 1])
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 82, in __
len__
self._result_cache.extend(list(self._iter))
File "C:\Python26\lib\site-packages\django\db\models\query.py", line 271, in i
terator
for row in compiler.results_iter():
File "C:\Python26\lib\site-packages\django\db\models\sql\compiler.py", line 67
7, in results_iter
for rows in self.execute_sql(MULTI):
File "C:\Python26\lib\site-packages\django\db\models\sql\compiler.py", line 73
2, in execute_sql
cursor.execute(sql, params)
File "C:\Python26\lib\site-packages\django\db\backends\util.py", line 15, in e
xecute
return self.cursor.execute(sql, params)
File "C:\Python26\lib\site-packages\django\db\backends\mysql\base.py", line 86
, in execute
return self.cursor.execute(query, args)
File "C:\Python26\lib\site-packages\MySQLdb\cursors.py", line 166, in execute
self.errorhandler(self, exc, value)
File "C:\Python26\lib\site-packages\MySQLdb\connections.py", line 35, in defau
lterrorhandler
raise errorclass, errorvalue
DatabaseError: (1146, "Table 'tvlistings.tvlistings_channels' doesn't exist")
Obviously it can't find the table tvlistings_channels as it's actually called channels. So how do I change the default?

You can override Model behavior in Django largely through the use of an inner Meta class
db_table allows you to rename the table name
specifying another field as the primary key will have it use that rather than a surrogate key (not in the Meta class, just in the model itself)

You should try and work your way all through the tutorial before you try and customise things. All these things are covered in the actual documentation, but it's best to have a basic understanding of things first before diving into that.
FWIW, here are the docs on defining your own primary key and specifying a table name. But really, do the tutorial as written first.