Develop Reference

Django foreign key from different app is NoneType

I have a model like this in a django 1.11.4 app which references a foreign key in different installed app (django_celery_results, listed in settings.INSTALLED_APPS):
class Worker(models.Model):
tasks = models.ForeignKey('django_celery_results.TaskResult', null = True)
I can run makemigrations happy_farm and migrate without error, but when I try to access worker.tasks it is NoneType rather than the TaskResult manager:
$ from happy_farm.models import Worker
$ type(Worker.objects.first().tasks)
NoneType
Typically I'd expect to see something like django.db.models.fields.related_descriptors.ManyRelatedManager.
I've also tried importing django_celery_results.models and referencing it directly, like:
import django_celery_results
class Worker(models.Model):
tasks = models.ForeignKey(django_celery_results.models.TaskResult, null = True)
With the same outcome.

Since my objective is to have many tasks associated with one worker, the answer is that ForeignKey relationships need to be set on the Many side of a One-to-Many relationship (i.e. on the TaskResult model rather than on the Worker).

Safely persisting django models with foreign keys

I implemented the storage of one-to-many models through a view. The model structure is as follows:
A DagRun model
Many DagRunModel, which have a FK to a DagRun
Many DagModelParam, which have a FK to a DagRunModel
So as follows, I Create all the instances first, to make sure there are no errors, and only in the very end, I persist them using save(). But this returns
django.db.utils.IntegrityError: NOT NULL constraint failed:
[28/Jul/2017 11:56:12] "POST /task/run HTTP/1.1" 500 19464
And this is the code of how I create the models and persist them in the end
def task_run(request):
dag_task = None
dag_tasks = DagTask.objects.filter(dag_id=request.POST["task"])
if len(dag_tasks) > 0:
dag_task = dag_tasks[0]
else:
raise ValueError("Task name is not a valid dag id")
dag_run = DagRun(
dag_task=dag_task,
database=request.POST["database"],
table=request.POST["table"],
label=request.POST["label"],
features=request.POST["features"],
user=request.user,
date=timezone.now()
)
dag_params = []
dag_models = []
models = json.loads(request.POST["models"])
for model in models:
dag_run_model = DagRunModel(
dag_run=dag_run,
dag_model=model["name"]
)
dag_models.append(dag_run_model)
for param in model["params"]:
dag_param = DagRunParam(
dag_run_model=dag_run_model,
dag_param=param["name"],
param_value=param["value"]
)
dag_params.append(dag_param)
dag_run.save()
for dag_model in dag_models:
dag_model.save()
for dag_param in dag_params:
dag_param.save()
If I instead decide to save them as I create them, this code works fine. So it looks like Foreign Keys can only be used once models are persisted, and this might be risked if my models fail to be created later in the hierarchy.
Is there any safer approach?

You may want to use a transaction so that you can enforce an "everything gets saved or nothing does" type of behavior. This would be the safest approach in my opinion.

There really isn't beyond finding ways to simplify the relationships between your models.
A ForeignKey relationship can't be safely established until the primary key of the related object is set. Otherwise, you could potentially kiss data integrity goodbye.
If you are deeply concerned about the possibility of failure somewhere along the chain, remember that you have access to the objects you just saved, and add error handling that deletes the entire chain of new objects on a failure and throws a useful error pointing out where the chain failed.

How to create initial revision for test objects when using django-reversion in test case

I'm creating some initial tests as I play with django-revisions. I'd like to be able to test that some of my api and view code correctly saves revisions. However, I can't get even a basic test to save a deleted version.
import reversion
from django.db import transaction
from django import test
from myapp import models
class TestRevisioning(test.TestCase):
fixtures = ['MyModel']
def testDelete(self):
object1 = models.MyModel.objects.first()
with transaction.atomic():
with reversion.create_revision():
object1.delete()
self.assertEquals(reversion.get_deleted(models.MyModel).count(), 1)
This fails when checking the length of the deleted QuerySet with:
AssertionError: 0 != 1
My hypothesis is that I need to create the initial revisions of my model (do the equivalent of ./manage.py createinitialrevisions). If this is the issue, how do I create the initial revisions in my test? If that isn't the issue, what else can I try?

So, the solution is pretty simple. I saved my object under revision control.
# imports same as question
class TestRevisioning(test.TestCase):
fixtures = ['MyModel']
def testDelete(self):
object1 = models.MyModel.objects.first()
# set up initial revision
with reversion.create_revision():
object1.save()
# continue with remainder of the test as per the question.
# ... etc.
I tried to override _fixture_setup(), but that didn't work. Another option would be to loop over the MyModel objects in the __init__(), saving them under reversion control.

'MyModel' is the name of the file with your fixtures?
If not, what you probably is missing is the data creation.
You can use fixtures (but a file, not the name of your model) or factories.
There's a whole chapter in Django documentation related to providing initial data in database for models: https://docs.djangoproject.com/en/1.7/howto/initial-data/
Hope it helps

Modify data as part of an alembic upgrade

I would like to modify some database data as part of an alembic upgrade.
I thought I could just add any code in the upgrade of my migration, but the following fails:
def upgrade():
### commands auto generated by Alembic - please adjust! ###
op.add_column('smsdelivery', sa.Column('sms_message_part_id', sa.Integer(), sa.ForeignKey('smsmessagepart.id'), nullable=True))
### end Alembic commands ###
from volunteer.models import DBSession, SmsDelivery, SmsMessagePart
for sms_delivery in DBSession.query(SmsDelivery).all():
message_part = DBSession.query(SmsMessagePart).filter(SmsMessagePart.message_id == sms_delivery.message_id).first()
if message_part is not None:
sms_delivery.sms_message_part = message_part
with the following error:
sqlalchemy.exc.UnboundExecutionError: Could not locate a bind configured on mapper Mapper|SmsDelivery|smsdelivery, SQL expression or this Session
I am not really understanding this error. How can I fix this or is doing operations like this not a possibility?

It is difficult to understand what exactly you are trying to achieve from the code excerpt your provided. But I'll try to guess. So the following answer will be based on my guess.
Line 4 - you import things (DBSession, SmsDelivery, SmsMessagePart) form your modules and then you are trying to operate with these objects like you do in your application.
The error shows that SmsDelivery is a mapper object - so it is pointing to some table. mapper objects should bind to valid sqlalchemy connection.
Which tells me that you skipped initialization of DB objects (connection and binding this connection to mapper objects) like you normally do in your application code.
DBSession looks like SQLAlchemy session object - it should have connection bind too.
Alembic already has connection ready and open - for making changes to db schema you are requesting with op.* methods.
So there should be way to get this connection.
According to Alembic manual op.get_bind() will return current Connection bind:
For full interaction with a connected database, use the “bind” available from the context:
from alembic import op
connection = op.get_bind()
So you may use this connection to run your queries into db.
PS. I would assume you wanted to perform some modifications to data in your table. You may try to formulate this modification into one update query. Alembic has special method for executing such changes - so you would not need to deal with connection.
alembic.operations.Operations.execute
execute(sql, execution_options=None)
Execute the given SQL using the current migration context.
In a SQL script context, the statement is emitted directly to the output stream. There is no return result, however, as this function is oriented towards generating a change script that can run in “offline” mode.
Parameters: sql – Any legal SQLAlchemy expression, including:
a string a sqlalchemy.sql.expression.text() construct.
a sqlalchemy.sql.expression.insert() construct.
a sqlalchemy.sql.expression.update(),
sqlalchemy.sql.expression.insert(), or
sqlalchemy.sql.expression.delete() construct. Pretty much anything
that’s “executable” as described in SQL Expression Language Tutorial.

Its worth noting that if you do this, you probably want to freeze a copy of your orm model inside the migration, like this:
class MyType(Base):
__tablename__ = 'existing_table'
__table_args__ = {'extend_existing': True}
id = Column(Integer, ...)
..
def upgrade():
Base.metadata.bind = op.get_bind()
for item in Session.query(MyType).all():
...
Otherwise you'll inevitably end up in a situation where you orm model changes and previous migrations no longer work.
Particularly note that you want to extend Base, not the base type itself (app.models.MyType) because your type might go away as some point, and once again, your migrations will fail.

You need to import Base also and then
Base.metatada.bind = op.get_bind()
and after this you can use your models like always without errors.

Problems using User model in django unit tests

I have the following django test case that is giving me errors:
class MyTesting(unittest.TestCase):
def setUp(self):
self.u1 = User.objects.create(username='user1')
self.up1 = UserProfile.objects.create(user=self.u1)
def testA(self):
...
def testB(self):
...
When I run my tests, testA will pass sucessfully but before testB starts, I get the following error:
IntegrityError: column username is not unique
It's clear that it is trying to create self.u1 before each test case and finding that it already exists in the Database. How do I get it to properly clean up after each test case so that subsequent cases run correctly?

setUp and tearDown methods on Unittests are called before and after each test case. Define tearDown method which deletes the created user.
class MyTesting(unittest.TestCase):
def setUp(self):
self.u1 = User.objects.create(username='user1')
self.up1 = UserProfile.objects.create(user=self.u1)
def testA(self):
...
def tearDown(self):
self.up1.delete()
self.u1.delete()
I would also advise to create user profiles using post_save signal unless you really want to create user profile manually for each user.
Follow-up on delete comment:
From Django docs:
When Django deletes an object, it
emulates the behavior of the SQL
constraint ON DELETE CASCADE -- in
other words, any objects which had
foreign keys pointing at the object to
be deleted will be deleted along with
it.
In your case, user profile is pointing to user so you should delete the user first to delete the profile at the same time.

If you want django to automatically flush the test database after each test is run then you should extend django.test.TestCase, NOT django.utils.unittest.TestCase (as you are doing currently).
It's good practice to dump the database after each test so you can be extra-sure you're tests are consistent, but note that your tests will run slower with this additional overhead.
See the WARNING section in the "Writing Tests" Django Docs.

Precisely, setUp exists for the very purpose of running once before each test case.
The converse method, the one that runs once after each test case, is named tearDown: that's where you delete self.u1 etc (presumably by just calling self.u1.delete(), unless you have supplementary specialized clean-up requirements in addition to just deleting the object).