Develop Reference

Flask-SQL Alchemy how to union two different models from different tables

Hello I'm new to flask and I have an application where i am creating different models and schemas for my entities. These two models and schemas are very close to each other except with few differences. I have base classes for my model and schema so i could inherit and re-use the same class. However, i'm having a problem when i need to deserialize them with marshall and return the union result.
I'm using marshmallow,sql-achemy and flaskapi-spec. I am not sure if there's a way to use the marshall_with decorator with multiple schemas since I want to union my results and return the aggregated model.
Here is the endpoint,models and classes I have.
Models;
class BasePublisher(Model):
__abstract__= True
id= Column(db.String(80),primary_key=True,nullable=False)
date = Column(db.DateTime, default=dt.datetime.utcnow, primary_key=True, nullable=False)
views = Column(db.Numeric)
clicks = Column(db.Numeric)
publisher = Column(db.String(80),primary_key=True,nullable=False)
class Facebook(BasePublisher):
__tablename__='facebook_table'
def __init__(self, **kwargs):
db.Model.__init__(self, **kwargs)
class Pinterest(BasePublisher):
__tablename__='pin_table'
def __init__(self, user, **kwargs):
db.Model.__init__(self, user=user, **kwargs)
Schemas
class PublisherSchema(Schema):
date = fields.DateTime(dump_only=True)
type = fields.DateTime(dump_only=True)
views = fields.Number(dump_only=True)
clicks = fields.Number(dump_only=True)
publisher = fields.Str(dump_only=True)
class FacebookSchema(PublisherSchema):
#post_dump
def dump_data(self,data):
data["type"]="Facebok"
class PinterestSchema(PublisherSchema):
#post_dump
def dump_data(self,data):
data["type"]="Pinterest
"
-View
#blueprint.route('/api/sample/publishers/<id>', methods=('GET',))
#use_kwargs({'type': fields.Str(), 'start_date': fields.Str(),'end_date':fields.Str()},location="query")
#marshal_with(facebook_schema)
def get_data(id, type, start_date=None,end_date=None):
facebook_data = Facebook.query.filter_by(id=id)
.filter(Facebook.date.between(start_date,end_date))
.limit(10).all()
Ideally i would like to do this in my view;
pinterest_data = Pinterest.query.filter_by(id=id)
.filter(Pinterest.date.between(start_date,end_date))
.limit(10).all()
facebook_data.query.union(pinterest_data)
Union like this throws an error in flask application and also i have slightly different schemas for each publisher and i don't know how i can return both of them when i de-serialize with marshall
something like this maybe?
#marshal_with(facebook_schema,pinterest_schema)

Finding objects without relationship in django

I am learning Django, and want to retrieve all objects that DON'T have a relationship to the current object I am looking at.
The idea is a simple Twitter copycat.
I am trying to figure out how to implement get_non_followers.
from django.db import models
RELATIONSHIP_FOLLOWING = 1
RELATIONSHIP_BLOCKED = 2
RELATIONSHIP_STATUSES = (
(RELATIONSHIP_FOLLOWING, 'Following'),
(RELATIONSHIP_BLOCKED, 'Blocked'),
)
class UserProfile(models.Model):
name = models.CharField(max_length=200)
website = models.CharField(max_length=200)
email = models.EmailField()
relationships = models.ManyToManyField('self', through='Relationship',
symmetrical=False,
related_name='related_to')
def __unicode__ (self):
return self.name
def add_relationship(self, person, status):
relationship, created = Relationship.objects.get_or_create(
from_person=self,
to_person=person,
status=status)
return relationship
def remove_relationship(self, person, status):
Relationship.objects.filter(
from_person=self,
to_person=person,
status=status).delete()
return
def get_relationships(self, status):
return self.relationships.filter(
to_people__status=status,
to_people__from_person=self)
def get_related_to(self, status):
return self.related_to.filter(
from_people__status=status,
from_people__to_person=self)
def get_following(self):
return self.get_relationships(RELATIONSHIP_FOLLOWING)
def get_followers(self):
return self.get_related_to(RELATIONSHIP_FOLLOWING)
def get_non_followers(self):
# How to do this?
return
class Relationship(models.Model):
from_person = models.ForeignKey(UserProfile, related_name='from_people')
to_person = models.ForeignKey(UserProfile, related_name='to_people')
status = models.IntegerField(choices=RELATIONSHIP_STATUSES)

This isn't particularly glamorous, but it gives correct results (just tested):
def get_non_followers(self):
UserProfile.objects.exclude(to_people=self,
to_people__status=RELATIONSHIP_FOLLOWING).exclude(id=self.id)
In short, use exclude() to filter out all UserProfiles following the current user, which will leave the user themselves (who probably shouldn't be included) and all users not following them.

i'v been searching for a method or some way to do that for like an hour, but i found nothing.
but there is a way to do that.
you can simply use a for loop to iterate through all objects and just remove all objects that they have a special attribute value.
there is a sample code here:
all_objects = className.objects.all()
for obj in all_objects:
if obj.some_attribute == "some_value":
all_objects.remove(obj)

Solution to the implementation of get_non_followers:
def get_non_following(self):
return UserProfile.objects.exclude(to_person__from_person=self, to_person__status=RELATIONSHIP_FOLLOWING).exclude(id=self.id)
This answer was posted as an edit to the question Finding objects without relationship in django by the OP Avi Meir under CC BY-SA 3.0.

current_userprofile = current_user.get_profile()
rest_of_users = Set(UserProfile.objects.filter(user != current_userprofile))
follow_relationships = current_userprofile.relationships.filter(from_person=current_user)
followers = Set();
for follow in follow_relationships:
followers.add(follow.to_person)
non_followeres = rest_of_users.difference(followers)
Here non_followers is the list of userprofiles you desire. current_user is the user whose non_followers you are trying to find.

I haven't tested this out, but it think it should do what you want.
def get_non_followers(self):
return self.related_to.exclude(
from_people__to_person=self)

Django Tastypie throws a 'maximum recursion depth exceeded' when full=True on reverse relation.

I get a maximum recursion depth exceeded if a run the code below:
from tastypie import fields, utils
from tastypie.resources import ModelResource
from core.models import Project, Client
class ClientResource(ModelResource):
projects = fields.ToManyField(
'api.resources.ProjectResource', 'project_set', full=True
)
class Meta:
queryset = Client.objects.all()
resource_name = 'client'
class ProjectResource(ModelResource):
client = fields.ForeignKey(ClientResource, 'client', full=True)
class Meta:
queryset = Project.objects.all()
resource_name = 'project'
# curl http://localhost:8000/api/client/?format=json
# or
# curl http://localhost:8000/api/project/?format=json
If a set full=False on one of the relations it works. I do understand why this is happening but I need both relations to bring data, not just the "resource_uri". Is there a Tastypie way to do it? I managed to solve the problem creating a serialization method on my Project Model, but it is far from elegant. Thanks.

You would have to override full_dehydrate method on at least one resource to skip dehydrating related resource that is causing the recursion.
Alternatively you can define two types of resources that use the same model one with full=Trueand another with full=False.

Thanks #astevanovic pointing the right direction.
I found that overriding dehydrate method to process only some specified fields is a bit less tedious than overriding full_hydrate method to skip fields.
In the pursuit of reusability, I came up with the following code snippets. Hope it would be useful to some:
class BeeModelResource(ModelResource):
def dehydrate(self, bundle):
bundle = super(BeeModelResource, self).dehydrate(bundle)
bundle = self.dehydrate_partial(bundle)
return bundle
def dehydrate_partial(self, bundle):
for field_name, resource_field in self.fields.items():
if not isinstance(resource_field, RelatedField):
continue
if resource_field.full: # already dehydrated
continue
if not field_name in self._meta.partial_fields:
continue
if isinstance(resource_field, ToOneField):
fk_object = getattr(bundle.obj, resource_field.attribute)
fk_bundle = Bundle(obj=fk_object, request=bundle.request)
fk_resource = resource_field.get_related_resource(fk_object)
bundle.data[field_name] = fk_resource.dehydrate_selected(
fk_bundle, self._meta.partial_fields[field_name]).data
elif isinstance(resource_field, ToManyField):
data = []
fk_objects = getattr(bundle.obj, resource_field.attribute)
for fk_object in fk_objects.all():
fk_bundle = Bundle(obj=fk_object, request=bundle.request)
fk_resource = resource_field.get_related_resource(fk_object)
fk_bundle = fk_resource.dehydrate_selected_fields(
fk_bundle, self._meta.partial_fields[field_name])
data.append(fk_bundle.data)
bundle.data[field_name] = data
return bundle
def dehydrate_selected_fields(self, bundle, selected_field_names):
# Dehydrate each field.
for field_name, field_object in self.fields.items():
# A touch leaky but it makes URI resolution work.
# (borrowed from tastypie.resources.full_dehydrate)
if field_name in selected_field_names and not self.is_special_fields(field_name):
if getattr(field_object, 'dehydrated_type', None) == 'related':
field_object.api_name = self._meta.api_name
field_object.resource_name = self._meta.resource_name
bundle.data[field_name] = field_object.dehydrate(bundle)
bundle.data['resource_uri'] = self.get_resource_uri(bundle.obj)
bundle.data['id'] = bundle.obj.pk
return bundle
#staticmethod
def is_special_fields(field_name):
return field_name in ['resource_uri']
With #sigmus' example, the resources will need 3 modifications:
both resource will use BeeModuleResource as its super class (or, add dehydrate_partial to one resource and dehydrate_selected to the other.)
unset full=True on either of the resource
add partial_fields into the resource Meta the unset resource
```
class ClientResource(BeeModelResource): # make BeeModelResource a super class
projects = fields.ToManyField(
'api.resources.ProjectResource', 'project_set'
) # remove full=True
class Meta:
queryset = Client.objects.all()
resource_name = 'client'
partial_fields = {'projects': ['memo', 'title']} # add partial_fields
class ProjectResource(BeeModelResource): # make BeeModelResource a super class
client = fields.ForeignKey(ClientResource, 'client', full=True)
class Meta:
queryset = Project.objects.all()
resource_name = 'project'

Dead simple solution: set the use_in = 'list' kwarg on both relationship fields!
The docs: http://django-tastypie.readthedocs.org/en/latest/fields.html#use-in

Django model subclassing: Get the subclass by querying the superclass

The following code is given:
class BaseMedium(models.Model):
title = models.CharField(max_length=40)
slug = models.SlugField()
class A(BaseMedium):
url = models.URLField()
class B(BaseMedium):
email = models.EmailField()
I now want to query every BaseMedium.
b = BaseMedium.objects.all()
How do I print every information including the subclass fields without knowing what the subclass type is?
b[0].a would print the information if b[0] is actually related to an A instance but if it's related to B it would print an DoesNotExist Exception.
This makes sense but I'd like to have a common variable or method that returns the related object.
Maybe my Database layout isn't really great to query that way if so I'd be glad if you'd recommend a better layout.
I thought about using a GenericForeignKey
class Generic(models.Model):
basemedium = models.ForeignKey('BaseMedium')
content_type = models.ForeignKey(ContentType)
object_id = models.PositiveIntegerField()
object = generic.GenericForeignKey('content_type', 'object_id')
but this solution seems to be to complicated and I think you guys have better solutions.

You should check the solution posted by Carl Meyer some time ago. It internally uses the ContentType approach, but it encapsulates it very elegantly .
He also points to an alternative, and more efficient solution, that doesn't need to store an aditional field on the database, but it will only work for direct child classes. If you have several inheritance levels, the first solution is better.

The only way to do this is to explicitly store on the base model what type it is. So have a derived_type (or whatever) field on BaseMedium, and set it on save. Then you can have a get_derived_type method:
def get_derived_type(self):
if self.derived_type == 'A':
return self.a
elif self.derived_type == 'B':
return self.b
and so on.

Thanks mr. Roseman for your reply.
I developed your idea a bit further.
Here is what I came up with:
def related_object(self, default_pointer_name='_ptr'):
models = [A,B] #models
object = None
argument = '%s%s' %(self.__class__.__name__.lower(), default_pointer_name)
query = { argument : self}
for model in models:
try:
object = model.objects.get(**query)
except model.DoesNotExist:
pass
else:
return object
if object == None:
raise RelatedObjectException
return object
This is a method used by BaseMedium.

This worked for me (using self.subclass_name_in_lower_case):
In this example subclasses are TextTreeItem, CategoryTreeItem and KeywordTreeItem.
class TreeItem(MPTTModel):
parent = TreeForeignKey('self', on_delete=models.CASCADE, verbose_name=_('Parent'),
null=True, blank=True, related_name='%(class)s_related')
objects = CustomTreeManager()
#property
def daughter(self):
try:
return self.texttreeitem
except TreeItem.texttreeitem.RelatedObjectDoesNotExist:
pass
try:
return self.categorytreeitem
except TreeItem.categorytreeitem.RelatedObjectDoesNotExist:
pass
try:
return self.keywordtreeitem
except TreeItem.keywordtreeitem.RelatedObjectDoesNotExist:
return self

email whitelist/blacklist in python/django

I am writing a django app that keeps track of which email addresses are allowed to post content to a user's account. The user can whitelist and blacklist addresses as they like.
Any addresses that aren't specified can either be handled per message or just default to whitelist or blacklist (again user specified).
Here are the django models I wrote... do you think is a good way to do it? or should I add a whitelist and blacklist field to each user's profile model?
class knownEmail(models.Model):
# The user who set this address' permission, NOT
# the user who the address belongs to...
relatedUser = models.ManyToManyField(User)
email = models.EmailField()
class whiteList(knownEmail):
pass
class blackList(knownEmail):
pass
Then I could do something like:
def checkPermission(user, emailAddress):
"Check if 'emailAddress' is allowed to post content to 'user's profile"
if whiteList.objects.filter(relatedUser=user, email=emailAddress):
return True
elif blackList.objects.filter(relatedUser=user, email=emailAddress):
return False
else:
return None
Is there a better way?

I would restructure it so both lists were contained in one model.
class PermissionList(models.Model):
setter = models.ManyToManyField(User)
email = models.EmailField(unique=True) #don't want conflicting results
permission = models.BooleanField()
Then, your lists would just be:
# whitelist
PermissionList.objects.filter(permission=True)
# blacklist
PermissionList.objects.filter(permission=False)
To check a particular user, you just add a couple functions to the model:
class PermissionList(...):
...
#classmethod
def is_on_whitelist(email):
return PermissionList.objects.filter(email=email, permission=True).count() > 0
#classmethod
def is_on_blacklist(email):
return PermissionList.objects.filter(email=email, permission=False).count() > 0
#classmethod
def has_permission(email):
if PermissionList.is_on_whitelist(email):
return True
if PermissionList.is_on_blacklist(email):
return False
return None
Having everything in one place is a lot simpler, and you can make more interesting queries with less work.

[Please start All Class Names With Upper Case Letters.]
Your code doesn't make use of your class distinction very well.
Specifically, your classes don't have any different behavior. Since both classes have all the same methods, it isn't clear why these are two different classes in the first place. If they have different methods, then your solution is good.
If, however, they don't have different methods, you might want to look at providing a customized manager for each of the two subsets of KnownEmail
class WhiteList( models.Manager ):
def get_query_set( self ):
return super( WhiteList, self ).get_query_set().filter( status='W' )
class BlackList( models.Manager )
def get_query_set( self ):
return super( BlackList, self ).get_query_set().filter( status='B' )
class KnownEmail( models.Model ):
relatedUser = models.ForeignKey(User)
email = models.EmailField()
status = models.CharField( max_length=1, choices=LIST_CHOICES )
objects = models.Manager() # default manager shows all lists
whiteList= WhiteList() # KnownEmail.whiteList.all() is whitelist subset
blackList= BlackList() # KnownEmail.blackList.all() is blackList subset

This class compares an email address with a blacklist of email domains. If you preffer you can download this module using pip install django-email-blacklist.
from django.conf import settings
import re
class DisposableEmailChecker():
"""
Check if an email is from a disposable
email service
"""
def __init__(self):
self.emails = [line.strip() for line in open(settings.DISPOSABLE_EMAIL_DOMAINS)]
def chunk(self, l, n):
return (l[i:i + n] for i in range(0, len(l), n))
def is_disposable(self, email):
for email_group in self.chunk(self.emails, 20):
regex = "(.*" + ")|(.*".join(email_group) + ")"
if re.match(regex, email):
return True
return False