Develop Reference

How to set an unspecified value to Meta class (How to create a template table class) django_tables2

I have more than 30 tables, so I don`t want to write the same code for all of them. Even its copypaste.
All my tables should look like this one:
class MyTable(tables.Table):
edit_link = tables.columns.Column(viewname='edit_table_field',
kwargs={'table_name': 'MyModel', 'pk': A('pk')}, text='edit)
...
class Meta:
model = MyModel
...
I want to make a tamplate from this class which will take the name of the model (MyModel) as an argument and create the same tablestables for each of them in one class. I don't know how to pass a value from a class to a metaclass.
Also, maybe someone know any frameworks for python that implements the mapping and CRUD methods for my tables like it`s in Vaadin (Java)?

Since python is a highly dynamic language, you can achieve this in multiple ways. One that comes to mind (you can declare classes inside the function and also pass / return them as regular objects):
def get_table(model_cls):
class ModelTable(tables.Table):
edit_link = tables.columns.Column(
viewname='edit_table_field',
kwargs={'table_name': model_cls.__name__, 'pk': A('pk')},
text='edit'
)
...
class Meta:
model = model_cls
...
return ModelTable
tables = [
get_table(model)
for model in (Brand, Product, Category) # Your models here
]
You can extend this to generate complete views as well.

Generating marshmallow schema automatically with JSON serializable enums

Long gone are the days of creating marshmallow schemas identical to my models. I found this excellent answer that explained how I could auto generate schemas from my SQA models using a simple decorator, so I implemented it and replaced the deprecated ModelSchema for the newer SQLAlchemyAutoSchema:
def add_schema(cls):
class Schema(SQLAlchemyAutoSchema):
class Meta:
model = cls
cls.Schema = Schema
return cls
This worked great... until I bumped into a model with a bloody Enum.
The error: Object of type MyEnum is not JSON serializable
I searched online and I found this useful answer.
But I'd like to implement it as part of the decorator so that it is generated automatically as well. In other words, I'd like to automatically overwrite all Enums in my model with EnumField(TheEnum, by_value=True) when generating the schema using the add_schema decorator; that way I won't have to overwrite all the fields manually.
What would be the best way to do this?

I have found that the support for enum types that was initially suggested only works if OneOf is the only validation class that exists in field_details. I added in some argument parsing (in a rudimentary way by looking for choices after stringifying the results _repr_args() from OneOf) to check the validation classes to hopefully make this implementation more universally usable:
def add_schema(cls):
class Schema(ma.SQLAlchemyAutoSchema):
class Meta:
model = cls
fields = Schema._declared_fields
# support for enum types
for field_name, field_details in fields.items():
if len(field_details.validate) > 0:
check = str(field_details.validate[0]._repr_args)
if check.__contains__("choices") :
enum_list = field_details.validate[0].choices
enum_dict = {enum_list[i]: enum_list[i] for i in range(0, len(enum_list))}
enum_clone = Enum(field_name.capitalize(), enum_dict)
fields[field_name] = EnumField(enum_clone, by_value=True, validate=validate.OneOf(enum_list))
cls.Schema = Schema
return cls
Thank you jgozal for the initial solution, as I really needed this lead for my current project.

This is my solution:
from marshmallow import validate
from marshmallow_sqlalchemy import SQLAlchemyAutoSchema
from marshmallow_enum import EnumField
from enum import Enum
def add_schema(cls):
class Schema(SQLAlchemyAutoSchema):
class Meta:
model = cls
fields = Schema._declared_fields
# support for enum types
for field_name, field_details in fields.items():
if len(field_details.validate) > 0:
enum_list = field_details.validate[0].choices
enum_dict = {enum_list[i]: enum_list[i] for i in range(0, len(enum_list))}
enum_clone = Enum(field_name.capitalize(), enum_dict)
fields[field_name] = EnumField(enum_clone, by_value=True, validate=validate.OneOf(enum_list))
cls.Schema = Schema
return cls
The idea is to iterate over the fields in the Schema and find those that have validation (usually enums). From there we can extract a list of choices which can then be used to build an enum from scratch. Finally we overwrite the schema field with a new EnumField.
By all means, feel free to improve the answer!

How to handle ReferenceField migration with flask mongoengine?

I've implemented a simple flask application with mongodb that now needs some upgrades.
Let's say to have a class model for Foo and a class model for Bar in which there is a reference field to Foo
class Foo(Document):
title = StringField()
class Bar(Document):
name = StringField()
foo = ReferenceField('Foo')
Let the flask application runs doing its job for a while, so that now there are some data in the DB.
Due to requirements changes, we need to refactor the Foo class subclassing it from a new super class:
class SuperFoo(Document):
meta = { 'allow_inheritance': True,}
#[...]
class Foo(SuperFoo):
#[...]
class Bar(Document):
name = StringField()
foo = ReferenceField('Foo')
The code above works well with an empty database.
But in case of some data in it, mongoengine raises an Exception when a flask admin tries to show a Bar instance (in edit mode)
File "[...]/site-packages/mongoengine/fields.py", line 1124, in __get__
raise DoesNotExist('Trying to dereference unknown document %s' % value)
mongoengine.errors.DoesNotExist: Trying to dereference unknown document DBRef('super_foo', ObjectId('5617a08939c6c70cbaa2af6e'))
I suppose data model needs to be migrated in some way.
How?
thanks,
alessandro.

After a little analyis I came up to solve the problem.
Mongoengine creates a new collection super_foo.
Documents of every inherited class goes into this super_foo collection with an additional attribute _cls.
The value is the CamelCased hierarchy path of that class. In this example documents will have
'_cls': 'SuperFoo.Foo' field.
What I've done is to copy every document from the old foo collection into the new super_foo one, adding the field {'_cls': u'SuperPlesso.Plesso'} to each.
The migration function should look like:
def migrationFunc():
from pymongo.errors import DuplicateKeyError
from my.app import models
_cls = {'_cls': u'SuperFoo.Foo'}
fromOldCollection = models.Foo._collection
toSuperCollection = models.Superfoo._collection
for doc in fromOldCollection.find():
doc.update(_cls)
try:
toSuperCollection.insert(doc)
except DuplicateKeyError:
logger.error('...')
Then I updated the base code of the models with the actual new hierarchy:
class SuperFoo(Document):
meta = { 'allow_inheritance': True,}
#[...]
# was class Foo(Document)
class Foo(SuperFoo):
#[...]
Al back references to Foo in Bar collections, or elsewhere, are preserved.

SQLAlchemy Mixin Expected Mapped Entity got Implementation

I have a joined-inheritance set of models.
class Asset(db.Model):
__tablename__ = 'assets'
asset_id = db.Column(db.Integer, primary_key=True, autoincrement=True)
asset_type_id = db.Column(db.ForeignKey('asset_types.asset_type_id'))
...
__mapper_args__ = {
'polymorphic_on': asset_type_id
}
class Vehicle(Asset):
__tablename__ = 'vehicles'
asset_id = db.Column(db.ForeignKey('assets.asset_id'), primary_key=True)
...
...
__mapper_args__ = {
'polymorphic_identity':2
}
I would like to mixin the Vehicle into a class with self referencing methods. Like so:
from library.models import Vehicle as EsdbVehicle
class Vehicle(EsdbVehicle):
def update_usage(self, db_session, odometer):
self.odometer = odometer
db_session.commit()
When I query the EsdbVehicle model vehicles = db_session.query(EsdbVehicle).all() I get 115 results, which matches my DB. However when I query the implementing Vehicle I get 0 results. What am I missing?

I cannot be 100% sure about this without setting up a DB connection and running your code but my instinct tells me that this is not working as expected because it looks to SQLAlchemy like you're trying to do table inheritance (but since you're not and the class definition is incomplete SQLAlchemy is confused) when what you really want is a mixin.
I would try something more like this:
class Vehicle(Asset):
# stuff...
__mapper_args__ = {
'polymorphic_on': 'vehicle_subtype',
'polymorphic_identity': 'base_vehicle'
}
class UsageMixin(object):
def update_usage(self, *args):
# stuff...
pass
class VehiclesThatUpdate(Vehicle, UsageMixin):
__mapper_args__ = {
'polymorphic_identity': 'updating_vehicles'
}
The reason that should work is because everytime you extend an ORM mapped class, you are essentially telling SQLAlchemy you want to perform some kind of table inheritance. In this case, you can get away with just adding a type (for STI) because you aren't adding attributes (columns) you're just adding functionality.
If this does not work out (or is incomplete) just add a comment and I'll adjust.
EDIT: Another thought. The other thing you could do is skip the inheritance all together and just write functions that operate on the Vehicle class. They do not need to be methods necessarily. I often do this to avoid extra coupling between my actual SQL model and the class / method structure.
The more you use ORM's the more you start to realize that often times using less of their features is better.

Add a dynamically generated Model to a models.py in an Django project

I am generating a Django model based on an abstract model class AbstractAttr and a normal model (let's say Foo).
I want my foo/models.py to look like this:
from bar.models import Attrs
# ...
class Foo(models.Model):
....
attrs = Attrs()
In the Attrs class which mimics a field I have a contribute_to_class that generates the required model using type(). The generated model c is called FooAttr.
Everything works. If I migrate, I see FooAttr appear in the proper table.
EXCEPT FOR ONE THING.
I want to be able to from foo.models import FooAttr. Somehow my generated FooAttr class is not bound to the models.py file in which it is generated.
If I change my models.py to this:
class Foo(models.Model):
# ...
FooAttr = generate_foo_attr_class(...)
it works, but this is not what I want (for example, this forces the dev to guess the generate class name).
Is what I want possible, define the class somewhat like in the first example AND bind it to the specific models.py module?
The project (pre-Alpha) is here (in develop branch):
https://github.com/zostera/django-mav
Some relevant code:
def create_model_attribute_class(model_class, class_name=None, related_name=None, meta=None):
"""
Generate a value class (derived from AbstractModelAttribute) for a given model class
:param model_class: The model to create a AbstractModelAttribute class for
:param class_name: The name of the AbstractModelAttribute class to generate
:param related_name: The related name
:return: A model derives from AbstractModelAttribute with an object pointing to model_class
"""
if model_class._meta.abstract:
# This can't be done, because `object = ForeignKey(model_class)` would fail.
raise TypeError("Can't create attrs for abstract class {0}".format(model_class.__name__))
# Define inner Meta class
if not meta:
meta = {}
meta['app_label'] = model_class._meta.app_label
meta['db_tablespace'] = model_class._meta.db_tablespace
meta['managed'] = model_class._meta.managed
meta['unique_together'] = list(meta.get('unique_together', [])) + [('attribute', 'object')]
meta.setdefault('db_table', '{0}_attr'.format(model_class._meta.db_table))
# The name of the class to generate
if class_name is None:
value_class_name = '{name}Attr'.format(name=model_class.__name__)
else:
value_class_name = class_name
# The related name to set
if related_name is None:
model_class_related_name = 'attrs'
else:
model_class_related_name = related_name
# Make a type for our class
value_class = type(
str(value_class_name),
(AbstractModelAttribute,),
dict(
# Set to same module as model_class
__module__=model_class.__module__,
# Add a foreign key to model_class
object=models.ForeignKey(
model_class,
related_name=model_class_related_name
),
# Add Meta class
Meta=type(
str('Meta'),
(object,),
meta
),
))
return value_class
class Attrs(object):
def contribute_to_class(self, cls, name):
# Called from django.db.models.base.ModelBase.__new__
mav_class = create_model_attribute_class(model_class=cls, related_name=name)
cls.ModelAttributeClass = mav_class

I see you create the model from within models.py, so I think you should be able to add it to the module's globals. How about this:
new_class = create_model_attribute_class(**kwargs)
globals()[new_class.__name__] = new_class
del new_class # no need to keep original around

Thanks all for thinking about this. I have updated the source code of the project at GitHub and added more tests. See https://github.com/zostera/django-mav
Since the actual generation of the models is done outside of foo/models.py (it takes place in mav/models.py, it seems Pythonically impossible to link the model to foo/models.py. Also, after rethinking this, it seems to automagically for Python (explicit is better, no magic).
So my new strategy is to use simple functions, a decorator to make it easy to add mav, and link the generated models to mac/attrs.py, so I can universally from mav.attrs import FooAttr. I also link the generated class to the Foo model as Foo._mav_class.
(In this comment, Foo is of course used as an example model that we want to add model-attribute-value to).