Does the Python (Flask) SQL Alchemy uses both DAO and ORM design, or simply just ORM?
I am learning design strategies and I thought of SQLAlchemy. Is it considered a DAO (clearly ORM) as well?
By default, it does not look like DAO.
What if I defined a class for an existing model class , for example given I have the following class:
class User(db.model):
id = db.Column(db.Integer, primary_key=True)
email = db.Column(db.String(120), unique=True, nullable=False)
verified= db.Column(db.String(5), unique=False, nullable=False)
I define another class, UserDao
class UserDao:
def addNewUser(user):
pass
def retrieveAllUsers(user):
users = User.query.limit(5).all()
And I instantitate and object of this UserDao class and call the respective methods to do some database operations through the respective method, does this make it a "DAO Pattern"?
Regarding your question "And I instantitate and object of this UserDao class and call the respective methods to do some database operations through the respective method, does this make it a "DAO Pattern"?" I would say yes, it is in that direction however you need to look at your whole application overall to answer that question.
The idea of a DAO is to keep the code of your application that does the business logic separate from the code that handles how you get and store the data. So the easiest way to answer your question is to look at your whole application and ask yourself "if tomorrow I want to work with mongodb (for example) instead of mysql (whether you are using sqlachemy or not) what code of my application would I need to change?". If you need to change code from the business logic code then it means you don't have a dao or at least not a strong one. If you only need to touch the code that handles database operations then you have a DAO. You can also look it in another way: what if tomorrow you decide that you are better off using django instead of sqlachemy? what code would you need to change? would you need to touch the code that does the business logic?
Sqlalchemy implements repository pattern, not dao pattern.
You don't have to implement any dao classes in your code when you use sqlalchemy.
Using sqlalchemy orm session(or classes based on db.model) means you are using repository.
References
https://techspot.zzzeek.org/2012/02/07/patterns-implemented-by-sqlalchemy/
Related
Context
I'm working on refactoring a Django 2.X app, particularly the core model, CoreModel. There's a single database (Postgres) containing all related tables.
Instances of CoreModel will no longer live in Postgres after this refactor, they will live somewhere else but outside the scope of the Django project, let's say some AWS No-SQL database service.
There also several satellites models SateliteModel to CoreModel which will continue to live on Postgres, but CoreModelis currently modelled as a foreign key field.
class CordeModel(models.Model):
pass
class SatelliteModel(models.Model):
core = models.ForeignKey(CoreModel)
def some_instance_method(self):
return self.core.calculate_stuff() # <- override self.core!
Problem
The code is filled with mentions to the CoreModel relation, and I haven't been able to successfully solved this issue.
My first naive approach was to implement a #property getter method, that way I had enough flexibility to do something like:
#property
def core(self):
try:
# ORM
return self.core
except CoreNotFound:
# External datastore
return aws_client.fetch_core()
With this snippet I have a circular dependency on the core name, so the idea is out.
I could rename the foreign key: but I would much rather not touch the database schema. After all I'm already refactoring the central part of the app, and that's an very error-prone process. I'd do this if there's no other choice.
I could rename the #property field, to something like current_core: This way I avoid the infinite recursion part, but this in turn would imply a very big task of searching the whole code base for mentions of the relation, and this being the central model, it would take a lot of time.
After some hours of research I'm beginning to doubt if the concept of overriding a getter for a foreign key field is possible, as I need it. Maybe this is isn't exactly what I'm looking for, it's a very unusual use case, but the requirement is also very unusual.
Any insights you can give are greatly appreciated.
UPDATE
I've forgotten to add the most crucial piece of information.
Most CoreModel will be removed for Postgres (the historic ones), but there's a tiny part of CoreModels that will remain and will be moved after a while. In essence, only the "active" CoreModels will stay in Postgres, but all will eventually be moved out, while new CoreModel will be created.
So that rules out the possibility of change the ForeignKey field for an integer.
You could retain but rename the foreign key and then add a property with the old name
class SatelliteModel(models.Model):
old_core = models.ForeignKey(CoreModel, null=True, blank=True, on_delete=models.SET_NULL)
#property
def core(self):
try:
return self.old_core
except CoreModel.DoesNotExist:
return aws_client.fetch_core()
This would change the column name in your schema, although you could override the column name to prevent this
old_core = models.ForeignKey(CoreModel, db_column='core_id', null=True, blank=True, on_delete=models.SET_NULL)
It may be possible to create a subclass of ForeignKey that would perform as you wished, if this answer is not sufficient I can share some thoughts
I'd like to create a 1:n relationship between two tables dynamically. My DB model is mapped via SQLAlchemy but due to some special features of my application I can not use the default declarative way.
E.g.
class Foo(Base):
id = Column(Integer, autoincrement=True, primary_key=True)
flag = Column(Boolean)
class Bar(Base):
id = Column(Integer, autoincrement=True, primary_key=True)
foo_id = Column(Integer, ForeignKey('foo.id'))
# declarative version:
# foo = relationship(Foo)
So I want to add relationship named "foo" to the mapped class "Bar" after Bar was defined and SQLAlchemy did its job of defining a mapper etc.
Update 2017-09-05: Why is this necessary for me? (I thought I could omit this because I think it mostly distracts from the actual problem to solve but since there were comments abouts it...)
First of all I don't have a single database but hundreds/thousands. Data in old databases must not be altered in any way but I want a single source code to access old data (even though data structure and calculation rules change significantly).
Currently we use multiple model definitions. Later definitions extend/modify previous ones. Often we manipulate SQLAlchemy models dynamically. We try not to have code in mapped classes because we think it will be much harder ensuring correctness of that code after changing a table many times (code must work in every intermediate step).
In many cases we extend tables (mapped classes) programatically in model X after it was initially defined in model X-1. Adding columns to an existing SQLAlchemy ORM class is manageable. Now we are adding a new reference column an existing table and a relationship() provides a nicer Python API.
Well, my question above is again a nice example of SQLAlchemy's super powers (and my limited understanding):
Bar.__mapper__.add_property('foo', relationship('Foo'))
Likely I was unable to get this working initially because some of my surrounding code mixed adding relationships and columns. Also there is one important difference to declaring columns:
Column('foo', Integer)
For columns the first parameter can be the column name but you can not use this for relationships. relationship('foo', 'Foo') triggers exceptions when passing it to .add_property().
I've been using mongoengine for a while now and have a ton of python data processing code that relies on a common set of Object Document Models.
Now I need to access the same mongodb instances from Flask. I'd like to use the same ODM definitions.
class User(Document):
email = StringField(required=True)
first_name = StringField(max_length=50)
last_name = StringField(max_length=50)
The problem is that flask-mongoengine requires you to first set up your flask context "db" and then build your ODM definitions, inheriting the document class and fieldtypes from "db" instead of the base mongoengine classes.
class User(db.Document):
email = db.StringField(required=True)
first_name = db.StringField(max_length=50)
last_name = db.StringField(max_length=50)
One solution, I suppose, is to make copies of all of the existing ODM definitions, import "db" from my main flask app, and then prepend everything with "db." If I do that, I'll have to maintain two sets of nearly identical ODM definitions.
If I simply change everything to the "db." version, that would probably break all of my legacy code.
So I'm thinking there might be a trick using super() on the document classes that can detect whether I'm importing my ODM into a Flask context or whether I'm importing it from a stand alone data processing script.
I'm also thinking I don't want to have to super() every fieldtype for every document, that I should be able to build or reference a common function that took care of that for me.
However, my super() skills are weak. I'm not even certain if that is the best approach. I was hoping someone might be able and willing to share some hints as to how to approach this.
I'm in the process of writing my first RESTful web service atop GAE and the Python 2.7 runtime; I've started out using Guido's shiny new ndb API.
However, I'm unsure how to solve a particular case without the implicit back-reference feature of the original db API. If the user-agent requests a particular resource and those resources 1 degree removed:
host/api/kind/id?depth=2
What's the best way to discover a related collection of entities from the "one" in a one-to-many relationship, given that the kind of the related entity is unknown at development time?
I'm unable to use a replacement query as described in a previous SO inquiry due to the latter restriction. The fact that my model is definable at runtime (and therefore isn't hardcoded) prevents me from using a query to filter properties for matching keys.
Ancestor and other kindless queries are also out due to the datastore limitation that prevents me from filtering on a property without the kind specified.
Thus far, the only idea I've had (beyond reverting to the db api) is to use a cross-group transaction to write my own reference on the "one", either by updating an ndb.StringProperty(repeat=True) containing all the related kinds when an entity of a new kind is introduced or by simply maintaining a list of keys on the "one" ndb.KeyProperty(repeat=True) every time a related "many" entity is written to the datastore.
I'm hoping someone more experienced than myself can suggest a better approach.
Given jmort253's suggestion, I'll try to augment my question with a concrete example adapted from the docs:
class Contact(ndb.Expando):
""" The One """
# basic info
name = ndb.StringProperty()
birth_day = ndb.DateProperty()
# If I were using db, a collection called 'phone_numbers' would be implicitly
# created here. I could use this property to retrieve related phone numbers
# when this entity was queried. Since NDB lacks this feature, the service
# will neither have a reference to query nor the means to know the
# relationship exists in the first place since it cannot be hard-coded. The
# data model is extensible and user-defined at runtime; most relationships
# will be described only in the data, and must be discoverable by the server.
# In this case, when Contact is queried, I need a way to retrieve the
# collection of phone numbers.
# Company info.
company_title = ndb.StringProperty()
company_name = ndb.StringProperty()
company_description = ndb.StringProperty()
company_address = ndb.PostalAddressProperty()
class PhoneNumber(ndb.Expando):
""" The Many """
# no collection_name='phone_numbers' equivalent exists for the key property
contact = ndb.KeyProperty(kind='Contact')
number = ndb.PhoneNumberProperty()
Interesting question! So basically you want to look at the Contact class and find out if there is some other model class that has a KeyProperty referencing it; in this example PhoneNumber (but there could be many).
I think the solution is to ask your users to explicitly add this link when the PhoneNumber class is created.
You can make this easy for your users by giving them a subclass of KeyProperty that takes care of this; e.g.
class LinkedKeyProperty(ndb.KeyProperty):
def _fix_up(self, cls, code_name):
super(LinkedKeyProperty, self)._fix_up(cls, code_name)
modelclass = ndb.Model._kind_map[self._kind]
collection_name = '%s_ref_%s_to_%s' % (cls.__name__,
code_name,
modelclass.__name__)
setattr(modelclass, collection_name, (cls, self))
Exactly how you pick the name for the collection and the value to store there is up to you; just put something there that makes it easy for you to follow the link back. The example would create a new attribute on Contact:
Contact.PhoneNumber_ref_contact_to_Contact == (PhoneNumber, PhoneNumber.contact)
[edited to make the code working and to add an example. :-) ]
Sound like a good use case for ndb.StructuredProperty.
In a Google App Engine solution (Python), I've used the db.ListProperty as a way to describe a many-to-many relation, like so:
class Department(db.Model):
name = db.StringProperty()
#property
def employees(self):
return Employee.all().filter('departments', self.key())
class Employee(db.Model):
name = db.StringProperty()
departments = db.ListProperty(db.Key)
I create many-to-many relations by simply appending the Department key to the db.ListProperty like so:
employee.departments.append(department.key())
The problem is that I don't know how to actually remove this relationship again, when it is no longer needed.
I've tried Googling it, but I can't seem to find any documentation that describes the db.ListProperty in details.
Any ideas or references?
The ListProperty is just a Python list with some helper methods to make it work with GAE, so anything that applies to a list applies to a ListProperty.
employee.departments.remove(department.key())
employee.put()
Keep in mind that the data must be deserialized/reserialized every time a change is made, so if you are looking for speed when adding or removing single values you may want to go with another method of modelling the relationship like the one in the Relationship Model section of this page.
The ListProperty method also has the disadvantage of sometimes producing very large indexes if you want to search through the lists in a datastore request.
This may not not be a problem for you since your Lists should be relatively small, but it's something to keep in mind for future projects.
Found it via trial and error:
employee.departments.remove(department.key())