I have multiple classes similar to the following:
class Weather(Base):
__tablename__ = "Weather"
id = Column(Integer, primary_key=True, nullable=False, autoincrement=True)
temperature = Column(Integer)
humidity = Column(Integer)
wind_speed = Column(Float)
wind_direction = Column(String)
I want to add a method df() that returns me the Pandas dataframe of that table. I know I can write it like this:
class Weather(Base):
__tablename__ = "Weather"
id = Column(Integer, primary_key=True, nullable=False, autoincrement=True)
temperature = Column(Integer)
humidity = Column(Integer)
wind_speed = Column(Float)
wind_direction = Column(String)
#staticmethod
def df():
with engine.connect() as conn:
return pd.read_sql_table(Weather.__tablename__ , conn)
But I want to implement this for every table. I guess if I can extend the Base class with this method I should be able to implement it once and use it in every class. Everything I have tried has failed because I do not have access to __tablename__ attribute.
SOLUTION
I ended up with a mix of both answers. I have used the first method proposed by #snakecharmerb (it allows to introduce the change without modifying the rest of the code) with the #classmethod proposed by #RomanPerekhrest (which is the bit I was missing).
class MyBase:
__tablename__ = None
#classmethod
def df(cls):
with engine.connect() as conn:
return pd.read_sql_table(cls.__tablename__ , conn)
Base = declarative_base(cls=MyBase)
You can do this by passing a custom class to the declarative_base function:
class MyBase:
__abstract__ = True
#staticmethod
def df():
with engine.connect() as conn:
return pd.read_sql_table(Weather.__tablename__ , conn)
Base = orm.declarative_base(cls=MyBase)
class MyModel(Base):
__tablename__ = 'tbl'
...
Alternatively, you can create a mixin that provides the static method and have classes inherit from it selectively.
class DFMixin:
#staticmethod
def df():
with engine.connect() as conn:
return pd.read_sql_table(Weather.__tablename__ , conn)
class MyModel(Base, DFMixin):
__tablename__ = 'tbl'
...
The mixin gives you more flexibility if not all of your models are going to need the dataframe functionality.
Declare an auxiliary class (say DfBase) with classmethod df(cls) having the desired behavior.
Then each derived class will access its __tablename__ attribute seamlessly via cls object which refers to the derived class itself.
class DfBase:
__tablename__ = None
#classmethod
def df(cls):
with engine.connect() as conn:
return pd.read_sql_table(cls.__tablename__ , conn)
class Weather(Base, DfBase):
__tablename__ = "Weather"
...
Related
I am using SQLAlchemy to create tables in my project. I have a requirement where all these tables should have some specific attributes and functions. I want to create a structure such that all tables inherit from an abstract class which includes these attributes and functions.
Here's an example of what I want to achieve:
Base = declarative_base()
# pseudo
class Table(ABC, Base):
# like #abstractattribute
some_attribtue = list()
#staticmethod
def some_func(self):
pass
class Users(Table):
__tablename__ = "users"
user_id = Column(Integer, primary_key=True)
username = Column(String, nullable=False)
some_attribute = list()
#staticmethod
def some_func():
do_something()
By doing this, I hope that I can use these classes in something like:
Base.metadata.create_all(engine)
while also being able to call:
Users.some_func()
I understand that this code wouldn't work as is, due to issues like having ABC and Base at the same time, not having #abstractattribute, and needing to add __tablename__ and a Primary-Key Column to the class Table.
I am thinking of using a decorator to achieve this, but I am not sure how to implement it correctly. This is the outline of my idea:
class Table(ABC):
some_attribute=None
#staticmethod
def some_func(self):
pass
# create decorator
def sql_table():
def decorator(abstract_class):
class SQLTable(Base): # How do I name the class correctly?
__tablename__ = abstract_class.__dict__["__tablename__"]
some_attribute = abstract_class.__dict__["some_attribute"]
for name, obj in abstract_class.__dict__.items():
if isinstance(obj, Column):
locals()[name] = obj
# How do I get the some_func function?
#sql_table
class Users(Table):
__tablename__ = "users"
user_id = Column(Integer, primary_key=True)
username = Column(String, nullable=False)
some_attribute = "some_val"
#staticmethod
def some_func():
do_something()
Any help or suggestions on how to implement this (not necessarily with decorators) would be greatly appreciated.
Thanks to #snakecharmerb and #ljmc I have found an solution that works for me, although there seem to be many ways one can achieve this.
The solution that works for me is:
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from sqlalchemy import Column, Integer, String
Base = declarative_base()
class Table(Base):
__abstract__ = True
#declared_attr
def __tablename__(cls) -> str: # so i don't have to specify no more
return cls.__name__.lower()
some_attribute = set() # this is the default
#staticmethod
def some_func(): # define default behavior (or pass)
do_something()
class Users(Table):
# define columns as usual
user_id = Column(Integer, primary_key=True)
username = Column(String, nullable=False)
some_attribute = set(["a"]) # overwrite the default
def some_func(): # overwrite the default behavior
do_something_else()
Now, this should be improved upon by specifying a type to some_attribute (typing is awesome).
On this question I learned how to set the schema on an ORM definition:
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Notification(Base):
__tablename__ = "dog"
__table_args__ = {"schema": "animal"}
id = Column(Integer, primary_key=True)
name = Column(String)
But I now need to make the schema configurable. I have tried passing the table_args parameter at object creation, but it's still trying the schema I put on the class definition.
The better solution I have found so far is to create a function that returns the class:
function get_notification_class(schema: str):
class Notification(Base):
__tablename__ = "dog"
__table_args__ = {"schema": schema}
id = Column(Integer, primary_key=True)
name = Column(String)
return Notification
And then, to use it
Notification = get_notification_class('animal')
obj = Notification('1', 'doggy')
I have code like this working fine:
def get_timestamp(ts):
return datetime.utcfromtimestamp(ts)
def set_timestamp(dt):
return time.mktime(dt.timetuple())
class Group(Base):
__tablename__ = 'group'
_created = Column('created', Integer, nullable=False)
#property
def created(self):
return get_timestamp(self._created)
#created.setter
def created(self, value):
self._created = set_timestamp(value)
I want some code like this, but it's not working:
created = synonym('_created',
descriptor=property(get_timestamp,
set_created))
Because it always passed in a self as the 1st param.
I'd like to use get_timestamp and set_timestamp across my project of cause. So I'm not going to make them methods of the class but stand alone function.
How can I achieve this?
EDIT: I would take Option2, and still open to other answers.
Option-1: Code below should work (you do not need to have a class in order to define self):
def pget_timestamp(self):
return datetime.utcfromtimestamp(self._created)
def pset_timestamp(self, dt):
self._created = time.mktime(dt.timetuple())
class Group(Base):
__tablename__ = 'group'
id = Column(Integer, primary_key=True)
_created = Column('created', Integer, nullable=False)
created = synonym(_created,
descriptor=property(pget_timestamp, pset_timestamp),
)
Option-2: If you do need the same on many classes, leverage Mixins
from sqlalchemy.ext.declarative import declared_attr
class _CreatedMixin(object):
_created = Column('created', Integer, nullable=False)
def pget_timestamp(self):
return datetime.utcfromtimestamp(self._created)
def pset_timestamp(self, dt):
self._created = time.mktime(dt.timetuple())
#declared_attr
def created(cls):
return synonym('_created',
descriptor=property(cls.pget_timestamp, cls.pset_timestamp),
)
class Group(_CreatedMixin, Base):
# #note: adding *_CreatedMixin* to bases defines both the column and the synonym
__tablename__ = 'group'
id = Column(Integer, primary_key=True)
Alternatively, if this is for all your classes, you could make _CreatedMixin a base class for all your models:
Base = declarative_base(engine, cls=_CreatedMixin)
class Group(Base):
__tablename__ = 'group'
id = Column(Integer, primary_key=True)
Option-3: You could do any of the above using Hybrid Attributes
Note: make your set/get functions in-sync: either both or none use UTC-enabled functionality. Currently (unless you are in UTC-0) setting one value to created will not return the same one back.
I'm now using a different implementation. It's not related to the original title, but in case you need it.
Use sqlalchemy.types.TypeDecorator. Defining a table with sqlalchemy with a mysql unix timestamp
class UTCTimestampType(TypeDecorator):
impl = Integer
def process_bind_param(self, value, dialect):
if value is None:
return None # support nullability
elif isinstance(value, datetime):
return int(time.mktime(value.timetuple()))
raise ValueError("Can operate only on datetime values. Offending value type: {0}".format(type(value).__name__))
def process_result_value(self, value, dialect):
if value is not None: # support nullability
return datetime.fromtimestamp(float(value))
class ModelA(Base):
__tablename__ = 'model_a'
id = Column(Integer, primary_key=True)
created = Column(UTCTimestampType, nullable=False)
issues about alembic. Alembic: How to migrate custom type in a model?
# manually change the line
sa.Column('created', sa.UTCTImestampType(), nullable=False),
# to
sa.Column('created', sa.Integer(), nullable=False),
This question already has an answer here:
Creating container relationship in declarative SQLAlchemy
(1 answer)
Closed 3 years ago.
I have already finished a good bit of my python/Elixir interface on my existing database. I am now considering to drop Elixir and move everything into pure SQLAlchemy, most likely wanting to use Declarative methods.
I am not sure where to even start with this particular inheritance relationship. I don't think sqlalchemy performs inheritance in this manner (or as "magically"), and I am a bit confused how the same would look in sqlalchemy:
This is a polymorphic multi-table join, with each class mapped to its own database table. When finished, another class (not included here) will have a OneToMany with 'Comp'. The Comp subclasses have a Primary Key that is a Foreign key to Comp.id.
class Comp(Entity):
using_options(inheritance='multi')
parent = ManyToOne('Assembly', onupdate='cascade', ondelete='set null')
quantity = Field(Numeric(4), default=1)
def __repr__(self):
return "<Comp>"
## If not familiar with Elixir, each of the following "refid" point to a different
## table depending on its class. This is the primary need for polymorphism.
class CompAssm(Comp):
using_options(inheritance='multi')
refid = ManyToOne('Assembly', onupdate='cascade', ondelete='set null')
def __repr__(self):
return "<CompAssm>"
class CompItem(Comp):
using_options(inheritance='multi')
refid = ManyToOne('Item', onupdate='cascade')
def __repr__(self):
return "<CompItem>"
class CompLabor(Comp):
using_options(inheritance='multi')
refid = ManyToOne('Labor', onupdate='cascade')
def __repr__(self):
return "<CompLabor>"
I think this is the general direction, but may still need tweaking.
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Assembly(Base):
__tablename__ = 'assembly'
assm_id = Column(Integer, primary_key=True)
children = relationship('Comp')
### other assembly stuff
class Comp(Base):
__tablename__ = 'components'
id = Column(Integer, primary_key=True)
comp_type = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': comp_type}
parent = Column(Integer, ForeignKey('assembly.assm_id'))
quantity = Column(Integer)
class CompAssm(Comp):
__tablename__ = 'compassm'
__mapper_args__ = {'polymorphic_identity': 'compassm'}
id = Column(Integer, ForeignKey('components.id'), primary_key=True)
refid = Column(Integer, ForeignKey('assembly.assm_id'))
class CompItem(Comp):
__tablename__ = 'compitem'
__mapper_args__ = {'polymorphic_identity': 'compitem'}
id = Column(Integer, ForeignKey('components.id'), primary_key=True)
refid = Column(Integer, ForeignKey('items.id'))
class CompLabor(Comp):
__tablename__ = 'complabor'
__mapper_args__ = {'polymorphic_identity': 'complabor'}
id = Column(Integer, ForeignKey('components.id'), primary_key=True)
refid = Column(Integer, ForeignKey('labors.id'))
So I have a bunch of tables using SQLAlchemy that are modelled as objects which inherit from the result to a call to declarative_base(). Ie:
Base = declarative_base()
class Table1(Base):
# __tablename__ & such here
class Table2(Base):
# __tablename__ & such here
Etc. I then wanted to have some common functionality available to each of my DB table classes, the easiest way to do this according to the docs is to just do multiple inheritance:
Base = declarative_base()
class CommonRoutines(object):
#classmethod
def somecommonaction(cls):
# body here
class Table1(CommonRoutines, Base):
# __tablename__ & such here
class Table2(CommonRoutines, Base):
# __tablename__ & such here
The thing I don't like about this is A) multiple inheritance in general is a bit icky (gets tricky resolving things like super() calls, etc), B) if I add a new table I have to remember to inherit from both Base and CommonRoutines, and C) really that "CommonRoutines" class "is-a" type of table in a sense. Really what CommonBase is is an abstract base class which defines a set of fields & routines which are common to all tables. Put another way: "its-a" abstract table.
So, what I'd like is this:
Base = declarative_base()
class AbstractTable(Base):
__metaclass__ = ABCMeta # make into abstract base class
# define common attributes for all tables here, like maybe:
id = Column(Integer, primary_key=True)
#classmethod
def somecommonaction(cls):
# body here
class Table1(AbstractTable):
# __tablename__ & Table1 specific fields here
class Table2(AbstractTable):
# __tablename__ & Table2 specific fields here
But this of course doesn't work, as I then have to A) define a __tablename__ for AbstractTable, B) the ABC aspect of things causes all sorts of headaches, and C) have to indicate some sort of DB relationship between AbstractTable and each individual table.
So my question: is it possible to achieve this in a reasonable way? Ideally I'd like to enforce:
No multiple inheritance
CommonBase/AbstractTable be abstract (ie cannot be instantiated)
SQLAlchemy version 0.7.3 introduced the __abstract__ directive which is used for abstract classes that should not be mapped to a database table, even though they are subclasses of sqlalchemy.ext.declarative.api.Base. So now you create a base class like this:
Base = declarative_base()
class CommonRoutines(Base):
__abstract__ = True
id = Column(Integer, primary_key=True)
def __init__(self):
# ...
Notice how CommonRoutines doesn't have a __tablename__ attribute. Then create subclasses like this:
class Foo(CommonRoutines):
__tablename__ = 'foo'
name = Column(...)
def __init__(self, name):
super().__init__()
self.name = name
# ...
This will map to the table foo and inherit the id attribute from CommonRoutines.
Source and more information: http://docs.sqlalchemy.org/en/rel_0_7/orm/extensions/declarative.html#abstract
It is pretty straigh-forward, you just make declarative_base() to return a Base class which inherits from your CommonBase using cls= parameter. Also shown in Augmenting The Base docs. Your code might then look similar to below:
class CommonBase(object):
#classmethod
def somecommonaction(cls):
# body here
Base = declarative_base(cls=CommonBase)
class Table1(Base):
# __tablename__ & Table1 specific fields here
class Table2(Base):
# __tablename__ & Table2 specific fields here
You can use AbstractConcreteBase to make an absract base model:
from sqlalchemy.ext.declarative import AbstractConcreteBase
class AbstractTable(AbstractConcreteBase, Base):
id = db.Column(db.Integer, primary_key=True)
#classmethod
def somecommonaction(cls):
# body here
If you want to have several models with common columns, then you can use __abstract__ and #declared_attr to inherit shared table attributes. Example:
Base = declarative_base()
class CommonRoutines(Base):
__abstract__ = True
id = Column(Integer, primary_key=True)
modified_at = Column(DateTime)
#declared_attr
def modified_by(self):
# `user.id` is another table called `user` with an `id` field
return Column(Integer, ForeignKey('user.id', name='fk_modified_by_user_id'))
def __init__(self):
self.modified_by = None
super().__init__()
class Foo(CommonRoutines):
__tablename__ = 'foo'
name = Column(...)
With this solution you will have a Foo table with the fields of Foo class (name) and the ones in CommonRoutines (id, modified_at and modified_by)