Consider the following sqlalchemy model that validates the primary_email field:
from sqlalchemy import Column, String
from sqlalchemy.orm import validates
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.dialects import postgresql
from sqlalchemy.ext.mutable import MutableList
Base = declarative_base()
class Test(Base):
id = Column(Integer, primary_key=True)
primary_email = Column(String(128))
#validates('primary_email')
def valid_email(self, key, value):
if not '#' in value:
raise ValueError(f"Invalid {key}: '{value}'")
return value
def __init__(self, email):
primary_email = email
It works as expected but now I want to add another column
secondary_email = Column(MutableList.as_mutable(postgresql.ARRAY(String(128))))
How do I perform validation on secondary_email.append(email) call?
When I add 'secondary_email' to validates decorator, it only gets triggered on assignment operations like
self.secondary_email = ['example#me.com']
I believe this is something that can be achieved with a custom class but I wonder if there is a simpler and more generic solution?
Related
Take the following code:
from sqlalchemy import create_engine
from sqlalchemy.orm import declarative_base
from sqlalchemy import Column, Integer, String
engine = create_engine('postgresql://postgres:password#localhost:5432/db', echo=True, echo_pool='debug')
Base = declarative_base()
class Item(Base):
__tablename__ = 'items'
id = Column(Integer, primary_key=True)
name = Column(String)
def __repr__(self):
return "<Item(id=%s, name='%s')>" % (self.id, self.name)
item = Item(name="computer")
Is there a way to get a python dict of the Item with all its fields? For example, I would like to get the following returned:
item.to_dict()
{"id": None, "name": "computer"}
Or do I have to write my own method to do that?
Here would be one way to do it:
class MyBase(Base):
__abstract__ = True
def to_dict(self):
return {field.name:getattr(self, field.name) for field in self.__table__.c}
class Item(MyBase):
# as before
item = Item(name="computer")
item.to_dict()
# {'id': None, 'name': 'computer'}
Also, a lot of these usability simplifications can be found in: https://github.com/absent1706/sqlalchemy-mixins.
Here is an absurd problem with sqlalchemy that seems easy! First, this is my config file for connecting to mysql database:
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
engine = create_engine('mysql://root:#localhost:3306/digi')
and then, I am trying to create a table called 'sale-history' :
from config import *
from sqlalchemy import *
class Sale(Base):
__tablename__ = 'sale-history'
order_id = column(Integer, primary_key= True)
customer_id = column(Integer)
item_id = column(Integer) #froeign key with product list
cartFinalize_dateTime = column(DATETIME)
amount_ordrered = column(Integer)
city_name = column(String(191))
quantity_ordered = column(Integer)
def __repr__(self):
return "<Sale(city_name='%s')>" % (self.city_name)
Sale.__table__
Base.metadata.create_all(engine)
Now, what I wonder is that
Sale.__table__
and
Base.metadata.create_all(engine)
are not known to my code. More accurate, these are not in suggestion options showed by pycharm editor. Debugging the code does not throw any error(returns 0). What should I do to create tables?
I appreciate your consideration so much!
The code is using column to define columns in the table but it should be using Column (note the upper-case "C").
A few tips/comments
Pycharm may provide better support if you avoid the from module import * idiom. You can alias module names if they are to long to type, for example import sqlalchemy as sa
You can see the SQL generated by the engine by passing echo=True to create_engine
Tablenames with hyphens need to be quoted with backticks to be valid. Sqlalchemy does this automatically, but other applications may not. Using underscores instead may be more convenient.
The final code might look like this:
config
from sqlalchemy import create_engine
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
engine = create_engine('mysql://root:#localhost:3306/test', echo=True)
Model
import sqlachemy as sa
import config
class Sale(Base):
__tablename__ = 'sale-history'
order_id = sa.Column(sa.Integer, primary_key=True)
customer_id = sa.Column(sa.Integer)
item_id = sa.Column(sa.Integer) # foreign key with product list
cartFinalize_dateTime = sa.Column(sa.DATETIME)
amount_ordrered = sa.Column(sa.Integer)
city_name = sa.Column(sa.String(191))
quantity_ordered = sa.Column(sa.Integer)
def __repr__(self):
return "<Sale(city_name='%s')>" % (self.city_name)
Base.metadata.create_all(config.engine)
I'm defining an PostgreSQL table using an SQLAlchemy declarative base, like this:
from sqlalchemy import Column, String, BigInteger
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class MyTable(Base):
__tablename__ = 'my_table'
id = Column('id', BigInteger, primary_key=True)
some_string = Column('some_string', String(256), nullable=False)
The nullable constraint guarantees that some_string cannot be null. However, I'd additionally like to give some_string a minimum length, or just forbid it from being the empty string. How can I do this?
Ideally we want the validation to be applied both at the database layer via a constraint and also in our model by rejecting attempts to set the property to a string that's too short even before trying to save it to the database. We can do the former with a CheckConstraint using the char_length function, and the latter by adding a validator. Below is an example that enforces a minimum length of 3 characters on some_string:
from sqlalchemy import Column, String, BigInteger
from sqlalchemy.schema import CheckConstraint
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import validates
Base = declarative_base()
class MyTable(Base):
__tablename__ = 'my_table'
id = Column('id', BigInteger, primary_key=True)
some_string = Column('some_string', String(256), nullable=False)
__table_args__ = (
CheckConstraint('char_length(some_string) > 2',
name='some_string_min_length'),
)
#validates('some_string')
def validate_some_string(self, key, some_string) -> str:
if len(some_string) <= 2:
raise ValueError('some_string too short')
return some_string
In order to handle a growing database table, we are sharding on table name. So we could have database tables that are named like this:
table_md5one
table_md5two
table_md5three
All tables have the exact same schema.
How do we use SQLAlchemy and dynamically specify the tablename for the class that corresponds to this? Looks like the declarative_base() classes need to have tablename pre-specified.
There will eventually be too many tables to manually specify derived classes from a parent/base class. We want to be able to build a class that can have the tablename set up dynamically (maybe passed as a parameter to a function.)
OK, we went with the custom SQLAlchemy declaration rather than the declarative one.
So we create a dynamic table object like this:
from sqlalchemy import MetaData, Table, Column
def get_table_object(self, md5hash):
metadata = MetaData()
table_name = 'table_' + md5hash
table_object = Table(table_name, metadata,
Column('Column1', DATE, nullable=False),
Column('Column2', DATE, nullable=False)
)
clear_mappers()
mapper(ActualTableObject, table_object)
return ActualTableObject
Where ActualTableObject is the class mapping to the table.
In Augmenting the Base you find a way of using a custom Base class that can, for example, calculate the __tablename__ attribure dynamically:
class Base(object):
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
The only problem here is that I don't know where your hash comes from, but this should give a good starting point.
If you require this algorithm not for all your tables but only for one you could just use the declared_attr on the table you are interested in sharding.
Because I insist to use declarative classes with their __tablename__ dynamically specified by given parameter, after days of failing with other solutions and hours of studying SQLAlchemy internals, I come up with the following solution that I believe is simple, elegant and race-condition free.
def get_model(suffix):
DynamicBase = declarative_base(class_registry=dict())
class MyModel(DynamicBase):
__tablename__ = 'table_{suffix}'.format(suffix=suffix)
id = Column(Integer, primary_key=True)
name = Column(String)
...
return MyModel
Since they have their own class_registry, you will not get that warning saying:
This declarative base already contains a class with the same class name and module name as mypackage.models.MyModel, and will be replaced in the string-lookup table.
Hence, you will not be able to reference them from other models with string lookup. However, it works perfectly fine to use these on-the-fly declared models for foreign keys as well:
ParentModel1 = get_model(123)
ParentModel2 = get_model(456)
class MyChildModel(BaseModel):
__tablename__ = 'table_child'
id = Column(Integer, primary_key=True)
name = Column(String)
parent_1_id = Column(Integer, ForeignKey(ParentModel1.id))
parent_2_id = Column(Integer, ForeignKey(ParentModel2.id))
parent_1 = relationship(ParentModel1)
parent_2 = relationship(ParentModel2)
If you only use them to query/insert/update/delete without any reference left such as foreign key reference from another table, they, their base classes and also their class_registry will be garbage collected, so no trace will be left.
you can write a function with tablename parameter and send back the class with setting appropriate attributes.
def get_class(table_name):
class GenericTable(Base):
__tablename__ = table_name
ID= Column(types.Integer, primary_key=True)
def funcation(self):
......
return GenericTable
Then you can create a table using:
get_class("test").__table__.create(bind=engine) # See sqlachemy.engine
Try this
import zlib
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Column, Integer, BigInteger, DateTime, String
from datetime import datetime
BASE = declarative_base()
ENTITY_CLASS_DICT = {}
class AbsShardingClass(BASE):
__abstract__ = True
def get_class_name_and_table_name(hashid):
return 'ShardingClass%s' % hashid, 'sharding_class_%s' % hashid
def get_sharding_entity_class(hashid):
"""
#param hashid: hashid
#type hashid: int
#rtype AbsClientUserAuth
"""
if hashid not in ENTITY_CLASS_DICT:
class_name, table_name = get_class_name_and_table_name(hashid)
cls = type(class_name, (AbsShardingClass,),
{'__tablename__': table_name})
ENTITY_CLASS_DICT[hashid] = cls
return ENTITY_CLASS_DICT[hashid]
cls = get_sharding_entity_class(1)
print session.query(cls).get(100)
Instead of using imperative creating Table object, you can use usual declarative_base and make a closure to set a table name as the following:
def make_class(Base, table_name):
class User(Base):
__tablename__ = table_name
id = Column(Integer, primary_key=True)
name= Column(String)
return User
Base = declarative_base()
engine = make_engine()
custom_named_usertable = make_class(Base, 'custom_name')
Base.metadata.create_all(engine)
session = make_session(engine)
new_user = custom_named_usertable(name='Adam')
session.add(new_user)
session.commit()
session.close()
engine.dispose()
just you need to create class object for Base.
from sqlalchemy.ext.declarative import declarative_base, declared_attr
class Base(object):
#declared_attr
def __tablename__(cls):
return cls.__name.lower()
Base = declarative_base(cls=Base)
With SQLAlchemy, it is possible to add a default value to every function. As I understand it, this may also be a callable (either without any arguments or with an optional ExecutionContext argument).
Now in a declarative scenario, I wonder if it is somehow possible to have a default function which is called with the object that is being stored. I.e. possibly like so:
Base = sqlalchemy.ext.declarative.declarative_base()
class BaseEntity(Base):
value = Column('value', String(40), default=BaseEntity.gen_default)
def gen_default(self):
# do something with self, for example
# generate a default value using some other data
# attached to the object
return self.default_value
Is something like this possible? Or do I have to somehow set up an before-insertion hook for this (how?)?
before_insert is documented here:
http://docs.sqlalchemy.org/en/rel_0_7/orm/events.html#sqlalchemy.orm.events.MapperEvents.before_insert
examples here:
http://docs.sqlalchemy.org/en/rel_0_7/orm/events.html#mapper-events
i.e.
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import event
Base= declarative_base()
class A(Base):
__tablename__ = "a"
id = Column(Integer, primary_key=True)
data = Column(String)
otherdata = Column(String)
#event.listens_for(A, "before_insert")
def gen_default(mapper, connection, instance):
instance.data = "Some default %s" % instance.otherdata
e = create_engine("sqlite://")
Base.metadata.create_all(e)
a = A(otherdata="some other data")
s = Session(e)
s.add(a)
s.commit()
assert a.data == "Some default some other data"