I'm trying to set up a database with a few specific fields (and I can't move away from the specification). One of the fields would be a column called metadata, but sqlalchemy prevents that:
sqlalchemy.exc.InvalidRequestError: Attribute name 'metadata' is reserved for the MetaData instance when using a declarative base class.
Is there a decent workaround for this? Do I need to monkeypatch the declarative_base function to rename the metadata attribute? I couldn't find an option to rename that attribute in the api docs.
Here's some example code that will fail with the above error:
#!/usr/bin/env python3.7
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from sqlalchemy import Column, Integer
class CustomBase(object):
#declared_attr
def __tablename__(cls):
return cls.__name__.lower()
DBBase = declarative_base(cls=CustomBase)
class Data(DBBase):
id = Column(Integer, primary_key=True)
metadata = Column(Integer)
if __name__ == "__main__":
print(dir(Data()))
You can use like:
class Data(DBBase):
id = Column(Integer, primary_key=True)
# metadata = Column(Integer)
metadata_ = Column("metadata", Integer)
The constructor of Column class has a name parameter. You can find it from https://docs.sqlalchemy.org/en/13/core/metadata.html#sqlalchemy.schema.Column
The name field may be omitted at construction time and applied later
In other words, you could write a name as you want originally.
I am trying to use an external library which defines a class model in my own program. I want the classes I define to be the same in all respects to their parents from the library, except that I want to append some helper methods to my local extensions. For example:
External Library:
Base = declarative_base()
class BaseUser(Base):
__tablename__ = 'user'
email = Column(String(100), nullable=False, unique=True)
password = Column(String(128), nullable=False)
address_uid = Column(Integer, ForeignKey('address.uid'))
address = relationship('BaseAddress', back_populates="users")
def __init__(self, **kwargs):
super().__init__(**kwargs)
self.address = BaseAddress()
class BaseAddress(Base):
__tablename__ = 'address'
street = Column(String(100))
unit = Column(String(32))
city = Column(String(64))
state = Column(String(32))
postal = Column(String(32))
country = Column(String(32))
users = relationship('user', back_populates="address")
Local model:
class User(BaseUser):
def in_country(county):
return self.address.country == country
class Address(BaseAddress):
pass
The goal here is to create subclasses which sqlalchemy need not distinguish from their parents. If I insert an Address into User.address, for example, sqlalchemy should not complain about a type mismatch (Address instead of the expected BaseAddress).
The only way of doing this that I can discern would involve using polymorphic_on in the parent classes. I don't want to do this, because it doesn't accurately model what is happening. It would require a discriminator, and it might behave strangely in the event I used a migration script locally. Is there a way with sqlalchemy to achieve polymorphism (I think it's called "ad-hoc polymorphism") without using discriminators, or some other way of achieving my goal?
UPDATE
I believe I could get part of the way there by using enable_typechecks=False on the relationships. However, this doesn't exactly get me what I want. I'd like to be able to do things like User.query() and get back a User rather than a BaseUser.
Let's say I have the following structure (using Flask-SqlAlchemy):
class User(db.Model):
id = db.Column(db.Integer, primary_key=True)
name = db.Column(db.String, nullable=False, index=True)
# The following line throws an error at runtime.
variant = db.Column(db.Integer, nullable=False, index=True,
default=select(func.count(User.id)).where(User.name == self.name))
def __init__(self, name):
super(User, self).__init__()
self.name = name
#property
def clause(self):
return '/'.join([str(self.variant), self.name])
Problem is, "User is not defined." I would like to model a system with Users who may choose the same name but add a field to differentiate between users in a systemic way without using (thereby exposing) the "id" field.
Anyone know how to make a self-referential query to use to populate a default value?
The issue of the default not referring to User here is solved by just assigning "default" to the Column once User is available. However, that's not going to solve the problem here because "self" means nothing either, there is no User method being called so you can't just refer to "self". The challenge with this statement is that you want it to be rendered as an inline sub-SELECT but it still needs to know the in-memory value of ".name". So you have to assign that sub-SELECT per-object in some way.
The usual way people approach ORM-level INSERT defaults like this is usually by using a before_insert handler.
Another way that's worth pointing out is by creating a SQL level INSERT trigger. This is overall the most "traditional" approach, as here you need to have access to the row being inserted; triggers define a means of getting at the row values that are being inserted.
As far as using a default at the column level, you'd need to use a callable function as the default which can look at the current value of the row being inserted, but at the moment that means that your SELECT statement will not be rendered inline with the INSERT statement, you'd need to pre-execute the SELECT which is not really what we want here.
Anyway, the basic task of rendering a SQL expression into the INSERT while also having that SQL expression refer to some local per-object state is achieved by assigning that expression to the attribute, the ORM picks up on this at flush time. Below we do this in the constructor, but this can also occur inside of before_insert() as well:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False, index=True)
variant = Column(Integer, nullable=False, index=True)
def __init__(self, name):
self.name = name
self.variant = select([func.count(User.id)]).where(User.name == self.name)
e = create_engine("sqlite://", echo=True)
Base.metadata.create_all(e)
s = Session(e)
s.add(User(name='n1'))
s.commit()
s.add(User(name='n1'))
s.commit()
print s.query(User.variant).all()
Simplified, I have the following class structure (in a single file):
Base = declarative_base()
class Item(Base):
__tablename__ = 'item'
id = Column(BigInteger, primary_key=True)
# ... skip other attrs ...
class Auction(Base):
__tablename__ = 'auction'
id = Column(BigInteger, primary_key=True)
# ... skipped ...
item_id = Column('item', BigInteger, ForeignKey('item.id'))
item = relationship('Item', backref='auctions')
I get the following error from this:
sqlalchemy.exc.InvalidRequestError
InvalidRequestError: When initializing mapper Mapper|Auction|auction, expression
'Item' failed to locate a name ("name 'Item' is not defined"). If this is a
class name, consider adding this relationship() to the Auction class after
both dependent classes have been defined.
I'm not sure how Python cannot find the Item class, as even when passing the class, rather than the name as a string, I get the same error. I've been struggling to find examples of how to do simple relationships with SQLAlchemy so if there's something fairly obvious wrong here I apologise.
This all turned out to be because of the way I've set SQLAlchemy up in Pyramid. Essentially you need to follow this section to the letter and make sure you use the same declarative_base instance as the base class for each model.
I was also not binding a database engine to my DBSession which doesn't bother you until you try to access table metadata, which happens when you use relationships.
if it's a subpackage class, add Item and Auction class to __init__.py in the subpackage.
The SQLAlchemy documentation on Importing all SQLAlchemy Models states in part:
However, due to the behavior of SQLAlchemy's "declarative" configuration mode, all modules which hold active SQLAlchemy models need to be imported before those models can successfully be used. So, if you use model classes with a declarative base, you need to figure out a way to get all your model modules imported to be able to use them in your application.
Once I imported all of the models (and relationships), the error about not finding the class name was resolved.
Note: My application does not use Pyramid, but the same principles apply.
Case with me
Two models defined in separate files, one is Parent and the other is Child, related with a Foreign Key. When trying to use Child object in celery, it gave
sqlalchemy.exc.InvalidRequestError: When initializing mapper Mapper|Child|child, expression 'Parent' failed to locate a name ("name 'Parent' is not defined"). If this is a class name, consider adding this relationship() to the <class 'app.models.child'>
parent.py
from app.models import *
class Parent(Base):
__tablename__ = 'parent'
id = Column(BigInteger, primary_key=True, autoincrement=True)
name = Column(String(60), nullable=False, unique=True)
number = Column(String(45), nullable=False)
child.py
from app.models import *
class Child(Base):
__tablename__ = 'child'
id = Column(BigInteger, primary_key=True, autoincrement=True)
parent_id = Column(ForeignKey('parent.id'), nullable=False)
name = Column(String(60), nullable=False)
parent = relationship('Parent')
Solution
Add an import statement for Parent in beginning of child.py
child.py (modified)
from app.models import *
from app.models.parent import Parent # import Parent in child.py 👈👈
class Child(Base):
__tablename__ = 'child'
id = Column(BigInteger, primary_key=True, autoincrement=True)
parent_id = Column(ForeignKey('parent.id'), nullable=False)
name = Column(String(60), nullable=False)
parent = relationship('Parent')
Why this worked
The order in which models get loaded is not fixed in SQLAlchemy.
So, in my case, Child was being loaded before Parent. Hence, SQLAlchemy can't find what is Parent. So, we just imported Parent before Child gets loaded.
Namaste 🙏
I've solved the same error by inheriting a 'db.Model' instead of 'Base'... but I'm doing the flask
Eg:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class someClass(db.Model):
someRelation = db.relationship("otherClass")
Also, even though this doesn't apply to the OP, for anyone landing here having gotten the same error, check to make sure that none of your table names have dashes in them.
For example, a table named "movie-genres" which is then used as a secondary in a SQLAlchemy relationship will generate the same error "name 'movie' is not defined", because it will only read as far as the dash. Switching to underscores (instead of dashes) solves the problem.
My Solution
One models file, or even further, if you need.
models.py
from sqlalchemy import Boolean, BigInteger, Column, DateTime, Float, ForeignKey, BigInteger, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from .parent import Parent
from .child import Child
parent.py
from sqlalchemy import Boolean, BigInteger, Column, DateTime, Float, ForeignKey, BigInteger, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
#Base = declarative_base()
class Parent(Base):
__tablename__ = 'parent'
id = Column(BigInteger, primary_key=True, autoincrement=True)
name = Column(String(60), nullable=False, unique=True)
number = Column(String(45), nullable=False)
child.py
from sqlalchemy import Boolean, BigInteger, Column, DateTime, Float, ForeignKey, BigInteger, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Child(Base):
__tablename__ = 'child'
id = Column(BigInteger, primary_key=True, autoincrement=True)
parent_id = Column(ForeignKey('parent.id'), nullable=False)
name = Column(String(60), nullable=False)
parent = relationship('Parent')
Why this worked
Same Deepam answer, but with just one models.py file to import another models
I had a different error, but the answers in here helped me fix it.
The error I received:
sqlalchemy.exc.InvalidRequestError: When initializing mapper mapped class Parent->parents, expression 'Child' failed to locate a name ('Child'). If this is a class name, consider adding this relationship() to the <class 'parent.Parent'> class after both dependent classes have been defined.
My set-up is similar toDeepam's answer.
Briefly what I do different:
I have multiple separate .py files for each db.Model.
I use a construct/fill database .py file that pre-fills db.Model objects in either Multi-threading or single threading way
What caused the error:
Only in multi-threaded set up the error occured
This construct/fill .py script did import Parent, but not Child.
What fixed it:
Adding an import to Child fixed it.
I had yet another solution, but this helped clue me in. I was trying to implement versioning, from https://docs.sqlalchemy.org/en/14/orm/examples.html#versioning-objects using the "history_mapper" class.
I got this same error. All I had to do to fix it was change the order in which my models were imported.
Use back_populates for relationship mapping in both models.
Also keep in mind to import both the models in the models/__init__.py
Base = declarative_base()
class Item(Base):
__tablename__ = 'item'
id = Column(BigInteger, primary_key=True)
# ... skip other attrs ...
auctions = relationship('Auction', back_populates='item')
class Auction(Base):
__tablename__ = 'auction'
id = Column(BigInteger, primary_key=True)
# ... skipped ...
item_id = Column('item', BigInteger, ForeignKey('item.id'))
item = relationship('Item', back_populates='auctions')
I'm looking for a way to describe an associative object the declarative way. Beyond storing the foreign keys in the association table, I need to store information like the creation date of the association.
Today, my model looks like that :
# Define the User class
class User(Base):
__tablename__ = 'users'
# Define User fields
id = schema.Column(types.Integer(unsigned=True),
schema.Sequence('users_seq_id', optional=True), primary_key=True)
password = schema.Column(types.Unicode(64), nullable=False)
# Define the UserSubset class
class UserSubset(Base):
__tablename__ = 'subsets'
# Define UserSubset fields
id = schema.Column(types.Integer(unsigned=True),
schema.Sequence('subsets_seq_id', optional=True), primary_key=True)
some_short_description = schema.Column(types.Unicode(50), nullable=False)
# Define the subset memberships table
subset_memberships = schema.Table('group_memberships', Base.metadata,
schema.Column('user_id', types.Integer(unsigned=True), ForeignKey('users.id')),
schema.Column('subset_id', types.Integer(unsigned=True), ForeignKey('subsets.id')),
schema.Column('created', types.DateTime(), default=now, nullable=False),
)
Can I connect everything in an associative object ? Or should I change stop using the declarative way ?
What you are using at the moment is just a Many-to-Many-relation. How to work with association objects is described in the docs.
There is also an extension called associationproxy which simplifies the relation.
As you can see in the manual, configuring a one to many relation is really simple:
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String(50))
addresses = relation("Address", backref="user")
class Address(Base):
__tablename__ = 'addresses'
id = Column(Integer, primary_key=True)
email = Column(String(50))
user_id = Column(Integer, ForeignKey('users.id'))
Many to many relations isn't much harder:
There’s nothing special about many-to-many with declarative. The secondary argument to relation() still requires a Table object, not a declarative class. The Table should share the same MetaData object used by the declarative base:
keywords = Table('keywords', Base.metadata,
Column('author_id', Integer, ForeignKey('authors.id')),
Column('keyword_id', Integer, ForeignKey('keywords.id'))
)
class Author(Base):
__tablename__ = 'authors'
id = Column(Integer, primary_key=True)
keywords = relation("Keyword", secondary=keywords)
You should generally not map a class and also specify its table in a many-to-many relation, since the ORM may issue duplicate INSERT and DELETE statements.
Anyway, what you seem to be doing might be better served with inheritance. Of course, there can be complex table relations that will be a pathological case for the declarative way, but this doesn't seem to be one of them.
One more thing, code comments should state what the following code does ans why, not how it does it. Having a # Define the User class comment is almost like having a line of code saying a = 1 # assing value 1 to variable "a".