So I have these two simple classes inside the same python file which I'm trying to map with SQLAlchemy, where User column 'password_id' is a foreign key to the table Password 'password_id' column as well
from sqlalchemy.orm import relationship, declarative_base
from sqlalchemy import Column, Integer, String
from sqlalchemy.sql.schema import ForeignKey
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
user_id = Column(Integer, primary_key=True)
name = Column(String)
email = Column(String)
password_id = Column(Integer, ForeignKey('password.password_id'))
parent = relationship("Password", back_populates="users")
class Password(Base):
__tablename__ = 'passwords'
password_id = Column(Integer, primary_key=True)
password = Column(String)
last_change_date = Column(DateTime)
valid_until = Column(DateTime)
child = relationship("User", back_populates="passwords", uselist=False)
Here's the db schema for context:
I'm following this guide from sql alchemy but for some reason I keep getting the error from the title 'Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.', which is strange because accorind to what I see in the guide, my classes have been mapped correctly so I can't understand why this error is happening.
Thank you in advance
I think the issue is in the following line (the table name in ForeignKey)...
password_id = Column(Integer, ForeignKey('password.password_id'))
should be passwords instead of password.
I'm starting with FastAPI and SQLAlchemy, and I have a question about loading models in the correct order to satisfy SQLAlchemy models relationship. I've got two models, Profile and Account:
Profile:
from sqlalchemy import Column, Integer, String
from sqlalchemy.orm import relationship
from project.core.database import Base
class Profile(Base):
__tablename__ = "profiles"
id = Column(Integer, primary_key=True)
name = Column(String(10), nullable=False)
slug = Column(String(10), nullable=False)
order = Column(Integer, unique=True, nullable=False)
accounts = relationship(
"Account", foreign_keys="[Account.profile_id]", back_populates="profile"
)
Account:
from sqlalchemy import Column, Integer, String, LargeBinary, ForeignKey
from sqlalchemy.orm import relationship
from project.core.database import Base
class Account(Base):
__tablename__ = "accounts"
id = Column(Integer, primary_key=True)
name = Column(String(255), nullable=False)
email = Column(String(255), nullable=False)
password = Column(LargeBinary, nullable=False)
profile_id = Column(Integer, ForeignKey("profiles.id"))
profile = relationship(
"Profile", foreign_keys=[profile_id], back_populates="accounts"
)
And in my project.core.database file, I created a method to import the models, since I was having issues with models not being located when the relationship was attempted to be made.
def import_all():
import project.models.profile
import project.models.account
def init_db():
import_all()
My question is, is there a smarter way to load the models in the correct order? Because now I only have two models, but soon it can grow to dozens of models and I think this will become such a monster to manage.
I looked for resources and examples, but everything I found created the models in a single file.
Thanks in advance!
Let's take a look at full-stack-fastapi-postgresql, a boilerplate template created by the author of FastAPI:
base.py
from app.db.base_class import Base
from app.models.item import Item
from app.models.user import User
init_db.py
from app.db import base # noqa: F401 import db models
# make sure all SQL Alchemy models are imported (app.db.base) before initializing DB
# otherwise, SQL Alchemy might fail to initialize relationships properly
# for more details: https://github.com/tiangolo/full-stack-fastapi-postgresql/issues/28
def init_db(db: Session) -> None:
...
So, as you can see it's a polucalar way to import models, here you can find additional discussion: https://github.com/tiangolo/full-stack-fastapi-postgresql/issues/28
I am trying to create an application using SQLAlchemy. It worked fine as long as I only had one file with one Class. Now I want to have multiple classes/tables in different files. I stumbled upon this question, and tried to do it like it was suggested there: I now have three files
base.py
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
blind.py
from sqlalchemy import Column, String
from .base import Base
class Blind(Base):
__tablename__ = 'blinds'
blind = Column(String)
data_processor_uuid = Column(String, primary_key=True)
data_source_uuid = Column(String)
timestamp = Column(String, primary_key=True)
and data.py
from sqlalchemy import Column, Integer, String, Float
from .base import Base
class Datum(Base):
__tablename__ = 'data'
data_source_uuid = Column(Integer, primary_key=True)
sensor_type = Column(String)
timestamp = Column(String, primary_key=True)
value = Column(Float)
I now want to initialize the database in db_setup.py using
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
from .base import Base
engine = create_engine('sqlite:///test.db', echo=True)
Base.metadata.bind = engine
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
def get_db_session():
return session
This works, however, it does not create the tables in the database as expected. When I try to insert something into the table, I get an error saying "table does not exist". Can someone tell me what I am doing wrong here?
The problem was that I wasn't importing the class definitions for Blinds and Datum anywhere, so they weren't evaluated! Before I split them up into different files, I had imported them to get to Base. Thanks to #IljaEverilä for this answer!
How does one generate the SQL/Migrate Code/Whatever with SQLAlchemy when using the Declarative Base?
from sqlalchemy import *
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
engine = create_engine('mysql://root:password#localhost/mydb_dev', echo=True)
metadata = MetaData(bind=engine)
class User(Base):
__table__ = Table("users", metadata, autoload=True)
id = Column(Integer, primary_key=True)
display_name = Column(String)
email = Column(String)
password = Column(String)
def __repr__(self):
return "<User(id='{}', display_name='{}', email='{}')>".format(self.id, self.display_name, self.email)
class Site(Base):
__table__ = Table("sites", metadata, autoload=True)
id = Column(Integer, primary_key=True)
name = Column(String)
urls = relationship("URL")
def __repr__(self):
return "<Site(id='{}', name='{}')>".format(self.id, self.name)
I have this so far, I'd like to see what SQLAlchemy would generate as a schema.
Or, does SQLAlchemy do this at all? Is this a case where I create and manage the database and it's schema separately, and I just update my entities to reflect it?
Do understand that I am used to Doctrine2, and im very new to SQLAlchemy
Thanks!
You can create the db models by calling:
metadata.create_all()
http://docs.sqlalchemy.org/en/latest/core/metadata.html#sqlalchemy.schema.MetaData.create_all
Note that this only works for the creation of previously non-existent models, and doesn't handle updates or downgrades. Check out alembic for even finer control:
http://alembic.zzzcomputing.com/en/latest/
Simplified, I have the following class structure (in a single file):
Base = declarative_base()
class Item(Base):
__tablename__ = 'item'
id = Column(BigInteger, primary_key=True)
# ... skip other attrs ...
class Auction(Base):
__tablename__ = 'auction'
id = Column(BigInteger, primary_key=True)
# ... skipped ...
item_id = Column('item', BigInteger, ForeignKey('item.id'))
item = relationship('Item', backref='auctions')
I get the following error from this:
sqlalchemy.exc.InvalidRequestError
InvalidRequestError: When initializing mapper Mapper|Auction|auction, expression
'Item' failed to locate a name ("name 'Item' is not defined"). If this is a
class name, consider adding this relationship() to the Auction class after
both dependent classes have been defined.
I'm not sure how Python cannot find the Item class, as even when passing the class, rather than the name as a string, I get the same error. I've been struggling to find examples of how to do simple relationships with SQLAlchemy so if there's something fairly obvious wrong here I apologise.
This all turned out to be because of the way I've set SQLAlchemy up in Pyramid. Essentially you need to follow this section to the letter and make sure you use the same declarative_base instance as the base class for each model.
I was also not binding a database engine to my DBSession which doesn't bother you until you try to access table metadata, which happens when you use relationships.
if it's a subpackage class, add Item and Auction class to __init__.py in the subpackage.
The SQLAlchemy documentation on Importing all SQLAlchemy Models states in part:
However, due to the behavior of SQLAlchemy's "declarative" configuration mode, all modules which hold active SQLAlchemy models need to be imported before those models can successfully be used. So, if you use model classes with a declarative base, you need to figure out a way to get all your model modules imported to be able to use them in your application.
Once I imported all of the models (and relationships), the error about not finding the class name was resolved.
Note: My application does not use Pyramid, but the same principles apply.
Case with me
Two models defined in separate files, one is Parent and the other is Child, related with a Foreign Key. When trying to use Child object in celery, it gave
sqlalchemy.exc.InvalidRequestError: When initializing mapper Mapper|Child|child, expression 'Parent' failed to locate a name ("name 'Parent' is not defined"). If this is a class name, consider adding this relationship() to the <class 'app.models.child'>
parent.py
from app.models import *
class Parent(Base):
__tablename__ = 'parent'
id = Column(BigInteger, primary_key=True, autoincrement=True)
name = Column(String(60), nullable=False, unique=True)
number = Column(String(45), nullable=False)
child.py
from app.models import *
class Child(Base):
__tablename__ = 'child'
id = Column(BigInteger, primary_key=True, autoincrement=True)
parent_id = Column(ForeignKey('parent.id'), nullable=False)
name = Column(String(60), nullable=False)
parent = relationship('Parent')
Solution
Add an import statement for Parent in beginning of child.py
child.py (modified)
from app.models import *
from app.models.parent import Parent # import Parent in child.py 👈👈
class Child(Base):
__tablename__ = 'child'
id = Column(BigInteger, primary_key=True, autoincrement=True)
parent_id = Column(ForeignKey('parent.id'), nullable=False)
name = Column(String(60), nullable=False)
parent = relationship('Parent')
Why this worked
The order in which models get loaded is not fixed in SQLAlchemy.
So, in my case, Child was being loaded before Parent. Hence, SQLAlchemy can't find what is Parent. So, we just imported Parent before Child gets loaded.
Namaste 🙏
I've solved the same error by inheriting a 'db.Model' instead of 'Base'... but I'm doing the flask
Eg:
from flask_sqlalchemy import SQLAlchemy
db = SQLAlchemy()
class someClass(db.Model):
someRelation = db.relationship("otherClass")
Also, even though this doesn't apply to the OP, for anyone landing here having gotten the same error, check to make sure that none of your table names have dashes in them.
For example, a table named "movie-genres" which is then used as a secondary in a SQLAlchemy relationship will generate the same error "name 'movie' is not defined", because it will only read as far as the dash. Switching to underscores (instead of dashes) solves the problem.
My Solution
One models file, or even further, if you need.
models.py
from sqlalchemy import Boolean, BigInteger, Column, DateTime, Float, ForeignKey, BigInteger, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
from .parent import Parent
from .child import Child
parent.py
from sqlalchemy import Boolean, BigInteger, Column, DateTime, Float, ForeignKey, BigInteger, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
#Base = declarative_base()
class Parent(Base):
__tablename__ = 'parent'
id = Column(BigInteger, primary_key=True, autoincrement=True)
name = Column(String(60), nullable=False, unique=True)
number = Column(String(45), nullable=False)
child.py
from sqlalchemy import Boolean, BigInteger, Column, DateTime, Float, ForeignKey, BigInteger, Integer, String
from sqlalchemy.orm import relationship
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Child(Base):
__tablename__ = 'child'
id = Column(BigInteger, primary_key=True, autoincrement=True)
parent_id = Column(ForeignKey('parent.id'), nullable=False)
name = Column(String(60), nullable=False)
parent = relationship('Parent')
Why this worked
Same Deepam answer, but with just one models.py file to import another models
I had a different error, but the answers in here helped me fix it.
The error I received:
sqlalchemy.exc.InvalidRequestError: When initializing mapper mapped class Parent->parents, expression 'Child' failed to locate a name ('Child'). If this is a class name, consider adding this relationship() to the <class 'parent.Parent'> class after both dependent classes have been defined.
My set-up is similar toDeepam's answer.
Briefly what I do different:
I have multiple separate .py files for each db.Model.
I use a construct/fill database .py file that pre-fills db.Model objects in either Multi-threading or single threading way
What caused the error:
Only in multi-threaded set up the error occured
This construct/fill .py script did import Parent, but not Child.
What fixed it:
Adding an import to Child fixed it.
I had yet another solution, but this helped clue me in. I was trying to implement versioning, from https://docs.sqlalchemy.org/en/14/orm/examples.html#versioning-objects using the "history_mapper" class.
I got this same error. All I had to do to fix it was change the order in which my models were imported.
Use back_populates for relationship mapping in both models.
Also keep in mind to import both the models in the models/__init__.py
Base = declarative_base()
class Item(Base):
__tablename__ = 'item'
id = Column(BigInteger, primary_key=True)
# ... skip other attrs ...
auctions = relationship('Auction', back_populates='item')
class Auction(Base):
__tablename__ = 'auction'
id = Column(BigInteger, primary_key=True)
# ... skipped ...
item_id = Column('item', BigInteger, ForeignKey('item.id'))
item = relationship('Item', back_populates='auctions')