Hi im having some trouble with foreign key in sqlalchemy not auto incrementing on a primary key ID
Im using: python 2.7, pyramid 1.3 and sqlalchemy 0.7
Here is my models
class Page(Base):
__tablename__ = 'page'
id = Column(Integer, ForeignKey('mapper.object_id'), autoincrement=True, primary_key=True)
title = Column(String(30), unique=True)
title_slug = Column(String(75), unique=True)
text = Column(Text)
date_added = Column(DateTime)
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(100), unique=True)
email = Column(String(100), unique=True)
password = Column(String(100))
class Group(Base):
__tablename__ = 'groups'
id = Column(Integer, primary_key=True)
name = Column(String(100), unique=True)
class Member(Base):
__tablename__ = 'members'
user_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
group_id = Column(Integer, ForeignKey('groups.id'), primary_key=True)
class Resource(Base):
__tablename__ = 'resource'
id = Column(Integer, primary_key=True)
tablename = Column(Text)
action = Column(Text)
class Mapper(Base):
__tablename__ = 'mapper'
resource_id = Column(Integer, ForeignKey('resource.id'), primary_key=True)
group_id = Column(Integer, ForeignKey('groups.id'), primary_key=True)
object_id = Column(Integer, primary_key=True)
and here is my RAW SQL query which i've written in SQLAlchemys ORM
'''
SELECT g.name, r.action
FROM groups AS g
INNER JOIN resource AS r
ON m.resource_id = r.id
INNER JOIN page AS p
ON p.id = m.object_id
INNER JOIN mapper AS m
ON m.group_id = g.id
WHERE p.id = ? AND
r.tablename = ?;
'''
obj = Page
query = DBSession().query(Group.name, Resource.action)\
.join(Mapper)\
.join(obj)\
.join(Resource)\
.filter(obj.id == obj_id, Resource.tablename == obj.__tablename__).all()
the raw SQL Query works fine without any relations between Page and Mapper, but SQLAlchemys ORM seem to require a ForeignKey link to be able to join them. So i decided to put the ForeignKey at Page.id since Mapper.object_id will link to several different tables.
This makes the SQL ORM query with the joins work as expected but adding new data to the Page table results in a exception.
FlushError: Instance <Page at 0x3377c90> has a NULL identity key.
If this is an auto- generated value, check that the database
table allows generation of new primary key values, and that the mapped
Column object is configured to expect these generated values.
Ensure also that this flush() is not occurring at an inappropriate time,
such as within a load() event.
here is my view code:
try:
session = DBSession()
with transaction.manager:
page = Page(title, text)
session.add(page)
return HTTPFound(location=request.route_url('home'))
except Exception as e:
print e
pass
finally:
session.close()
I really don't know why, but i'd rather have the solution in SQLalchemy than doing the RAW SQL since im making this project for learning purposes :)
I do not think autoincrement=True and ForeignKey(...) play together well.
In any case, for join to work without any ForeignKey, you can just specify the join condition in the second parameter of the join(...):
obj = Page
query = DBSession().query(Group.name, Resource.action)\
.join(Mapper)\
.join(Resource)\
.join(obj, Resource.tablename == obj.__tablename__)\
.filter(obj.id == obj_id)\
.all()
Related
Context: I'm making an auctioning website for which I am using Flask-SQLAlchemy. My tables will need to have a many-to-many relationship (as one artpiece can have many user bids and a user can bid on many artpieces)
My question is: it is possible to add another column to my joining table to contain the id of the user bidding, the id of artpiece that they are bidding on and also how much they bid? Also if yes, how would I include this bid in the table when I add a record to said table?
bid_table = db.Table("bid_table",
db.Column("user_id", db.Integer, db.ForeignKey("user.user_id")),
db.Column("item_id", db.Integer, db.ForeignKey("artpiece.item_id"))
)
class User(db.Model):
user_id = db.Column(db.Integer, unique=True, primary_key=True, nullable=False)
username = db.Column(db.Integer, unique=True, nullable=False)
email = db.Column(db.String(50), unique =True, nullable=False)
password = db.Column(db.String(60), nullable=False)
creation_date = db.Column(db.DateTime, default=str(datetime.datetime.now()))
bids = db.relationship("Artpiece", secondary=bid_table, backref=db.backref("bids", lazy="dynamic"))
class Artpiece(db.Model):
item_id = db.Column(db.Integer, unique=True, primary_key=True, nullable=False)
artist = db.Column(db.String(40), nullable=False)
buyer = db.Column(db.String(40), nullable=False)
end_date = db.Column(db.String(40))
highest_bid = db.Column(db.String(40))
It is possible to do this with SQL Alchemy, but it's very cumbersome in my opinion.
SQLAlchemy uses a concept called an Association Proxy to turn a normal table into an association table. This table can have whatever data fields you want on it, but you have to manually tell SQLAlchemy which columns are foreign keys to the other two tables in question.
This is a good example from the documentation.
In your case, the UserKeyword table is the association proxy table that you want to build for your user/bid scenario.
The special_key column is the arbitrary data you would store like the bid amount.
from sqlalchemy import Column, Integer, String, ForeignKey
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.orm import backref, declarative_base, relationship
Base = declarative_base()
class User(Base):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
name = Column(String(64))
# association proxy of "user_keywords" collection
# to "keyword" attribute
keywords = association_proxy('user_keywords', 'keyword')
def __init__(self, name):
self.name = name
class UserKeyword(Base):
__tablename__ = 'user_keyword'
user_id = Column(Integer, ForeignKey('user.id'), primary_key=True)
keyword_id = Column(Integer, ForeignKey('keyword.id'), primary_key=True)
special_key = Column(String(50))
# bidirectional attribute/collection of "user"/"user_keywords"
user = relationship(User,
backref=backref("user_keywords",
cascade="all, delete-orphan")
)
# reference to the "Keyword" object
keyword = relationship("Keyword")
def __init__(self, keyword=None, user=None, special_key=None):
self.user = user
self.keyword = keyword
self.special_key = special_key
class Keyword(Base):
__tablename__ = 'keyword'
id = Column(Integer, primary_key=True)
keyword = Column('keyword', String(64))
def __init__(self, keyword):
self.keyword = keyword
def __repr__(self):
return 'Keyword(%s)' % repr(self.keyword)
Check out the full documentation for instructions on how to access and create this kind of model.
Having used this in a real project, it's not particularly fun and if you can avoid it, I would recommend it.
https://docs.sqlalchemy.org/en/14/orm/extensions/associationproxy.html
I have two classes like:
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String, nullable=False, unique = True)
class Order(Base):
__tablename__ = 'orders'
id = Column(Integer, primary_key=True)
user_id = Column(Integer, ForeignKey('Users.id'))
sent = Column(Boolean, default=False)
user = relationship('User', backref=backref('orders'))
def __init__(self, user=None, sent=False):
self.user = user
self.sent = sent
When I execute the following code :
u = session.query(User)[0]
o = Order()
u.orders.append(o)
session.commit()
SqlAlachemy issues an insert statement on the orders table. When I execute :
del u.orders[1]
session.commit()
sqlalchemy issues an update statement on the orders table setting user_id to NULL.
I need SqlAlchemy to issue a delete statement and the records to disappear from the database, not just being set to NULL. I've tried setting the cascade='save-update' keyword argument in the backref call, but did not succeed. I thing there must be some keyword argument configuration to achieve this, but I could not find it yet.
How could I reach the explained behaviour ? Namely, the issue of delete statements on the child tables?
I have a table with some text fields and a lot of many to many relationships. I'm having trouble querying only some of the fields. Here's how I query the table:
with db.session(raise_err=True) as session:
result = session.query(
ClientKnowledge
).options(
*[defaultload(getattr(ClientKnowledge, table.table.name))
.load_only(KqFactory().get_tables([table.table.name])[0].get_pk().name)
for table in ClientKnowledge.__mapper__.relationships]
).filter(
ClientKnowledge.id == 1
)
This works fine, SQLAlchemy is doing all the joins for me and uses the lazy=joinedload param well.
When I specify fields in the query parameters I have to do all the joins myself, but I don't want that; also tried to use with_entities() and to make the query from the other side of the relationship, but got the same result.
Is there a way I can query only specific fields without losing SQLAlchemy's ability to make the joins for me?
PS: I can't really give a complete working example since all my tables are dynamically generated. Please tell me if more context is needed.
EDIT:
here is an example of how my tables looks like
#as_declarative()
class RightTable1:
id = Column(Integer, primary_key=True)
desc = Text()
#as_declarative()
class RightTable2:
id = Column(Integer, primary_key=True)
desc = Text()
#as_declarative()
class Association1:
__tablename__ = 'association1'
left_id = Column(Integer, ForeignKey(ClientKnowledge.rel_field1), primary_key=True)
right_id = Column(Integer, ForeignKey(RightTable1.id), primary_key=True)
#as_declarative()
class Association2:
__tablename__ = 'association2'
left_id = Column(Integer, ForeignKey(ClientKnowledge.rel_field2), primary_key=True)
right_id = Column(Integer, ForeignKey(RightTable2.id), primary_key=True)
#as_declarative()
class ClientKnowledge:
id = Column(Integer, primary_key=True)
text_field = Text()
rel_field1 = relationship(
kq_table, secondary=Association1.__table__, lazy='joined',
backref=backref(RightTable1.__table__.name, lazy='joined')
)
rel_field2 = relationship(
kq_table, secondary=Association2.__table__, lazy='joined',
backref=backref(RightTable2.__table__.name, lazy='joined')
)
what i want is being able to query ClientKnowledge.rel_field1.
I have tried:
session.query(ClientKnowledge.rel_field1)
But i have to do all the join myself.
I have rather simple models like these:
TableA2TableB = Table('TableA2TableB', Base.metadata,
Column('tablea_id', BigInteger, ForeignKey('TableA.id')),
Column('tableb_id', Integer, ForeignKey('TableB.id')))
class TableA(Base):
__tablename__ = 'TableA'
id = Column(BigInteger, primary_key=True)
infohash = Column(String, unique=True)
url = Column(String)
tablebs = relationship('TableB', secondary=TableA2TableB, backref='tableas')
class TableB(Base):
__tablename__ = 'TableB'
id = Column(Integer, primary_key=True)
url = Column(String, unique=True)
However, sqla generates queries like
SELECT "TableB".id, "TableB".url AS "TableB_url" FROM "TableB", "TableA2TableB"
WHERE "TableA2TableB".tableb_id = "TableB".id AND "TableA2TableB".tablea_id = 408997;
But why is there a cartesian product in the query when the attributes selected are those in TableB? TableA2TableB shouldn't be needed.
Thanks
As it is right now, there is a backref relationship in TableB (tableas) and it's loaded because the default loading mode is set to select.
You may want to change the TableA.tablebs to
tablebs = relationship('TableB', secondary=TableA2TableB, backref='tableas', lazy="dynamic")
Im using sqlalchemy to design a forum style website. I started knocking out the design but everytime I try to test it with a few inserts, it dumps a brick;
NoForeignKeysError: Could not determine join condition between parent/child
tables on relationship Thread.replies - there are no foreign keys linking
these tables. Ensure that referencing columns are associated with a
ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
Here are my "models"
from sqlalchemy import Integer, Column, String, create_engine, ForeignKey
from sqlalchemy.orm import relationship, sessionmaker, backref
from .database import Base # declarative base instance
class User(Base):
__tablename__ = "user"
id = Column(Integer, primary_key=True)
username = Column(String, unique=True)
email = Column(String, unique=True)
threads = relationship("Thread", backref="user")
posts = relationship("Post", backref="user")
class Post(Base):
__tablename__ = "post"
id = Column(Integer, primary_key=True)
name = Column(String)
body = Column(String)
author = Column(Integer, ForeignKey("user.id"))
class Thread(Base):
__tablename__ = "thread"
id = Column(Integer, primary_key=True)
name = Column(String)
desc = Column(String)
replies = relationship("Post", backref="thread")
author_id = Column(Integer, ForeignKey("user.id"))
board_id = Column(Integer, ForeignKey("board.id"))
class Board(Base):
__tablename__ = "board"
id = Column(Integer, primary_key=True)
name = Column(String)
desc = Column(String)
threads = relationship("Thread", backref="board")
category_id = Column(Integer, ForeignKey("category.id"))
class Category(Base):
__tablename__ = "category"
id = Column(Integer, primary_key=True)
name = Column(String)
desc = Column(String)
threads = relationship("Board", backref="category")
engine = create_engine('sqlite:///:memory:', echo=True)
Base.metadata.create_all(engine)
session_factory = sessionmaker(bind=engine)
session = session_factory()
Your Post model has no thread reference. Add a column to Post referencing the Thread a post belongs to:
class Post(Base):
__tablename__ = "post"
id = Column(Integer, primary_key=True)
name = Column(String)
body = Column(String)
author = Column(Integer, ForeignKey("user.id"))
thread_id = Column(Integer, ForeignKey('thread.id'))
We can't use the name thread because that's what the Post.replies relationship will add to retrieved Thread instances.
This is a One to Many relationship as documented in the SQLAlchemy Relationship Configuration documentation.
You should add a field in the Post model that reads:
thread_id = Column(Integer, ForeignKey("thread.id"), nullable=True, default=None)
How is SQLAlchemy supposed to know how this relationship you defined supposed to link a thhread to a post? That's why you should have a foreign key from a post to its thread. You can allow it to be null, if it does not belong to a thread, it depends on your use case.