I'd like to create a 1:n relationship between two tables dynamically. My DB model is mapped via SQLAlchemy but due to some special features of my application I can not use the default declarative way.
E.g.
class Foo(Base):
id = Column(Integer, autoincrement=True, primary_key=True)
flag = Column(Boolean)
class Bar(Base):
id = Column(Integer, autoincrement=True, primary_key=True)
foo_id = Column(Integer, ForeignKey('foo.id'))
# declarative version:
# foo = relationship(Foo)
So I want to add relationship named "foo" to the mapped class "Bar" after Bar was defined and SQLAlchemy did its job of defining a mapper etc.
Update 2017-09-05: Why is this necessary for me? (I thought I could omit this because I think it mostly distracts from the actual problem to solve but since there were comments abouts it...)
First of all I don't have a single database but hundreds/thousands. Data in old databases must not be altered in any way but I want a single source code to access old data (even though data structure and calculation rules change significantly).
Currently we use multiple model definitions. Later definitions extend/modify previous ones. Often we manipulate SQLAlchemy models dynamically. We try not to have code in mapped classes because we think it will be much harder ensuring correctness of that code after changing a table many times (code must work in every intermediate step).
In many cases we extend tables (mapped classes) programatically in model X after it was initially defined in model X-1. Adding columns to an existing SQLAlchemy ORM class is manageable. Now we are adding a new reference column an existing table and a relationship() provides a nicer Python API.
Well, my question above is again a nice example of SQLAlchemy's super powers (and my limited understanding):
Bar.__mapper__.add_property('foo', relationship('Foo'))
Likely I was unable to get this working initially because some of my surrounding code mixed adding relationships and columns. Also there is one important difference to declaring columns:
Column('foo', Integer)
For columns the first parameter can be the column name but you can not use this for relationships. relationship('foo', 'Foo') triggers exceptions when passing it to .add_property().
Related
I'm experimenting with SQLModel (https://sqlmodel.tiangolo.com/) and I get to the point that I had to create a composite index between several fields and I can't how to do it using SQLModel library.
Db Model
The only work around I found was to use directly sqlalchemy Index, rather than index=true (from SQLModel documentation when creating indexes for unique fields - )
class Jump(SQLModel, table=True):
"""
SQL Table abstraction: Jump
Contains data belonging to a connection between a questionnaire-version and another
questionnaire-version
"""
origin_name: str = Field(primary_key=True)
origin_version: int = Field()
destination_name: str = Field()
__table_args__ = (
Index(
"compound_index_origin_name_version_destination_name", "origin_name", "origin_version", "destination_name"
),
)
This is not a "workaround". This is exactly how it is supposed to be done (as of now). The idea behind SQLModel presumably is to provide a toolkit for constructing table models that is very familiar to people coming from SQLAlchemy, while also providing most of the goodies coming from Pydantic models.
In some cases, SQLModel obviously does things differently and in some regards it tries to simplify existing interfaces. E.g. providing the foreign_key parameter on the Field constructor, so that you don't need to import and instantiate ForeignKey from SQLAlchemy.
But in this case, I really don't see the point in trying to change the existing tools. SQLAlchemy declarative ORM models allow you to set composite indices and other table parameters via the __table_args__ class-attribute. SQLModel's meta class inherits this feature. So why reinvent the wheel? 🙂
Unless you have an idea how to simplify this further. In that case, I am sure Sebastián will be more than happy about a corresponding PR or a suggestion in the issue tracker.
I'm trying to figure out the best way to develop constraints around a set of models
class ParentDefinition:
child_definition = relationship()
class ChildDefinition:
parent_definition_id = ForeignKey()
class Parent:
parent_definition = relationship()
parent_definition_id = ForeignKey()
class Child:
parent = relationship()
parent_id = ForeignKey()
child_definition = relationship()
child_definition_id = ForeignKey()
I want to ensure that Child.child_definition_id == Child.parent.parent_definition.child_definition.id but I'm not sure the best way to do that.
I know that this probably isn't the best model design but there are pre-existing architecture considerations I'm working around.
Any help would be appreciated!
According to the documentation here you need to define constraint to a table or a column. As far as I know, SQL constraints can't be setup over multiple table.
If you really want to go through constraints, you can use another table that handle the association between each entities constrainable fields (ids of each items and add your constraint to id_parent_def = id_child_def).
If you don't want to create this table, you can always use listeners so you can check before inserting data but this may be inefficient for your needs.
Another way would to use database specific functions or triggers that could check conditions for you, I guess you would write plain SQL inside your migrations file to create them (or manually in your DB).
This is a follow-up to a post I opened regarding the creation of dynamic tables and columns using SQLALCHEMY. SQLAlchemy create dynamic tables and columns
Most of the tutorials that show how to create dynamic Tables and columns in Python and SQLAlchemy focus on creating a class object - using Declarative, such that every use of a class creates a table with the pre-defined columns and column metadata defined in that class (see below). The problem with this way of creating tables is that at least as I understand it, does not allow me to create columns dynamically as well. I am reading an API and attaining metadata for tables which can change daily, therefore I need a method of creating these tables dynamically - see my orignal post: SQLAlchemy create dynamic tables and columns
The following construct for instance does not allow for creating dynamic columns for tables, as far as I understand it, such like my original post:
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
name = Column(String)
fullname = Column(String)
password = Column(String)
The reason I ask about class objects in this regard, is because the tutorials that show how to query the database require the use of a class object in the query.
So for instance, every method I've researched of querying the database to find the lastest value in each table (maximum id assuming it is the primary key or maximum timestamp) seems to require the use of a class object which currently I am not doing.
Session = sessionmaker(bind=engine)
session = Session()
User.query.order_by('timestamp desc').limit(1)
Is it possible to Is it possible to construct a class object that will create dynamic columns? Using a class object ?
If not, Is there another best-practice for querying the database using SQLAlchemy to find the latest record in a table ?
See this blog post on SQLAlchemy - Great tutorial, i've read.
I am working on implementing sets of sqlalchemy declarative models for multiple database schemas. These database schemas are highly similar in table design. I would like to be able to reuse as much of the common model code (column definitions, relations etc.) as possible across the sets of models for each schema.
(Please note, the schemas in question are 99% identical.)
As a small example:
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Integer, String
Base = declarative_base()
class Person(Base):
__tablename__ = 'person'
name = Column(String, primary_key=True)
age = Column(Integer)
If I now want to introduce a new attribute Person column 'height' that only exists in one schema and not the other, how can I go about defining it whilst reusing the existing Person column definitions?
So far, I have tried using declarative mixins to define the common functionality and then mix these into classes that inherit from different schema-specific bases. However, this is unsatisfactory because even the majority of identical models have to be declared as mixins, and a declarative class declared once for schema (inheriting from the appropriate base and the mixin). Given that the schemas are similar this seems like a lot of boilerplate code.
Is there a better way to do it?
I'm trying to reuse a primary key in one of my tables with SQLAlchemy and am getting foreign key constraint error.
In a nutshell:
PostgreSQL 8.4
Python 2.7
SQLAlchemy 0.7
I have 3 tables: User, Inventories and Devices. Inventories and Devices have a one-to-one relationship with User. User.id is Inventories.user_id and Devices.user_id foreign keyed.
I've got User, Devices and Inventories set up in models/ according to standard python practices.
Within interactive python I can issue the following commands no problem:
>>>newUser = User.create()
>>>newUser.device = User.create_device(<*args>)
>>>Session.add(newUser)
>>>Session.commit()
(an inventory record is automatically created in code)
Now, let's say I want to re-use User record 1 (it's the only record that will allow a method called reset in code for security and internal testing reasons)
>>>oldUser = User.retrieve(1)
>>>Session.delete(oldUser)
>>>Session.commit()
(confirm that user 1 no longer exists)
>>>newUser = User.create()
>>>newUser.device = User.create_device(<*args>)
>>>newUser.id = 1
>>>Session.add(newUser)
>>>Session.commit()
At this point I'll either get an eror that Key(id)=(<id>) is still referenced from table "devices" (or "inventories") where <id> is the newUser.id before re-assigning it to be id 1
I've looked into cascading and have tried the various options (all, save-update, etc) with no effect.
Any information pointing to where I'm going wrong would greatly be appreciated,
Thanks,
Krys
To address the error you're seeing, you could update the foreign keys on all of the Device and Inventory models associated with that User model before committing. You'll have to make sure that your User model doesn't auto-increment the id (i.e., that it isn't a PostgreSQL sequence).
For example, the SQLAlchemy model declaration should be
class User(base):
__tablename__ = 'user'
id = Column('id', Integer, primary_key=True, unique=True, nullable=False)
instead of
class User(base):
__tablename__ = 'user'
id = Column('id', Integer, Sequence('user_id_seq'), primary_key=True)
BUT, this is probably not the right way to do it! It would be a better design to use a sequence on User.id (like in the second model declaration), and add another field on the user table that indicates if the user is an admin (for the security/testing purposes you mentioned). This way you don't have to rely on magic numbers in your application (e.g., the user id) for application logic, especially security.
I ma not using SQLAlchemy, so i do not have a proper answer, but i can say that you must ask yourself what you want is really necessary?
Because,
You probably will break the data integrity, and that may couse serious problems.
You will need to break the auto-increment structure of the ID, so until then, you have to assign id's by hand or use a hand-written pre-save trigger to get a proper id.
If you have tables that have a User foreginkey that sets NOT null, thn you probably will have problem with freeing records related to a deleted user. If you do not null them, a re-used id will create a serious data-integrity problem (wrongly referanced relations)...
So first of all, you must decide if it worth it?
Since this is a problem that shouldn't be seen in production, just use SET CONSTRAINTS. You could use INITIALLY DEFERRED on your FOREIGN KEYs but I wouldn't recommend that since you're not dealing with a cyclic dependency that exists in production.