It's my first attempt at sqlalchemy. I have a json file with my usr information and I would like to put them in a sqlite3 database file. It works but I find the instance initialization verbose since there are many columns in the table, as you can see below.
Is it possible to use a dictionary as input to initialize User()? Something like a = User(usr)?
import json
from sqlalchemy import *
from sqlalchemy.ext.declarative import declarative_base
engine = create_engine('sqlite:///tutorial.db', echo=True)
Base = declarative_base()
class User(Base):
__tablename__ = 'users'
id = Column(Integer, primary_key=True)
bbs_id = Column(String)
name = Column(String)
sex = Column(String, nullable=False)
city = Column(String)
state = Column(String)
class_type = Column(String, nullable=False)
class_id = Column(String, nullable=False)
latitude = Column(Float)
longitude = Column(Float)
def __repr__(self):
return "<User(bbs_id='%s', name='%s'>" % (self.bbs_id, self.name)
Base.metadata.create_all(engine)
with open('mydata.json') as fin:
usrs = json.load(fin)
usr = usrs[0]
a = User(id=usr['id'], bbs_id=usr['bbs_id'], name=usr['name'])
If you know the property names in the JSON object match the column names of the Python model, you can just change:
a = User(id=usr['id'], bbs_id=usr['bbs_id'], name=usr['name'])
to:
a = User(**usr)
Double-star/dict unpacking passes each key/value pair of the dict as if it were an argument being passed by keyword. Since you didn't override __init__ for your model, it already allows and expects the arguments to be optional and passed by keyword, so this lines up perfectly.
Related
On this question I learned how to set the schema on an ORM definition:
from sqlalchemy import Column, Integer, String
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Notification(Base):
__tablename__ = "dog"
__table_args__ = {"schema": "animal"}
id = Column(Integer, primary_key=True)
name = Column(String)
But I now need to make the schema configurable. I have tried passing the table_args parameter at object creation, but it's still trying the schema I put on the class definition.
The better solution I have found so far is to create a function that returns the class:
function get_notification_class(schema: str):
class Notification(Base):
__tablename__ = "dog"
__table_args__ = {"schema": schema}
id = Column(Integer, primary_key=True)
name = Column(String)
return Notification
And then, to use it
Notification = get_notification_class('animal')
obj = Notification('1', 'doggy')
Please could someone help me with this error? I have actually been really struggling to find solid, simple examples for SQLAlchemy. Whilst there are plenty of Model examples of there is not much examples of how to use these Models.
The Error:
sqlalchemy.exc.NoForeignKeysError:
Could not determine join condition between parent/child tables on relationship Species.sc_genus
- there are no foreign keys linking these tables.
Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
The Code
from sqlalchemy import Integer, Column, String, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relation
Base = declarative_base()
class Genus(Base):
__tablename__ = 'genus'
id = Column(Integer, primary_key=True)
common_name = Column(String)
scientific_name = Column(String)
sc_sub_family = "sc_sub_family"
def __repr__(self):
return "<Genus(common_name='%s')>" % (self.scientific_name)
# Species is a child of Genus
class Species(Base):
__tablename__ = 'species'
id = Column(Integer, primary_key=True)
common_name = Column(String)
scientific_name = Column(String)
sc_genus = relation("Genus", backref="species")
def __repr__(self):
return "<Species(common_name='%s')>" % (self.scientific_name)
def addSpecies(session):
species = Species()
species.common_name = "House Cat"
species.scientific_name = "Felis catus"
genus = Genus()
genus.scientific_name = "Felis"
session.add(genus)
species.sc_genus = genus
session.add(species)
session.commit()
if __name__ == "__main__":
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
## A bunch of stuff to make the connection to the database work.
engine = create_engine('sqlite:///foos.db', echo=True)
Base.metadata.create_all(engine)
Session = sessionmaker(bind=engine)
session = Session()
addSpecies(session)
I needed to specify a foreign key for the relationship.
class Genus(Base):
__tablename__ = 'genus'
id = Column(Integer, primary_key=True)
scientific_name = Column(String)
# sc_sub_family = "sc_sub_family"
def __repr__(self):
return "<Genus(common_name='%s')>" % (self.scientific_name)
# Species is a child of Genus
class Species(Base):
__tablename__ = 'species'
id = Column(Integer, primary_key=True)
common_name = Column(String)
scientific_name = Column(String)
sc_genus = relation("Genus", backref="species")
sc_genus_id = Column(Integer, ForeignKey('genus.id'))
def __repr__(self):
return "<Species(common_name='%s')>" % (self.scientific_name)
I'm trying to create two separate databases in a single file using SQLAlchemy. Here's the code I have so far:
from sqlalchemy import create_engine, Column, String, Integer, inspect
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Table1(Base):
__tablename__ = 'table_1'
id = Column(Integer, primary_key=True)
name = Column(String)
class Table2(Base):
__tablename__ = 'table_2'
id = Column(Integer, primary_key=True)
name = Column(String)
engine1 = create_engine('sqlite:///db1.db')
engine2 = create_engine('sqlite:///db2.db')
Base.metadata.drop_all(bind=engine1)
Base.metadata.drop_all(bind=engine2)
Base.metadata.create_all(bind=engine1)
Base.metadata.create_all(bind=engine2)
print(inspect(engine1).get_table_names())
# ['table_1', 'table_2']
print(inspect(engine2).get_table_names())
# ['table_1', 'table_2']
I want to create only Table1 in db1 and only Table2 in db2; however, I'm getting both tables in both databases.
Is there anyway to resolve this or do I need to create the databases in two separate files.
Your problem isn't caused by trying to create the two databases in the same module. Rather, you are calling create_all() on the same metadata object that has mapped both of the tables. E.g.
print(Base.metadata.tables)
result:
dict_keys(['table_1', 'table_2'])
From the docs about MetaData.create_all():
This method will issue queries that first check for the existence of
each individual table, and if not found will issue the CREATE
statements...
Key point is it checks for the existence of each table. So here:
Base.metadata.create_all(bind=engine1)
Base.metadata.create_all(bind=engine2)
... it first checks for both tables in the db referenced by engine1, doesn't find them and creates them. Then, it checks for both tables in the db referenced by engine2, doesn't find them, and creates them.
There's a couple of options.
Have different Base objects (i.e. a distinct MetaData instance) for each database:
Base1 = declarative_base()
Base2 = declarative_base()
class Table1(Base1):
__tablename__ = 'table_1'
id = Column(Integer, primary_key=True)
name = Column(String)
class Table2(Base2):
__tablename__ = 'table_2'
id = Column(Integer, primary_key=True)
name = Column(String)
engine1 = create_engine('sqlite:///db1.db')
engine2 = create_engine('sqlite:///db2.db')
Base1.metadata.drop_all(bind=engine1)
Base2.metadata.drop_all(bind=engine2)
Base1.metadata.create_all(bind=engine1)
Base2.metadata.create_all(bind=engine2)
print(inspect(engine1).get_table_names())
# ['table_1']
print(inspect(engine2).get_table_names())
# ['table_2']
Or, selectively create the tables while binding to the desired engine:
Base = declarative_base()
class Table1(Base):
__tablename__ = 'table_1'
id = Column(Integer, primary_key=True)
name = Column(String)
class Table2(Base):
__tablename__ = 'table_2'
id = Column(Integer, primary_key=True)
name = Column(String)
engine1 = create_engine('sqlite:///db1.db')
engine2 = create_engine('sqlite:///db2.db')
Base.metadata.drop_all(bind=engine1)
Base.metadata.drop_all(bind=engine2)
Base.metadata.tables['table_1'].create(bind=engine1)
Base.metadata.tables['table_2'].create(bind=engine2)
print(inspect(engine1).get_table_names())
# ['table_1']
print(inspect(engine2).get_table_names())
# ['table_2']
How to add objects in the constructor with relationship? The id is not yet ready when constructor is evaluated. In simpler cases it is possible to just provide a list, calculated beforehand. In the example below I tried to say there is a complex_cls_method, in a way it is more like black box.
from sqlalchemy import create_engine, MetaData, Column, Integer, String, ForeignKey
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy.orm import relationship
from sqlalchemy.orm import sessionmaker
DB_URL = "mysql://user:password#localhost/exampledb?charset=utf8"
engine = create_engine(DB_URL, encoding='utf-8', convert_unicode=True, pool_recycle=3600, pool_size=10)
session = sessionmaker(autocommit=False, autoflush=False, bind=engine)()
Model = declarative_base()
class User(Model):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
simple = Column(String(255))
main_address = Column(String(255))
addresses = relationship("Address",
cascade="all, delete-orphan")
def __init__(self, addresses, simple):
self.simple = simple
self.main_address = addresses[0]
return # because the following does not work
self.addresses = Address.complex_cls_method(
user_id_=self.id, # <-- this does not work of course
key_="address",
value_=addresses
)
class Address(Model):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
keyword = Column(String(255))
value = Column(String(255))
user_id = Column(Integer, ForeignKey('user.id'), nullable=False)
parent_id = Column(Integer, ForeignKey('address.id'), nullable=True)
#classmethod
def complex_cls_method(cls, user_id_, key_, value_):
main = Address(keyword=key_, value="", user_id=user_id_, parent_id=None)
session.add_all([main])
session.flush()
addrs = [Address(keyword=key_, value=item, user_id=user_id_, parent_id=main.id) for item in value_]
session.add_all(addrs)
return [main] + addrs
if __name__ == "__main__":
# Model.metadata.create_all(engine)
user = User([u"address1", u"address2"], "simple")
session.add(user)
session.flush()
# as it can't be done in constructor, these additional statements needed
user.addresses = Address.complex_cls_method(
user_id_=user.id,
key_="address",
value_=[u"address1", u"address2"]
)
session.commit()
The question is, is there syntactically elegant (and technically sound) way to do this with User's constructor, or is it safer to just call a separate method of User class after session.flush to add desired objects to relationships (as in the example code)?
Giving up on constructor altogether is still possible, but less desirable option as resulting signature change would require significant refactorings.
Instead of manually flushing and setting ids etc. you could let SQLAlchemy handle persisting your object graph. You'll just need one more adjacency list relationship in Address and you're all set:
class User(Model):
__tablename__ = 'user'
id = Column(Integer, primary_key=True)
simple = Column(String(255))
main_address = Column(String(255))
addresses = relationship("Address",
cascade="all, delete-orphan")
def __init__(self, addresses, simple):
self.simple = simple
self.main_address = addresses[0]
self.addresses = Address.complex_cls_method(
key="address",
values=addresses
)
class Address(Model):
__tablename__ = 'address'
id = Column(Integer, primary_key=True)
keyword = Column(String(255))
value = Column(String(255))
user_id = Column(Integer, ForeignKey('user.id'), nullable=False)
parent_id = Column(Integer, ForeignKey('address.id'), nullable=True)
# For handling parent/child relationships in factory method
parent = relationship("Address", remote_side=[id])
#classmethod
def complex_cls_method(cls, key, values):
main = cls(keyword=key, value="")
addrs = [cls(keyword=key, value=item, parent=main) for item in values]
return [main] + addrs
if __name__ == "__main__":
user = User([u"address1", u"address2"], "simple")
session.add(user)
session.commit()
print(user.addresses)
Note the absence of manual flushes etc. SQLAlchemy automatically figures out the required order of insertions based on the object relationships, so that dependencies between rows can be honoured. This is a part of the Unit of Work pattern.
I'm building an inheritance table schema like the following:
Spec Code
class Person(Base):
__tablename__ = 'people'
id = Column(Integer, primary_key=True)
discriminator = Column('type', String(50))
updated = Column(DateTime, server_default=func.now(), onupdate=func.now())
name = Column(String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
class Engineer(Person):
__mapper_args__ = {'polymorphic_identity': 'engineer'}
start_date = Column(DateTime)
class Manager(Person):
__mapper_args__ = {'polymorphic_identity': 'manager'}
start_date = Column(DateTime)
UPDATED (WORKING) CODE
import os
import sys
from sqlalchemy import Column, create_engine, ForeignKey, Integer, String, DateTime
from sqlalchemy.orm import sessionmaker
from sqlalchemy.sql import func
from sqlalchemy.ext.declarative import declarative_base
try:
os.remove('test.db')
except FileNotFoundError:
pass
engine = create_engine('sqlite:///test.db', echo=True)
Session = sessionmaker(engine)
Base = declarative_base()
class People(Base):
__tablename__ = 'people'
discriminator = Column('type', String(50))
__mapper_args__ = {'polymorphic_on': discriminator}
id = Column(Integer, primary_key=True)
name = Column(String(50))
updated = Column(DateTime, server_default=func.now(), onupdate=func.now())
class Engineer(People):
__tablename__ = 'engineer'
__mapper_args__ = {'polymorphic_identity': 'engineer'}
id = Column(Integer, ForeignKey('people.id'), primary_key=True)
kind = Column(String(100), nullable=True)
Base.metadata.create_all(engine)
session = Session()
e = Engineer()
e.name = 'Mike'
session.add(e)
session.flush()
session.commit()
# works when updating the object
e.name = "Doug"
session.add(e)
session.commit()
# works using the base class for the query
count = session.query(People).filter(
People.name.is_('Doug')).update({People.name: 'James'})
# fails when using the derived class
count = session.query(Engineer).filter(
Engineer.name.is_('James')).update({Engineer.name: 'Mary'})
session.commit()
print("Count: {}".format(count))
Note: this is slightly modified example from sql docs
If I try to update the name for Engineer two things should happen.
update statement to the People table on column name
automatic trigger of update to the updated column on the People table
For now, i'd like to focus on number 1. Things like the example below (as also documented in the full code example) will result in invalid SQL
session.query(Engineer).filter(
Engineer.name.is_('James')).update({Engineer.name: 'Mary'})
I believe the above generates the following:
UPDATE engineer SET name=?, updated=CURRENT_TIMESTAMP FROM people WHERE people.name IS ?
Again, this is invalid. The statement is trying to update rows in incorrect table. name is in the base table.
I'm a little unclear about how inheritance tables should work but it seems like updates should work transparently with the derived object. Meaning, when I update Engineer.name querying against the Engineer object SQLAlchemy should know to update the People table. To complicate things a bit more, what happens if I try to update columns which exist in two tables
session.query(Engineer).filter(
Engineer.name.is_('James')).update({Engineer.name: 'Mary', Engineer.start_date: '1997-01-01'})
I suspect SQLAlchemy will not issue two update statements.