Develop Reference

I have a back end task which im not doing well , i would appreciate if you help me

For the back-end, attached you will be able to find an XML file, where you need:
To create a parser in nodejs/php/python to read xml
Creating a MYSQL database (schema) saves the xml data
To use ORM to communicate with the database
The script should handle insert/update/delete
No need for frontend (CLI is enough)
I tried to solve this in python but im stuck at the function i need to create to store data from XML to Database table.
# import xml element tree
import xml.etree.ElementTree as ET
# import mysql connector
import mysql.connector
from sqlalchemy import create_engine, Column, Integer, String
from sqlalchemy.orm import sessionmaker
from sqlalchemy.ext.declarative import declarative_base
# give the connection parameters
# user name is root
# password is 2634687
# server is localhost
database = mysql.connector.connect(user='root', password='2634687', host='localhost')
# reading xml file , file name is dataset.xml
tree = ET.parse('dataset.xml')
# creating the cursor object
c = database.cursor()
# c.execute("CREATE DATABASE testingdb")
# print("testdb Data base is created")
# Connect to a MySQL database
engine = create_engine('mysql+pymysql://root:2634687#localhost/testingdb', echo=True)
# Define the Product model
Base = declarative_base()
class Product(Base):
__tablename__ = 'Product'
productId = Column(Integer, primary_key=True)
cedi = Column(String(100))
childWeightFrom = Column(String(100))
childWeightTo = Column(Integer)
color_code = Column(Integer)
color_description = Column(String(100))
countryImages = Column(String(100))
defaultSku = Column(String(100))
preferredEan = Column(Integer)
sapAssortmentLevel = Column(String(100))
sapPrice = Column(Integer)
season = Column(String(100))
showOnLineSku = Column(String(100))
size_code = Column(String(100))
size_description = Column(String(100))
skuID = Column(Integer)
skuName = Column(String(100))
stateOfArticle = Column(String(100))
umSAPprice = Column(String(10))
volume = Column(Integer)
weight = Column(Integer)
# Create the users table
Base.metadata.create_all(engine)
# Create a session to interact with the database
Session = sessionmaker(bind=engine)
session = Session()
# Insert a new product
new_product = Product(cedi='CD01')
session.add(new_product)
session.commit()
# Update the user's age
Product = session.query(Product).filter_by(cedi='CD01').first()
Product.childWeightFrom = 31
session.commit()
# Delete the user
session.delete(Product)
session.commit()

Create one-to-one and one-to-many between two tables in sqlalchemy

What I'm trying to do seems simple. I'd like to have a parent Organization which has child Groups. However, one group will be the "main_group". Each Organization should have a main group. Each Group should have a reference to the Organization in which it belongs.
I've been able to create_all and use these Models but in my tests when I do a drop_all, I get a
sqlalchemy.exc.CircularDependencyError: Can't sort tables for DROP; an unresolvable foreign key dependency exists between tables: groups, organizations. Please ensure that the ForeignKey and ForeignKeyConstraint objects involved in the cycle have names so that they can be dropped using DROP CONSTRAINT.
Here is a minimum example to show what I'm trying to do. I've left some commented lines in to show what all I've tried.
from sqlalchemy.orm import relationship
from sqlalchemy.sql.schema import ForeignKey
from sqlalchemy.sql.sqltypes import Boolean, Date, Float
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy import Integer, ForeignKey, String, Column
Base = declarative_base()
class Organization(Base):
__tablename__ = "organizations"
id = Column(Integer(), primary_key=True)
name = Column(String(100))
### Define one-to-one with Group
main_group_id = Column(Integer, ForeignKey("groups.id"))
main_group = relationship(
"Group",
# uselist=False,
# back_populates="organization",
foreign_keys=[main_group_id],
primaryjoin="Group.id==Organization.main_group_id",
post_update=True
# backref="organization"
)
####
## Defines the one-to-many collection with Group
# groups = relationship(
# "Group",
# uselist=True,
# # foreign_keys="Group.organization_id",
# primaryjoin="Group.id==Organization.main_group_id",
# # back_populates="organization",
# )
class Group(Base):
__tablename__ = "groups"
id = Column(Integer, primary_key=True)
name = Column(String(100))
organization_id = Column(Integer, ForeignKey("organizations.id"), nullable=False)
organization = relationship(
"Organization",
uselist=False,
# backref="groups",
foreign_keys=[organization_id],
primaryjoin="Group.organization_id==Organization.id",
# primaryjoin="Group.id==Organization.main_group_id",
post_update=True,
)
from sqlalchemy import create_engine
from sqlalchemy.orm.session import sessionmaker
conn_string = "sqlite:///example.sql"
engine = create_engine(conn_string)
Base.metadata.create_all(engine) # here we create all tables
Session = sessionmaker(bind=engine)
session = Session()
new_org = Organization(name="my new org")
print(new_org)
session.add(new_org)
org = session.query(Organization).get(1)
new_group = Group(name="my main group", organization=org, organization_id=org.id)
new_org.main_group = new_group
session.commit()
Base.metadata.drop_all(engine)

From The Docs
There is actually an example pretty similar to what you want here, which uses an explicitly named foreign key, fk_favorite_entry:
rows-that-point-to-themselves-mutually-dependent-rows
That doesn't seem to fully solve the drop warning, which maybe seems to be dialect dependent. You can fully solve it with use_alter=True which is explained here:
creating-dropping-foreign-key-constraints-via-alter
Likely Solution
Best case would be to name the atypical constraint and to set use_alter=True on it as well.
class Organization(Base):
# ...
main_group_id = Column(Integer, ForeignKey("groups.id", name="fk_main_group", use_alter=True))
# ...

Use a column_property (with a calculation) to define a relationship in SQLAlchemy

I have the following setup:
A set of competitions taking place at given dates, global competitors with birthdates that can be registered for some of these competitions, age classes defined for each competition and at last the registration of a specific competitor for a specific competition. These registrations should finally contain the correct age class. I could achieve this with e.g. hybrid_property, but then I'd run into the N+1 issue. My goal is, to get the registrations eagerly loaded with one large joined query, including the age classes.
Here is what I have tried so far:
from sqlalchemy import create_engine, Column, ForeignKey, Integer, Date, select, func, cast, and_
from sqlalchemy.ext.declarative import declarative_base, declared_attr
from sqlalchemy.orm import scoped_session, sessionmaker, relationship, column_property, joinedload, Load, foreign, remote
import transaction
import inflection
import datetime
import logging
logging.basicConfig()
logging.getLogger('sqlalchemy.engine').setLevel(logging.INFO)
engine = create_engine('sqlite://')
db = scoped_session(sessionmaker())
db.configure(bind=engine)
DBModel = declarative_base()
DBModel.metadata.bind = engine
class Competition(DBModel):
__tablename__ = 'competitions'
id = Column(Integer, primary_key=True)
date = Column(Date, nullable=False)
class Competitor(DBModel):
__tablename__ = 'competitors'
id = Column(Integer, primary_key=True)
birthdate = Column(Date, nullable=False)
class AgeClass(DBModel):
__tablename__ = 'age_classes'
id = Column(Integer, primary_key=True)
competition_id = Column(ForeignKey(Competition.id), nullable=False)
min = Column(Integer, nullable=False)
competition = relationship(Competition)
class Registration(DBModel):
__tablename__ = 'registrations'
id = Column(Integer, primary_key=True)
competition_id = Column(ForeignKey(Competition.id), nullable=False)
competitor_id = Column(ForeignKey(Competitor.id), nullable=False)
competition = relationship(Competition)
competitor = relationship(Competitor)
age = column_property(select([cast((func.julianday(Competition.date) - func.julianday(Competitor.birthdate)) / 365.2425, Integer)]).
where(and_(Competition.id == competition_id,
Competitor.id == competitor_id)).
correlate_except())
age_class_id = column_property(select([AgeClass.id]).
where(and_(AgeClass.competition_id == competition_id,
AgeClass.min <= age)).
order_by(AgeClass.min.desc()).limit(1).
correlate_except(AgeClass))
# age_class = relationship(AgeClass, viewonly=True)
# age_class = relationship(AgeClass, foreign_keys=[age_class_id], viewonly=True)
# age_class = relationship(AgeClass, primaryjoin=AgeClass.id == remote(age_class_id), viewonly=True)
age_class = relationship(AgeClass, primaryjoin=foreign(AgeClass.id) == remote(age_class_id), viewonly=True)
DBModel.metadata.create_all()
with transaction.manager:
competition = Competition(date=datetime.date(2020, 2, 9))
competitor = Competitor(birthdate=datetime.date(2000, 2, 9))
db.add(AgeClass(competition=competition, min=6))
db.add(AgeClass(competition=competition, min=10))
db.add(Registration(competition=competition, competitor=competitor))
query = db.query(Registration).options(joinedload(Registration.competition, innerjoin=True),
joinedload(Registration.competitor, innerjoin=True),
joinedload(Registration.age_class, innerjoin=True),
Load(Registration).raiseload('*'))
print([x.age_class for x in query.first()])
The first two versions of the age_class attribute lead to the error message
sqlalchemy.exc.NoForeignKeysError: Could not determine join condition between parent/child tables on relationship Registration.age_class - there are no foreign keys linking these tables. Ensure that referencing columns are associated with a ForeignKey or ForeignKeyConstraint, or specify a 'primaryjoin' expression.
Hence I tried to specify a primary join condition, which gives me the following error message for versions 3 and 4:
sqlalchemy.exc.ArgumentError: Relationship Registration.age_class could not determine any unambiguous local/remote column pairs based on join condition and remote_side arguments. Consider using the remote() annotation to accurately mark those elements of the join condition that are on the remote side of the relationship.
Is there any chance to get this going or am I trying to do something impossible here? Or is there another way to approach this question?

SqlAlchemy "contains_eager" does not appear to load nested relationships

I have a schema as follows:
Thing # Base class for below tables
- id
Ball (Thing)
- color
Bin (Thing)
- ball -> Ball.id
Court (Thing)
- homeBin -> Bin.id
- awayBin -> Bin.id
I'd like to ensure that whenever I load a set of Courts, it includes the latest Ball column values. From what I understand, contains_eager() might be able to help with that:
Indicate that the given attribute should be eagerly loaded from columns stated manually in the query.
I have a test that queries every few seconds for any Courts. I'm finding that, even with contains_eager, I only ever see the same value for Ball.color, even though I've explicitly updated the column's value in the database.
Why does sqlalchemy appear to reuse this old data?
Below is a working example of what's happening:
from sqlalchemy import *
from sqlalchemy.orm import *
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
Base = declarative_base()
class Thing(Base):
__tablename__ = "Things"
id = Column(Integer, primary_key=True)
name = Column(String(256))
thingType = Column(String(256))
__mapper_args__ = {
'with_polymorphic':'*',
'polymorphic_on':"thingType",
'polymorphic_identity':"thing"
}
class Ball(Thing):
__tablename__ = "Balls"
id = Column('id', Integer, ForeignKey('Things.id'), primary_key=True)
color = Column('color', String(256))
__mapper_args__ = {
'polymorphic_identity':'ball'
}
class Bin(Thing):
__tablename__ = "Bins"
id = Column('id', Integer, ForeignKey('Things.id'), primary_key=True)
shape = Column('shape', String(256))
ballId = Column('ballId', Integer, ForeignKey('Balls.id'))
ball = relationship(Ball, foreign_keys=[ballId], backref="outputBins")
__mapper_args__ = {
'polymorphic_identity':'bin'
}
pass
class Court(Thing):
__tablename__ = "Courts"
id = Column('id', Integer, ForeignKey('Things.id'), primary_key=True)
homeBinId = Column('homeBinId', Integer, ForeignKey('Bins.id'))
awayBinId = Column('awayBinId', Integer, ForeignKey('Bins.id'))
homeBin = relationship(Bin, foreign_keys=[homeBinId], backref="homeCourts")
awayBin = relationship(Bin, foreign_keys=[awayBinId], backref="awayCourts")
__mapper_args__ = {
'polymorphic_identity':'court'
}
metadata = MetaData()
engine = create_engine("postgresql://localhost:5432/")
Session = sessionmaker(bind=engine)
session = Session()
def courtQuery():
awayBalls = aliased(Ball, name="awayBalls")
homeBalls = aliased(Ball, name="homeBalls")
awayBins = aliased(Bin, name="awayBins")
homeBins = aliased(Bin, name="homeBins")
query = session.query(Court)\
.outerjoin(awayBins, Court.awayBinId == awayBins.id)\
.outerjoin(awayBalls, awayBins.ballId == awayBalls.id)\
.outerjoin(homeBins, Court.homeBinId == homeBins.id)\
.outerjoin(homeBalls, homeBins.ballId == homeBalls.id)\
.options(contains_eager(Court.awayBin, alias=awayBins).contains_eager(awayBins.ball, alias=awayBalls))\
.options(contains_eager(Court.homeBin, alias=homeBins).contains_eager(homeBins.ball, alias=homeBalls))
return [r for r in query]
import time
while(True):
results = courtQuery()
court = results[0]
ball = court.homeBin.ball
print(ball.color) # does not change
time.sleep(2)
Environment:
Python 2.7.14
SqlAlchemy 1.3.0b1
PostGres 11.3 (though I've seen this
on Oracle as well)

Dictionary of tags in declarative SQLAlchemy?

I am working on a quite large code base that has been implemented using sqlalchemy.ext.declarative, and I need to add a dict-like property to one of the classes. What I need is the same as in this question, but in a declarative fashion. Can anyone with more knowledge in SQLAlchemy give me an example?
Thanks in advance...

Declarative is just another way of defining things. Virtually you end up with the exact same environment than if you used separated mapping.
Since I answered the other question, I'll try this one as well. Hope it gives more upvotes ;)
Well, first we define the classes
from sqlalchemy import Column, Integer, String, Table, create_engine
from sqlalchemy import orm, MetaData, Column, ForeignKey
from sqlalchemy.orm import relation, mapper, sessionmaker
from sqlalchemy.orm.collections import column_mapped_collection
from sqlalchemy.ext.associationproxy import association_proxy
from sqlalchemy.ext.declarative import declarative_base
engine = create_engine('sqlite:///:memory:', echo=True)
Base = declarative_base(bind=engine)
class Note(Base):
__tablename__ = 'notes'
id_item = Column(Integer, ForeignKey('items.id'), primary_key=True)
name = Column(String(20), primary_key=True)
value = Column(String(100))
def __init__(self, name, value):
self.name = name
self.value = value
class Item(Base):
__tablename__ = 'items'
id = Column(Integer, primary_key=True)
name = Column(String(20))
description = Column(String(100))
_notesdict = relation(Note,
collection_class=column_mapped_collection(Note.name))
notes = association_proxy('_notesdict', 'value', creator=Note)
def __init__(self, name, description=''):
self.name = name
self.description = description
Base.metadata.create_all()
Now let's make a test:
Session = sessionmaker(bind=engine)
s = Session()
i = Item('ball', 'A round full ball')
i.notes['color'] = 'orange'
i.notes['size'] = 'big'
i.notes['data'] = 'none'
s.add(i)
s.commit()
print i.notes
I get:
{u'color': u'orange', u'data': u'none', u'size': u'big'}
Now let's check the notes table...
for note in s.query(Note):
print note.id_item, note.name, note.value
I get:
1 color orange
1 data none
1 size big
It works!! :D