I have query in sqlserver and I wanted to write a model corresponding to it using sqlalchemy orm. I need to know how to convert ON [PRIMARY] into sqlalchemy.
Here is my query:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[UserModel](
[userid] [nvarchar](255) NULL,
[username] [nvarchar](255) NULL,
[serialnumber] [nvarchar](255) NULL
) ON [PRIMARY]
GO
Here is my model:
class UserModel(db.Model):
__tablename__ = 'UserModel'
userid = Column('userid', Unicode(255))
username = Column('username', Unicode(255))
serialnumber = Column('serialnumber', Unicode(255))
Any suggestions, how would I achieve this
Currently I m getting the following error as below:
sqlalchemy.exc.ArgumentError: Mapper Mapper|UserModel|UserModel could not assemble any primary key columns for mapped table 'UserModel'
I know there there is no primary primary key defined in my model but the above query is working fine. So I wanted to translate the above query using sqlalchemy orm.
You cannot translate the above DDL using most ORMs, at least directly, because they expect Python objects to correspond to uniquely identifiable rows in the database. Now, if you know for sure that for example UserModel.userid uniquely identifies rows, you can instruct SQLAlchemy to use it as a "primary key":
class UserModel(db.Model):
__tablename__ = 'UserModel'
userid = Column('userid', Unicode(255))
username = Column('username', Unicode(255))
serialnumber = Column('serialnumber', Unicode(255))
__mapper_args__ = {
'primary_key': [userid]
}
Keep in mind that you've lied to the ORM and any consequences are on you.
As to defining the filegroup the table should be stored in using ON [PRIMARY], you'll have to augment the mssql dialect a bit:
from sqlalchemy import Table
from sqlalchemy.schema import CreateTable
from sqlalchemy.ext.compiler import compiles
# Add our custom dialect specific argument
Table.argument_for('mssql', 'on', None)
#compiles(CreateTable, 'mssql')
def compile_create_table(create, compiler, **kw):
stmt = compiler.visit_create_table(create, **kw)
filegroup = create.element.kwargs.get('mssql_on')
if filegroup:
stmt = stmt.rstrip() # Prettier output, remove newlines
filegroup = compiler.preparer.quote(filegroup)
stmt = f"{stmt} ON {filegroup}\n\n"
return stmt
and add the necessary table args to your model definition:
class UserModel(db.Model):
...
__table_args__ = {
'mssql_on': 'PRIMARY'
}
You can verify that the resulting DDL is as desired:
In [4]: print(CreateTable(UserModel.__table__).compile(dialect=engine.dialect))
CREATE TABLE [UserModel] (
userid NVARCHAR(255) NULL,
username NVARCHAR(255) NULL,
serialnumber NVARCHAR(255) NULL
) ON [PRIMARY]
Note that at least according to this Q/A primary is the default and as such ON [PRIMARY] could just be omitted.
Related
I am trying to solve the following two problems:
Constructing an ORM model to use in SQLAlchemy with a table that is generated from a user defined function in PostgreSQL
Filtering said table (either in ORM model or not)
I have defined a function that returns a table called transformed. It takes a user_id as an input and returns 3 columns. Looks something like this below (this is extremely simplified, the query is obviously a little more complicated).
CREATE OR REPLACE FUNCTION transformed (user_id integer)
RETURNS TABLE (
id integer,
user varchar,
description varchar,
)
AS $body$
SELECT
id,
user,
description
FROM table
WHERE id = $1
$body$
LANGUAGE sql;
def build_query(user_id):
base_query: Query = select(
Column("id", Integer),
Column("user", String),
Column("description", String),
).select_from(func.transformed)_data(user_id))
return base_query
I have no problem getting the three columns. The challenge is to filter and sort them.
query = build_query(user_id).filter_by(description = "Foo")
This returns the following error: sqlalchemy.exc.InvalidRequestError: Entity namespace for "transformed(:transformed_1) has no property "description".
I would love to know how to either a) properly filter or b) turn the pure columns I am defining into some sort of ORM model that I can then use to filter on the properties of the ORM model.
SQLAlchemy is pretty awesome. You can define the function using DDLEvents and then call the function using func and define the output with table_valued there is a different method for a scalar but its in the same docs.
import sys
from sqlalchemy import (
create_engine,
Integer,
String,
event,
column,
)
from sqlalchemy.schema import (
Column,
DDL,
)
from sqlalchemy.sql import select, func
from sqlalchemy.orm import declarative_base, Session
Base = declarative_base()
username, password, db = sys.argv[1:4]
engine = create_engine(f"postgresql+psycopg2://{username}:{password}#/{db}", echo=True)
class User(Base):
__tablename__ = "users"
id = Column(Integer, primary_key=True)
name = Column(String(8), index=True)
event.listen(
User.__table__,
"after_create",
DDL("""CREATE OR REPLACE FUNCTION transformed (int) RETURNS TABLE (id integer, username varchar)
AS $$ SELECT id, name AS username FROM users WHERE id = $1 $$
LANGUAGE sql;
"""))
Base.metadata.create_all(engine)
with Session(engine) as session, session.begin():
for name in ['User 1', 'User 2', 'User 3']:
session.add(User(name=name))
with Session(engine) as session, session.begin():
# Get a single row with id 1
fn = func.transformed(1).table_valued(
column("id", Integer),
column("username", String))
# This filtering is redundant because we selected
# a user by its id but it demonstrates referencing the column
# we defined above.
q = select(fn).where(fn.c.username.like('User %'))
for id, user in session.execute(q).all():
print(id, user)
This prints out
1 User 1
The actual query looks like this, via echo=True:
SELECT anon_1.id, anon_1.username
FROM transformed(%(transformed_1)s) AS anon_1
WHERE anon_1.username LIKE %(username_1)s
There is one possible solution but it's very similar to simply constructing the query via raw text:
query = build_query(user_id).where(text("id = 4"))
Not a huge fan since it's not easy to expand on this dynamically.
I would like to be able to get info about what types will be created during SQLAlchemy's create_all(). Yes, they can be printed if I set up echo-ing of generated SQL, but how can i print it without actually hitting database? For example, I have a model:
class MyModel(Base):
id = Column(Integer, primary_key=True)
indexed_field = Column(String(50))
enum_field = Column(Enum(MyEnum))
__table_args__ = (
Index("my_ix", indexed_field),
)
where MyEnum is:
class MyEnum(enum.Enum):
A = 0
B = 1
I can get CREATE TABLE statement and all CREATE INDEX statements like this:
from sqlalchemy.schema import CreateTable, CreateIndex
print(str(CreateTable(MyModel.__table__).compile(postgres_engine)))
for idx in MyModel.__table__.indexes:
print(str(CreateIndex(idx)).compile(postgres_engine))
Result will be something like that:
CREATE TABLE my_model (
id SERIAL NOT NULL,
indexed_field VARCHAR(50),
enum_field myenum,
PRIMARY KEY (id)
)
CREATE INDEX my_ix ON my_model (indexed_field)
Notice the line enum_field myenum. How can I get generated SQL for CREATE TYPE myenum... statement?
I've found the answer!
from sqlalchemy.dialects.postgresql.base import PGInspector
PGInspector(postgres_engine).get_enums()
It returns a list of all created enums, which IMO is even better than raw sql, documentation is here.
I am using SQLAlchemy automap. When I described structure Declarative I have got backref property:
The above configuration establishes a collection of Address objects on User called User.addresses.
But now with automap my code is like next:
engine = create_engine('sqlite:///sql_test.db', echo=True)
Session = sessionmaker(bind=engine)
sess = Session()
Base = automap_base()
Base.prepare(engine, reflect=True)
User = Base.classes.Users
addresses = Base.classes.addresses
answer = sess.query(User).filter(User.id==1).first()
print('type:', type(answer)) # will print: class User
But how I can get access to addresses here? I tried: answer.addresses and so one, but it is not working.
Users:
CREATE TABLE "Users"(
"id" Integer PRIMARY KEY,
"name" Text,
CONSTRAINT "unique_id" UNIQUE ( "id" ) )
Addresses:
CREATE TABLE "addresses"(
"id" Integer PRIMARY KEY,
"email" Text,
"user_id" Integer,
CONSTRAINT "lnk_Users_addresses" FOREIGN KEY ( "user_id" ) REFERENCES "Users"( "id" ),
CONSTRAINT "unique_id" UNIQUE ( "id" ) )
The default naming scheme for collection relationships is:
return referred_cls.__name__.lower() + "_collection"
So given that you have a model class addresses, then your relationship should be
User.addresses_collection
If you wish to alter this behaviour, pass your own implementation as the name_for_collection_relationship= keyword argument to AutomapBase.prepare().
I'm trying to use SQLAlchemy's #aggregated decorator to define an attribute ('gross_amount)' for a class, Receipt. This gross_amount attribute is the sum of the Item.gross_amount for all Item instances associated with the Receipt instance by a foreign id.
I.E., a receipt is made up of items, and I want to define a receipt 'gross_amount' value which is just the total $ of all of the items on the receipt.
I've modeled my code after this document http://sqlalchemy-utils.readthedocs.io/en/latest/aggregates.html
So it looks like this...
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy.sql import func
from sqlalchemy import orm
class Receipt(Base):
__tablename__ = "receipts"
__table_args__ = {'extend_existing': True}
id = Column(Integer, index = True, primary_key = True, nullable = False)
#aggregated('itemz', Column(Integer))
def gross_amount(self):
return func.sum(Item.gross_amount)
itemz = orm.relationship(
'Item',
backref='receipts'
)
class Item(Base):
__tablename__ = "items"
id = Column(Integer, index = True, primary_key = True, nullable = False)
'''
FE relevant
'''
gross_amount = Column(Integer)
receipt_id = Column(Integer, ForeignKey("receipts.id"), nullable=False)
In my migration, am I supposed to have a column in the receipts table for gross_amount?
1) When I DO define this column in the receipts table, any Receipt.gross_amount for any instance just points to the gross_amount values defined in the receipts table.
2) When I DO NOT define this column in the receipts table, I get a SQLAlchemy error whenever I execute a SELECT against the database:
ProgrammingError: (psycopg2.ProgrammingError) column receipts.gross_amount does not exist
FWIW, my SQLAlchemy package is the latest distributed thru PIP...
SQLAlchemy==1.1.11
SQLAlchemy-Utils==0.32.14
And my local db on which I'm running this for now is PostgreSQL 9.6.2
What am I doing wrong here? Any patient help would be greatly appreciated!
Yes, you do need to add the column to table:
CREATE TABLE receipts (
id INTEGER NOT NULL,
gross_amount INTEGER, -- <<< See, it's here :)
PRIMARY KEY (id)
);
INSERT INTO receipts VALUES(1,7);
INSERT INTO receipts VALUES(2,7);
CREATE TABLE items (
id INTEGER NOT NULL,
gross_amount INTEGER,
receipt_id INTEGER NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY(receipt_id) REFERENCES receipts (id)
);
Tested with this self-contained snippet:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, orm
from sqlalchemy.orm import sessionmaker
from sqlalchemy.sql import func
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy_utils import aggregated
Base = declarative_base()
class Receipt(Base):
__tablename__ = "receipts"
__table_args__ = {'extend_existing': True}
id = Column(Integer, index = True, primary_key = True, nullable = False)
#aggregated('itemz', Column(Integer))
def gross_amount(self):
return func.sum(Item.gross_amount)
itemz = orm.relationship('Item', backref='receipts')
class Item(Base):
__tablename__ = "items"
id = Column(Integer, index = True, primary_key = True, nullable = False)
gross_amount = Column(Integer)
receipt_id = Column(Integer, ForeignKey("receipts.id"), nullable=False)
def __init__(self, amount):
self.gross_amount=amount
engine = create_engine('sqlite:///xxx.db', echo=True)
Base.metadata.create_all(engine)
session = sessionmaker(bind=engine)()
receipt = Receipt()
receipt.itemz.append(Item(5))
receipt.itemz.append(Item(2))
session.add(receipt)
session.commit()
print (receipt.gross_amount)
Of course, there's also another approach called hybrid_property, which basically allows you to do both orm- and database level queries without adding extra column do your database:
#hybrid_property
def gross_sum(self):
return sum(i.gross_amount for i in self.itemz)
#gross_sum.expression
def gross_sum(cls):
return select([func.sum(Item.gross_amount)]).\
where(Item.receipt_id==cls.id).\
label('gross_sum')
The reason you're getting this error is because the new column you're adding (gross_amount) has not been created in the receipts table in the database.
Meaning, your current database table only has one created column (id). For the aggregated column to work, it needs to contain an additional column called gross_amount.
This additional column has to allow null values.
One way to go about doing that is through SQL directly in PostgreSQL:
ALTER TABLE receipts ADD gross_amount int;
Alternatively, if there's no data yet, you can drop and recreate the table via SQLAlchemy. It should create this extra column automatically.
I'm not sure what you mean by the last part:
When I DO define this column in the receipts table, any
Receipt.gross_amount for any instance just points to the gross_amount
values defined in the receipts table.
That's where it's supposed to point. I'm not sure what you mean by that. Do you mean that it doesn't contain any values, even though there are values for this receipt's items in Item? If so, I would double check that this is the case (and per their examples here, refresh the database session before seeing the results).
I am using sqlalchemy 0.7 and MySQL server version 5.1.63.
I have the following table on my database:
CREATE TABLE `g_domains` (
`id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`name` VARCHAR(255) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `name` (`name`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
The corresponding model is :
class GDomain(Base):
__tablename__ = 'g_domains'
__table_args__ = {
'mysql_engine': 'InnoDB',
'mysql_charset': 'utf8',
'mysql_collate': 'utf8_general_ci'
}
id = Column(mysql.BIGINT(unsigned=True), primary_key=True)
name = Column(mysql.VARCHAR(255, collation='utf8_general_ci'),
nullable=False, unique=True)
The following query in sql alchemy returns no rows :
session.query(GDomain).filter(GDomain.name.in_(domain_set)).
limit(len(domain_set)).all()
where domain_set is a python list containing some domain names like
domain_set = ['www.google.com', 'www.yahoo.com', 'www.AMAZON.com']
Although the table has a row (1, www.amazon.com) the above query returns only
(www.google.com, www.yahoo.com).
When I run the sql query :
SELECT * FROM g_domains
WHERE name IN ('www.google.com', 'www.yahoo.com', 'www.AMAZON.com')
Do you have an idea why this is happening?
Thanks in advance
What is the model_domain variable? Usually it looks like this:
session.query(GDomain).filter(GDomain.name.in_(domain_set)).
limit(len(domain_set)).all()
Note that the GDomain is used in both places. Alternatively you can use aliases:
domains = orm.aliased(GDomain, name='domain')
session.query(domains).filter(domains.name.in_(domain_set))
You can always try debugging, print the query that produced by sqlalchemy (see: SQLAlchemy: print the actual query)