I am using SQLAlchemy automap. When I described structure Declarative I have got backref property:
The above configuration establishes a collection of Address objects on User called User.addresses.
But now with automap my code is like next:
engine = create_engine('sqlite:///sql_test.db', echo=True)
Session = sessionmaker(bind=engine)
sess = Session()
Base = automap_base()
Base.prepare(engine, reflect=True)
User = Base.classes.Users
addresses = Base.classes.addresses
answer = sess.query(User).filter(User.id==1).first()
print('type:', type(answer)) # will print: class User
But how I can get access to addresses here? I tried: answer.addresses and so one, but it is not working.
Users:
CREATE TABLE "Users"(
"id" Integer PRIMARY KEY,
"name" Text,
CONSTRAINT "unique_id" UNIQUE ( "id" ) )
Addresses:
CREATE TABLE "addresses"(
"id" Integer PRIMARY KEY,
"email" Text,
"user_id" Integer,
CONSTRAINT "lnk_Users_addresses" FOREIGN KEY ( "user_id" ) REFERENCES "Users"( "id" ),
CONSTRAINT "unique_id" UNIQUE ( "id" ) )
The default naming scheme for collection relationships is:
return referred_cls.__name__.lower() + "_collection"
So given that you have a model class addresses, then your relationship should be
User.addresses_collection
If you wish to alter this behaviour, pass your own implementation as the name_for_collection_relationship= keyword argument to AutomapBase.prepare().
Related
I have query in sqlserver and I wanted to write a model corresponding to it using sqlalchemy orm. I need to know how to convert ON [PRIMARY] into sqlalchemy.
Here is my query:
SET ANSI_NULLS ON
GO
SET QUOTED_IDENTIFIER ON
GO
CREATE TABLE [dbo].[UserModel](
[userid] [nvarchar](255) NULL,
[username] [nvarchar](255) NULL,
[serialnumber] [nvarchar](255) NULL
) ON [PRIMARY]
GO
Here is my model:
class UserModel(db.Model):
__tablename__ = 'UserModel'
userid = Column('userid', Unicode(255))
username = Column('username', Unicode(255))
serialnumber = Column('serialnumber', Unicode(255))
Any suggestions, how would I achieve this
Currently I m getting the following error as below:
sqlalchemy.exc.ArgumentError: Mapper Mapper|UserModel|UserModel could not assemble any primary key columns for mapped table 'UserModel'
I know there there is no primary primary key defined in my model but the above query is working fine. So I wanted to translate the above query using sqlalchemy orm.
You cannot translate the above DDL using most ORMs, at least directly, because they expect Python objects to correspond to uniquely identifiable rows in the database. Now, if you know for sure that for example UserModel.userid uniquely identifies rows, you can instruct SQLAlchemy to use it as a "primary key":
class UserModel(db.Model):
__tablename__ = 'UserModel'
userid = Column('userid', Unicode(255))
username = Column('username', Unicode(255))
serialnumber = Column('serialnumber', Unicode(255))
__mapper_args__ = {
'primary_key': [userid]
}
Keep in mind that you've lied to the ORM and any consequences are on you.
As to defining the filegroup the table should be stored in using ON [PRIMARY], you'll have to augment the mssql dialect a bit:
from sqlalchemy import Table
from sqlalchemy.schema import CreateTable
from sqlalchemy.ext.compiler import compiles
# Add our custom dialect specific argument
Table.argument_for('mssql', 'on', None)
#compiles(CreateTable, 'mssql')
def compile_create_table(create, compiler, **kw):
stmt = compiler.visit_create_table(create, **kw)
filegroup = create.element.kwargs.get('mssql_on')
if filegroup:
stmt = stmt.rstrip() # Prettier output, remove newlines
filegroup = compiler.preparer.quote(filegroup)
stmt = f"{stmt} ON {filegroup}\n\n"
return stmt
and add the necessary table args to your model definition:
class UserModel(db.Model):
...
__table_args__ = {
'mssql_on': 'PRIMARY'
}
You can verify that the resulting DDL is as desired:
In [4]: print(CreateTable(UserModel.__table__).compile(dialect=engine.dialect))
CREATE TABLE [UserModel] (
userid NVARCHAR(255) NULL,
username NVARCHAR(255) NULL,
serialnumber NVARCHAR(255) NULL
) ON [PRIMARY]
Note that at least according to this Q/A primary is the default and as such ON [PRIMARY] could just be omitted.
I'm trying to use SQLAlchemy's #aggregated decorator to define an attribute ('gross_amount)' for a class, Receipt. This gross_amount attribute is the sum of the Item.gross_amount for all Item instances associated with the Receipt instance by a foreign id.
I.E., a receipt is made up of items, and I want to define a receipt 'gross_amount' value which is just the total $ of all of the items on the receipt.
I've modeled my code after this document http://sqlalchemy-utils.readthedocs.io/en/latest/aggregates.html
So it looks like this...
from sqlalchemy import Column, Integer, ForeignKey
from sqlalchemy.sql import func
from sqlalchemy import orm
class Receipt(Base):
__tablename__ = "receipts"
__table_args__ = {'extend_existing': True}
id = Column(Integer, index = True, primary_key = True, nullable = False)
#aggregated('itemz', Column(Integer))
def gross_amount(self):
return func.sum(Item.gross_amount)
itemz = orm.relationship(
'Item',
backref='receipts'
)
class Item(Base):
__tablename__ = "items"
id = Column(Integer, index = True, primary_key = True, nullable = False)
'''
FE relevant
'''
gross_amount = Column(Integer)
receipt_id = Column(Integer, ForeignKey("receipts.id"), nullable=False)
In my migration, am I supposed to have a column in the receipts table for gross_amount?
1) When I DO define this column in the receipts table, any Receipt.gross_amount for any instance just points to the gross_amount values defined in the receipts table.
2) When I DO NOT define this column in the receipts table, I get a SQLAlchemy error whenever I execute a SELECT against the database:
ProgrammingError: (psycopg2.ProgrammingError) column receipts.gross_amount does not exist
FWIW, my SQLAlchemy package is the latest distributed thru PIP...
SQLAlchemy==1.1.11
SQLAlchemy-Utils==0.32.14
And my local db on which I'm running this for now is PostgreSQL 9.6.2
What am I doing wrong here? Any patient help would be greatly appreciated!
Yes, you do need to add the column to table:
CREATE TABLE receipts (
id INTEGER NOT NULL,
gross_amount INTEGER, -- <<< See, it's here :)
PRIMARY KEY (id)
);
INSERT INTO receipts VALUES(1,7);
INSERT INTO receipts VALUES(2,7);
CREATE TABLE items (
id INTEGER NOT NULL,
gross_amount INTEGER,
receipt_id INTEGER NOT NULL,
PRIMARY KEY (id),
FOREIGN KEY(receipt_id) REFERENCES receipts (id)
);
Tested with this self-contained snippet:
from sqlalchemy import Column, Integer, ForeignKey, create_engine, orm
from sqlalchemy.orm import sessionmaker
from sqlalchemy.sql import func
from sqlalchemy.ext.declarative import declarative_base
from sqlalchemy_utils import aggregated
Base = declarative_base()
class Receipt(Base):
__tablename__ = "receipts"
__table_args__ = {'extend_existing': True}
id = Column(Integer, index = True, primary_key = True, nullable = False)
#aggregated('itemz', Column(Integer))
def gross_amount(self):
return func.sum(Item.gross_amount)
itemz = orm.relationship('Item', backref='receipts')
class Item(Base):
__tablename__ = "items"
id = Column(Integer, index = True, primary_key = True, nullable = False)
gross_amount = Column(Integer)
receipt_id = Column(Integer, ForeignKey("receipts.id"), nullable=False)
def __init__(self, amount):
self.gross_amount=amount
engine = create_engine('sqlite:///xxx.db', echo=True)
Base.metadata.create_all(engine)
session = sessionmaker(bind=engine)()
receipt = Receipt()
receipt.itemz.append(Item(5))
receipt.itemz.append(Item(2))
session.add(receipt)
session.commit()
print (receipt.gross_amount)
Of course, there's also another approach called hybrid_property, which basically allows you to do both orm- and database level queries without adding extra column do your database:
#hybrid_property
def gross_sum(self):
return sum(i.gross_amount for i in self.itemz)
#gross_sum.expression
def gross_sum(cls):
return select([func.sum(Item.gross_amount)]).\
where(Item.receipt_id==cls.id).\
label('gross_sum')
The reason you're getting this error is because the new column you're adding (gross_amount) has not been created in the receipts table in the database.
Meaning, your current database table only has one created column (id). For the aggregated column to work, it needs to contain an additional column called gross_amount.
This additional column has to allow null values.
One way to go about doing that is through SQL directly in PostgreSQL:
ALTER TABLE receipts ADD gross_amount int;
Alternatively, if there's no data yet, you can drop and recreate the table via SQLAlchemy. It should create this extra column automatically.
I'm not sure what you mean by the last part:
When I DO define this column in the receipts table, any
Receipt.gross_amount for any instance just points to the gross_amount
values defined in the receipts table.
That's where it's supposed to point. I'm not sure what you mean by that. Do you mean that it doesn't contain any values, even though there are values for this receipt's items in Item? If so, I would double check that this is the case (and per their examples here, refresh the database session before seeing the results).
I have a model with a unique_together defined for 3 fields to be unique together:
class MyModel(models.Model):
clid = models.AutoField(primary_key=True, db_column='CLID')
csid = models.IntegerField(db_column='CSID')
cid = models.IntegerField(db_column='CID')
uuid = models.CharField(max_length=96, db_column='UUID', blank=True)
class Meta(models.Meta):
unique_together = [
["csid", "cid", "uuid"],
]
Now, if I attempt to save a MyModel instance with an existing csid+cid+uuid combination, I would get:
IntegrityError: (1062, "Duplicate entry '1-1-1' for key 'CSID'")
Which is correct. But, is there a way to customize that key name? (CSID in this case)
In other words, can I provide a name for a constraint listed in unique_together?
As far as I understand, this is not covered in the documentation.
Its not well documented, but depending on if you are using Django 1.6 or 1.7 there are two ways you can do this:
In Django 1.6 you can override the unique_error_message, like so:
class MyModel(models.Model):
clid = models.AutoField(primary_key=True, db_column='CLID')
csid = models.IntegerField(db_column='CSID')
cid = models.IntegerField(db_column='CID')
# ....
def unique_error_message(self, model_class, unique_check):
if model_class == type(self) and unique_check == ("csid", "cid", "uuid"):
return _('Your custom error')
else:
return super(MyModel, self).unique_error_message(model_class, unique_check)
Or in Django 1.7:
class MyModel(models.Model):
clid = models.AutoField(primary_key=True, db_column='CLID')
csid = models.IntegerField(db_column='CSID')
cid = models.IntegerField(db_column='CID')
uuid = models.CharField(max_length=96, db_column='UUID', blank=True)
class Meta(models.Meta):
unique_together = [
["csid", "cid", "uuid"],
]
error_messages = {
NON_FIELD_ERRORS: {
'unique_together': "%(model_name)s's %(field_labels)s are not unique.",
}
}
Changing index name in ./manage.py sqlall output.
You could run ./manage.py sqlall yourself and add in the constraint name yourself and apply manually instead of syncdb.
$ ./manage.py sqlall test
BEGIN;
CREATE TABLE `test_mymodel` (
`CLID` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
`CSID` integer NOT NULL,
`CID` integer NOT NULL,
`UUID` varchar(96) NOT NULL,
UNIQUE (`CSID`, `CID`, `UUID`)
)
;
COMMIT;
e.g.
$ ./manage.py sqlall test
BEGIN;
CREATE TABLE `test_mymodel` (
`CLID` integer AUTO_INCREMENT NOT NULL PRIMARY KEY,
`CSID` integer NOT NULL,
`CID` integer NOT NULL,
`UUID` varchar(96) NOT NULL,
UNIQUE constraint_name (`CSID`, `CID`, `UUID`)
)
;
COMMIT;
Overriding BaseDatabaseSchemaEditor._create_index_name
The solution pointed out by #danihp is incomplete, it only works for field updates (BaseDatabaseSchemaEditor._alter_field)
The sql I get by overriding _create_index_name is:
BEGIN;
CREATE TABLE "testapp_mymodel" (
"CLID" integer NOT NULL PRIMARY KEY AUTOINCREMENT,
"CSID" integer NOT NULL,
"CID" integer NOT NULL,
"UUID" varchar(96) NOT NULL,
UNIQUE ("CSID", "CID", "UUID")
)
;
COMMIT;
Overriding BaseDatabaseSchemaEditor.create_model
based on https://github.com/django/django/blob/master/django/db/backends/schema.py
class BaseDatabaseSchemaEditor(object):
# Overrideable SQL templates
sql_create_table_unique = "UNIQUE (%(columns)s)"
sql_create_unique = "ALTER TABLE %(table)s ADD CONSTRAINT %(name)s UNIQUE (%(columns)s)"
sql_delete_unique = "ALTER TABLE %(table)s DROP CONSTRAINT %(name)s"
and this is the piece in create_model that is of interest:
# Add any unique_togethers
for fields in model._meta.unique_together:
columns = [model._meta.get_field_by_name(field)[0].column for field in fields]
column_sqls.append(self.sql_create_table_unique % {
"columns": ", ".join(self.quote_name(column) for column in columns),
})
Conclusion
You could:
override create_model to use _create_index_name for unique_together contraints.
modify sql_create_table_unique template to include a name parameter.
You may also be able to check a possible fix on this ticket:
https://code.djangoproject.com/ticket/24102
Integrity error is raised from database but from django:
create table t ( a int, b int , c int);
alter table t add constraint u unique ( a,b,c); <-- 'u'
insert into t values ( 1,2,3);
insert into t values ( 1,2,3);
Duplicate entry '1-2-3' for key 'u' <---- 'u'
That means that you need to create constraint with desired name in database. But is django in migrations who names constraint. Look into _create_unique_sql :
def _create_unique_sql(self, model, columns):
return self.sql_create_unique % {
"table": self.quote_name(model._meta.db_table),
"name": self.quote_name(self._create_index_name(model, columns, suffix="_uniq")),
"columns": ", ".join(self.quote_name(column) for column in columns),
}
Is _create_index_name who has the algorithm to names constraints:
def _create_index_name(self, model, column_names, suffix=""):
"""
Generates a unique name for an index/unique constraint.
"""
# If there is just one column in the index, use a default algorithm from Django
if len(column_names) == 1 and not suffix:
return truncate_name(
'%s_%s' % (model._meta.db_table, self._digest(column_names[0])),
self.connection.ops.max_name_length()
)
# Else generate the name for the index using a different algorithm
table_name = model._meta.db_table.replace('"', '').replace('.', '_')
index_unique_name = '_%x' % abs(hash((table_name, ','.join(column_names))))
max_length = self.connection.ops.max_name_length() or 200
# If the index name is too long, truncate it
index_name = ('%s_%s%s%s' % (
table_name, column_names[0], index_unique_name, suffix,
)).replace('"', '').replace('.', '_')
if len(index_name) > max_length:
part = ('_%s%s%s' % (column_names[0], index_unique_name, suffix))
index_name = '%s%s' % (table_name[:(max_length - len(part))], part)
# It shouldn't start with an underscore (Oracle hates this)
if index_name[0] == "_":
index_name = index_name[1:]
# If it's STILL too long, just hash it down
if len(index_name) > max_length:
index_name = hashlib.md5(force_bytes(index_name)).hexdigest()[:max_length]
# It can't start with a number on Oracle, so prepend D if we need to
if index_name[0].isdigit():
index_name = "D%s" % index_name[:-1]
return index_name
For the current django version (1.7) the constraint name for a composite unique constraint looks like:
>>> _create_index_name( 'people', [ 'c1', 'c2', 'c3'], '_uniq' )
'myapp_people_c1_d22a1efbe4793fd_uniq'
You should overwrite _create_index_name in some way to change algorithm. A way, maybe, writing your own db backend inhering from mysql and overwriting _create_index_name in your DatabaseSchemaEditor on your schema.py (not tested)
I believe you have to do that in your Database;
MySQL:
ALTER TABLE `votes` ADD UNIQUE `unique_index`(`user`, `email`, `address`);
I believe would then say ... for key 'unique_index'
One solution is you can catch the IntegrityError at save(), and then make custom error message as you want as below.
try:
obj = MyModel()
obj.csid=1
obj.cid=1
obj.uuid=1
obj.save()
except IntegrityError:
message = "IntegrityError: Duplicate entry '1-1-1' for key 'CSID', 'cid', 'uuid' "
Now you can use this message to display as error message.
I am using sqlalchemy 0.7 and MySQL server version 5.1.63.
I have the following table on my database:
CREATE TABLE `g_domains` (
`id` BIGINT(20) UNSIGNED NOT NULL AUTO_INCREMENT,
`name` VARCHAR(255) NOT NULL,
PRIMARY KEY (`id`),
UNIQUE INDEX `name` (`name`)
)
COLLATE='utf8_general_ci'
ENGINE=InnoDB
The corresponding model is :
class GDomain(Base):
__tablename__ = 'g_domains'
__table_args__ = {
'mysql_engine': 'InnoDB',
'mysql_charset': 'utf8',
'mysql_collate': 'utf8_general_ci'
}
id = Column(mysql.BIGINT(unsigned=True), primary_key=True)
name = Column(mysql.VARCHAR(255, collation='utf8_general_ci'),
nullable=False, unique=True)
The following query in sql alchemy returns no rows :
session.query(GDomain).filter(GDomain.name.in_(domain_set)).
limit(len(domain_set)).all()
where domain_set is a python list containing some domain names like
domain_set = ['www.google.com', 'www.yahoo.com', 'www.AMAZON.com']
Although the table has a row (1, www.amazon.com) the above query returns only
(www.google.com, www.yahoo.com).
When I run the sql query :
SELECT * FROM g_domains
WHERE name IN ('www.google.com', 'www.yahoo.com', 'www.AMAZON.com')
Do you have an idea why this is happening?
Thanks in advance
What is the model_domain variable? Usually it looks like this:
session.query(GDomain).filter(GDomain.name.in_(domain_set)).
limit(len(domain_set)).all()
Note that the GDomain is used in both places. Alternatively you can use aliases:
domains = orm.aliased(GDomain, name='domain')
session.query(domains).filter(domains.name.in_(domain_set))
You can always try debugging, print the query that produced by sqlalchemy (see: SQLAlchemy: print the actual query)
I want to have a base entity with a field deleted which marks a deleted record. And i have 2 subclasses, each of them to have their own table with all own columns:
from elixir import *
from sqlalchemy import create_engine
class Catalog(Entity):
using_options(inheritance='concrete')
deleted = Boolean
class Contact(Catalog):
using_options(inheritance='concrete')
name = Field(String(60))
class Location(Catalog):
using_options(inheritance='concrete')
name = Field(String(100))
setup_all()
metadata.bind = create_engine('sqlite:///', echo=True)
metadata.create_all()
And the result:
CREATE TABLE __main___catalog (
id INTEGER NOT NULL,
PRIMARY KEY (id)
)
CREATE TABLE __main___contact (
id INTEGER NOT NULL,
name VARCHAR(60),
PRIMARY KEY (id)
)
CREATE TABLE __main___location (
id INTEGER NOT NULL,
name VARCHAR(100),
PRIMARY KEY (id)
)
Questions:
How to avoid creation of a table for the base entity? - solved: using_options(abstract = True)
Why field deleted is not in the created tables? - this solved - i forgot to put it inside a Field
I want to avoid typing in each subclass using_options(inheritance='concrete') but still have "concrete inheritance". Is there a way to make it default for all subclasses?
This works:
class Catalog(Entity):
deleted = Field(Boolean)
using_options(abstract = True, inheritance = 'concrete')
class Contact(Catalog):
name = Field(String(60))
class Location(Catalog):
name = Field(String(100))
and creates the following tables:
CREATE TABLE __main___contact (
id INTEGER NOT NULL,
deleted BOOLEAN,
name VARCHAR(60),
PRIMARY KEY (id),
CHECK (deleted IN (0, 1))
)
CREATE TABLE __main___location (
id INTEGER NOT NULL,
deleted BOOLEAN,
name VARCHAR(100),
PRIMARY KEY (id),
CHECK (deleted IN (0, 1))
)