I have a model (A) that has 2 foreign keys: b and c.
b and c should unique together INCLUDING NULL
therefore b and c should only be NULL together ONCE
However, I am unable to accomplish this in Django. Here is the code I have so far. Any help is appreciated!
-_-
class A(models.Model):
b = models.ForeignKey('B', blank = True, null = True)
c = models.ForeignKey('C', blank = True, null = True)
class Meta:
unique_together = (
('b', 'c')
)
This code will produce this unwanted result in the database:
+----+------+------+
| id | b_id | c_id |
+----+------+------+
| 1 | 2 | 3 |
| 2 | 2 | 77 |
| 3 | 2 | NULL |
| 4 | 2 | NULL |
| 5 | NULL | NULL |
| 6 | NULL | NULL |
+----+------+------+
The first 2 rows can only be inserted once by django. which is great :)
However, the remaining rows are duplicate entries for me, and i'd like to restrict this.
UPDATE
I've found something that gets the job done, but it seems really hacky..
Any thoughts?
class A(models.Model):
def clean(self):
from django.core.exceptions import ValidationError
if not any([self.b, self.c]):
if Setting.objects.filter(b__isnull = True, c__isnull = True).exists():
raise ValidationError("Already exists")
elif self.b and not self.c:
if Setting.objects.filter(c__isnull = True, b = self.b).exists():
raise ValidationError("Already exists")
elif self.c and not self.user:
if Setting.objects.filter(c = self.c, b__isnull = True).exists():
raise ValidationError("Already exists")
It's not a problem with Django but with the SQL spec itself - NULL is not a value, so it must NOT be taken into account for uniqueness constraints checks.
You can either have a "pseudo-null" B record and a "pseudo-null" C record in your db and make them the defaults (and not allow NULL of course), or have a denormalized field like OBu suggests.
Maybe there is a better solution out there, but you could do the following:
create a new attribute d and find a generic way to combine b_id and c_id (e.g. str(b_id) + "*" + str(c_id) and do this automatically on model creation (the signals mechanism might come in handy, here)
use d as primary_key
This is more a work around then a solution, but it should do the trick.
One more thought: Would it be an option to check whether there is aready an existing instance with "Null"/"Null" on creation / update of your instance? This would not solve your problem on database level, but the logics would work as expected.
You can use the Unique constraint for b_id column. It wont allow the duplicate entries. Even for a_id column, primary key can be used. Primary key means the combination of unique key and not null constraints.
Related
N.B. I have tagged this with SQLAlchemy and Python because the whole point of the question was to develop a query to translate into SQLAlchemy. This is clear in the answer I have posted. It is also applicable to MySQL.
I have three interlinked tables I use to describe a book. (In the below table descriptions I have eliminated extraneous rows to the question at hand.)
MariaDB [icc]> describe edition;
+-----------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+-----------+------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
+-----------+------------+------+-----+---------+----------------+
7 rows in set (0.001 sec)
MariaDB [icc]> describe line;
+------------+--------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------+--------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| edition_id | int(11) | YES | MUL | NULL | |
| line | varchar(200) | YES | | NULL | |
+------------+--------------+------+-----+---------+----------------+
5 rows in set (0.001 sec)
MariaDB [icc]> describe line_attribute;
+------------+------------+------+-----+---------+-------+
| Field | Type | Null | Key | Default | Extra |
+------------+------------+------+-----+---------+-------+
| line_id | int(11) | NO | PRI | NULL | |
| num | int(11) | YES | | NULL | |
| precedence | int(11) | YES | MUL | NULL | |
| primary | tinyint(1) | NO | MUL | NULL | |
+------------+------------+------+-----+---------+-------+
5 rows in set (0.001 sec)
line_attribute.precedence is the hierarchical level of the given heading. So if War and Peace has Books > Chapters, all of the lines have an attribute that corresponds to the Book they're in (e.g., Book 1 has precedence=1 and num=1) and an attribute for the Chapter they're in (e.g., Chapter 2 has precedence=2 and num=2). This allows me to translate the hierarchical structure of books with volumes, books, parts, chapters, sections, or even acts and scenes. The primary column is a boolean, so that each and every line has one attribute that is primary. If it is a book heading, it is the Book attribute, if it is a chapter heading, it is the Chapter attribute. If it is a regular line in text, it is a line attribute, and the precedence is 0 since it is not a part of the hierarchical structure.
I need to be able to query for all lines with a particular edition_id and that also have the intersection of two line_attributes.
(This would allow me to get all lines from a particular edition that are in, say, Book 1 Chapter 2 of War and Peace).
I can get all lines that have Book 1 with
SELECT
line.*
FROM
line
INNER JOIN
line_attribute
ON
line_attribute.line_id=line.id
WHERE
line.edition_id=2 AND line_attribute.precedence=1 AND line_attribute.num=1;
and I can get all lines that have Chapter 2:
SELECT
line.*
FROM
line
INNER JOIN
line_attribute
ON
line_attribute.line_id=line.id
WHERE
line.edition_id=2 AND line_attribute.precedence=2 AND line_attribute.num=1;
Except the second query returns each chapter 2 from every book in War and Peace.
How do I get from these two queries to just the lines from book 1 chapter 2?
Warning from Raymond Nijland in the comments:
Note for future readers.. Because this question is tagged MySQL.. MySQL does not support INTERSECT keyword.. MariaDB is indeed a fork off the MySQL source code but supports extra features which MySQL does not support.. In MySQL you can simulate the INTERSECT keyword with a INNER JOIN or IN()
Trying to write up a question on SO helps me get my thoughts clear and eventually solve the problem before I have to ask the question. The queries above are much clearer than my initial queries and the question pretty much answers itself, but I never found a clear answer that talks about the intersect utility, so I'm posting this answer anyway.
The solution was the INTERSECT operator.
The solution is simply the intersection of those two queries:
SELECT
line.*
FROM
line
INNER JOIN
line_attribute
ON
line_attribute.line_id=line.id
WHERE
line.edition_id=2 AND line_attribute.precedence=1 AND line_attribute.num=1
INTERSECT /* it is literally this simple */
SELECT
line.*
FROM
line
INNER JOIN
line_attribute
ON
line_attribute.line_id=line.id
WHERE
line.edition_id=2 AND line_attribute.precedence=2 AND line_attribute.num=2;
This also means I could get all of the book and chapter headings for a particular book by simply adding an additional constraint (line_attribute.primary=1).
This solution seems broadly applicable to me. Assuming, for instance, you have questions in a StackOverflow clone, which are tagged, you can get the intersection of questions with two tags (e.g., all posts that have both the SQLAlchemy and Python tags). I am certainly going to use this method for that sort of query.
I coded this up in MySQL because it helps me get the query straight to translate it into SQLAlchemy.
The SQLAlchemy query is this simple:
[nav] In [10]: q1 = Line.query.join(LineAttribute).filter(LineAttribute.precedence==1, LineAttribute.num==1)
[ins] In [11]: q2 = Line.query.join(LineAttribute).filter(LineAttribute.precedence==2, LineAttribute.num==1)
[ins] In [12]: q1.intersect(q2).all()
Hopefully the database structure in this question helps someone down the road. I didn't want to delete the question after I solved my own problem.
This question already has answers here:
How can I reset a autoincrement sequence number in sqlite
(5 answers)
SQLite Reset Primary Key Field
(5 answers)
Closed 4 years ago.
how do I reset the increment count in flask-sqlalchemy after deleting a row so that the next insert will get the id of deleted row?
ie :
table users:
user_id | name |
________________
3 | mbuvi
4 | meshack
5 | You
when I delete user with id=5;
the next insertion into users is having id = 6 but I want it to have id=5;
user_id | name |
________________
3 | mbuvi
4 | meshack
6 | me
How do I solve this?
Your database will keep track your auto increment id! so you can't do something like this.BTW it's no about the flask-sqlalchemy question! If you really want to do this, you have to calculater the left id which you can used and fill it with that number! for example:
+----+--------+
| id | number |
+----+--------+
| 1 | 819 |
| 2 | 829 |
| 4 | 829 |
| 5 | 829 |
+----+--------+
And you have to find the id (3) and then insert with id. so this cause a query all the table util you got that position! don't no why you need to do this, but still have solution!
step01, you gotta use a cache to do this! here I recommand use redis
step02, If you want to delete any row, just simply cache your id into the redis list, the Order-Set is best optionl for you! before delete any row, save it to the cache!
step03, before insert any new row, check see if there any id aviable in your redis! if true, pop it out and insert it with the id which you pop out!
step04, the code should like below:
def insert_one(data):
r = redis.Redis()
_id = r.pop('ID_DB')
if _id:
cursor.execute("insert into xxx(id, data)values(%s, %s)", data)
else:
# use default increment id
cursor.execute("insert into xxx(data)values(%s)", data)
def delete(data, id):
# query the target which you will delete
# if you delete by condtion `id` that's best
r = redis.Redis()
r.push('ID_DB',id)
# DO the rest of you want ...
# delete ....
I have a table with rows of data for different experiments.
experiment_id data_1 data_2
------------- ------ -------
1
2
3
4
..
I have a user database on django, and I would like to store permissions indicating which users can access which rows and then return only the rows the user is authorized for.
What format should I use to store the permissions? Simply a table with a row for each user and a column for each experiment with Boolean? And in that case I would have to add a row to this table each time an experiment is added?
user experiment_1 experiment_2 experiment_3 ..
---- ------------ ------------ ------------ --
user_1 True False False ..
user_2 False True False ..
..
Any reference literature on the topic would also be great, preferably related to sqlite3 functionality since that is my current db backend.
I'm not 100% sure what all will work best for you but in the past I find using a solution as follows to be the easiest to query against and maintain in the future.
Table: Experiment
Experiment_Id | data_1 | data_2
-----------------------------------
1 | ... | ...
2 | ... | ...
Table: User
User_Name | Password | ...
----------------------------
User1 | ...
User2 | ...
Table: User_Experiment_Permissions
User_Name | Experiment | Can_Read | Can_Edit
--------------------------------------------
User1 | 1 | true | false
User2 | 1 | false | false
User1 | 2 | true | true
User2 | 2 | true | false
As you can see, in the new table we reference both the user and the experiment. This allows us fine grain control over the relationship between the user and the experiment. Also, if this relationship had a new permission that arose, such as can_delete then you can simply add this to the new cross reference table with a default and the change will be retrofit into your your system :-)
It depends on the way you will use the permissions for.
- In case you will use this values inside a query
you have two options for example to get the users with specific permiss
Create a bit masking number fields and every bit will represent permission, and you can use AND/OR to get whatever combinations of permissions you need.
Advantage : small size, very efficient
Disadvantage: complex to implement
Create a field for each permission ( your solution ).
Advantage : To easy to add
Disadvantage: Have to edit schema with each permission
- In case you will not use it for any query and will process it at the code you can just dump a JSON into one column include all the permission the user has like :
{"experiment_1": 1, "experiment_2": 0, "experiment_3": 1}
My question is rather specific, if you have a better title please suggest one. Also, formatting is bad - didn't know how to combine lists and codeblocks.
I have an SQLite3 database with the following (relevant parts of the) .schema:
CREATE TABLE users (id INTEGER PRIMARY KEY NOT NULL, user TEXT UNIQUE);
CREATE TABLE locations (id INTEGER PRIMARY KEY NOT NULL, name TEXT UNIQUE);
CREATE TABLE purchases (location_id INTEGER, user_id INTEGER);
CREATE TABLE sales (location_id integer, user_id INTEGER);
purchases has about 4.5mil entries, users about 300k, sales about 100k, and locations about 250 - just to gauge the data volume.
My desired use would be to generate a JSON object to be handed off to another application, very much condensed in volume by doing the following:
-GROUPing both purchases and sales into one common table BY location_id,user_id - IOW, getting the number of "actions" per user per location. That I can do, result is something like
loc | usid | loccount
-----------------------
1 | 1246 | 123
1 | 2345 | 1
13 | 1246 | 46
13 | 8732 | 4
27 | 2345 | 41
(At least it looks good, always hard to tell with such volumes; query:
select location_id,user_id,count(location_id) from
(select location_id,user_id from purchases
union all
select location_id,user_id from sales)
group by location_id,user_id order by user_id`
)
-Then, transposing that giant table such that I would get:
usid | loc1 | loc13 | loc27
---------------------------
1246 | 123 | 46 | 0
2345 | 1 | 0 | 41
8732 | 0 | 4 | 0
That I cannot do, and it's my absolutely crucial point for this question. I tried some things I found online, especially here, but I just started SQLite a little while ago and don't understand many queries.
-Lastly, translate the table into plain text in order to write it to JSON:
user | AAAA | BBBBB | CCCCC
---------------------------
zeta | 123 | 46 | 0
beta | 1 | 0 | 41
iota | 0 | 4 | 0
That I probably could do with quite a bit of experimentation and inner join, although I'm always very unsure what way is the best approach to handle such data volumes, hence I wouldn't mind a pointer.
The whole thing is written in Python's sqlite3 interface, if it matters. In the end, I'd love to have something I could just do a "for" loop per user over in order to generate the JSON, which would then of course be very simple. It doesn't matter if the query takes a long time (<10min would be nice), it's only run twice per day as a sort of backup. I've only got a tiny VPS available, but being limited to a single core the performance is as good as on my reasonably powerful desktop. (i5-3570k running Debian.)
The table headers are just examples because I wasn't quite sure if I could use integers for them (didn't discover the syntax if so), as long as I'm somehow able to look up the numeric part in the locations table I'm fine. Same for translating the user IDs into names. The number of columns is known beforehand - they're after all just INTEGER PRIMARY KEYs and I have a list() of them from some other operation. The number of rows can be determined reasonably quickly, ~3s, if need be.
Consider using subqueries to achieve your desired transposed output:
SELECT DISTINCT m.usid,
IFNULL((SELECT t1.loccount FROM tablename t1
WHERE t1.usid = m.usid AND t1.loc=1),0) AS Loc1,
IFNULL((SELECT t2.loccount FROM tablename t2
WHERE t2.usid = m.usid AND t2.loc=13),0) AS Loc13,
IFNULL((SELECT t3.loccount FROM tablename t3
WHERE t3.usid = m.usid AND t3.loc=27),0) AS Loc27
FROM tablename As m
Alternatively, you can use nested IF statements (or in the case of SQLite that uses CASE/WHEN) as derived table:
SELECT temp.usid, Max(temp.loc1) As Loc1,
Max(temp.loc13) As Loc13, Max(temp.loc27) As Loc27
FROM
(SELECT tablename.usid,
CASE WHEN loc=1 THEN loccount ELSE 0 As Loc1 END,
CASE WHEN loc=13 THEN loccount ELSE 0 As Loc13 END,
CASE WHEN loc=27 THEN loccount ELSE 0 As Loc27 END
FROM tablename) AS temp
GROUP BY temp.usid
I've got a model that looks like this,
class PL(models.Model):
locid = models.AutoField(primary_key=True)
mentionedby = models.ManyToManyField(PRT)
class PRT(models.Model):
tid = ..
The resulting many to many table in mysql is formed as,
+------------------+------------+------+-----+---------+----------------+
| Field | Type | Null | Key | Default | Extra |
+------------------+------------+------+-----+---------+----------------+
| id | int(11) | NO | PRI | NULL | auto_increment |
| PL_id | int(11) | NO | MUL | NULL | |
| PRT_id | bigint(64) | NO | MUL | NULL | |
+------------------+------------+------+-----+---------+----------------+
Now, if pl is an object of PL and prt that of PRT, then doing
pl.mentionedby.add(prt)
gives me an error
Incorrect integer value: 'PRT object'
for column 'prt_id' at row 1"
whereas
pl.mentionedby.add(prt.tid)
works fine - with one caveat.
I can see all the elements in pl.mentionedby.all(), but I can't go to a mentioned PRT object and see its prt.mentionedby_set.all().
Does anyone know why this happens? Whats the best way to fix it?
Thanks!
Adding prt directly should work on first try. How are you retrieving pl and prt? Assuming you have some data in your database, try those commands from the Django shell and see if it works. There seems to be some missing information from the question. After running python manage.py shell:
from yourapp.models import PL
pl = PL.objects.get(id=1)
prt = PRT.objects.get(id=1)
pl.mentionedby.add(prt)
Are these the complete models? I can only assume that something's been overriden somewhere, that probably shouldn't have been.
Can you post the full code?