This question already has answers here:
How can I reset a autoincrement sequence number in sqlite
(5 answers)
SQLite Reset Primary Key Field
(5 answers)
Closed 4 years ago.
how do I reset the increment count in flask-sqlalchemy after deleting a row so that the next insert will get the id of deleted row?
ie :
table users:
user_id | name |
________________
3 | mbuvi
4 | meshack
5 | You
when I delete user with id=5;
the next insertion into users is having id = 6 but I want it to have id=5;
user_id | name |
________________
3 | mbuvi
4 | meshack
6 | me
How do I solve this?
Your database will keep track your auto increment id! so you can't do something like this.BTW it's no about the flask-sqlalchemy question! If you really want to do this, you have to calculater the left id which you can used and fill it with that number! for example:
+----+--------+
| id | number |
+----+--------+
| 1 | 819 |
| 2 | 829 |
| 4 | 829 |
| 5 | 829 |
+----+--------+
And you have to find the id (3) and then insert with id. so this cause a query all the table util you got that position! don't no why you need to do this, but still have solution!
step01, you gotta use a cache to do this! here I recommand use redis
step02, If you want to delete any row, just simply cache your id into the redis list, the Order-Set is best optionl for you! before delete any row, save it to the cache!
step03, before insert any new row, check see if there any id aviable in your redis! if true, pop it out and insert it with the id which you pop out!
step04, the code should like below:
def insert_one(data):
r = redis.Redis()
_id = r.pop('ID_DB')
if _id:
cursor.execute("insert into xxx(id, data)values(%s, %s)", data)
else:
# use default increment id
cursor.execute("insert into xxx(data)values(%s)", data)
def delete(data, id):
# query the target which you will delete
# if you delete by condtion `id` that's best
r = redis.Redis()
r.push('ID_DB',id)
# DO the rest of you want ...
# delete ....
Related
I have loaded with succes large CSV files (with 1 header row) to a mysql table from Python with the command:
LOAD DATA LOCAL INFILE 'file.csv' INTO TABLE 'table" FIELDS TERMINATED BY ';' IGNORE 1 LINES (#vone, #vtwo, #vthree) SET DatumTijd = #vone, Debiet = NULLIF(#vtwo,''),Boven = NULLIF(#vthree,'')
The file contains historic data back to 1970. Every month I get an update with roughly 4320 rows that need to be added to the existing table. Sometimes there is an overlap with the existing table, so I would like to use REPLACE. But this does not seem to work in combination with IGNORE 1 LINES. The primary key is DatumTijd, which follows the mysql datetime format.
I tried several combinations of REPLACE and IGNORE in different order, before the INTO TABLE "table" and behind FIELDS TERMINATED part.
Any suggestions how to solve this?
Apart from the possible typo of enclosing the table name in single quotes rather than backticks the load statement works fine on my windows device given the following data
one;two;three
2023-01-01;1;1
2023-01-02;2;2
2023-01-01;3;3
2022-01-04;;;
note I prefer coalesce to nullif and have included an auto_increment id to demonstrate what replace actually does , ie delete and insert.
drop table if exists t;
create table t(
id int auto_increment primary key,
DatumTijd date,
Debiet varchar(1),
boven varchar(1),
unique index key1(DatumTijd)
);
LOAD DATA INFILE 'C:\\Program Files\\MariaDB 10.1\\data\\sandbox\\data.txt'
replace INTO TABLE t
FIELDS TERMINATED BY ';'
IGNORE 1 LINES (#vone, #vtwo, #vthree)
SET DatumTijd = #vone,
Debiet = coalesce(#vtwo,''),
Boven = coalesce(#vthree,'')
;
select * from t;
+----+------------+--------+-------+
| id | DatumTijd | Debiet | boven |
+----+------------+--------+-------+
| 2 | 2023-01-02 | 2 | 2 |
| 3 | 2023-01-01 | 3 | 3 |
| 4 | 2022-01-04 | | |
+----+------------+--------+-------+
3 rows in set (0.001 sec)
It should not matter that replace in effect creates a new record but if it does to you consider loading to a staging table then insert..on duplicate key to target
ive written some code to parse a website, and input it into a mysql db.
The problem is I am getting a lot of duplicates per FKToTech_id like:
id | ref | FKToTech_id |
+----+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+
| 1 | website.com/path | 1 |
| 2 | website.com/path | 1 |
| 3 | website.com/path | 1
What Im looking for is instead to have (1) row in this database, based on if ref has been entered already for FKToTech_id and not have multiple of the same row like:
id | ref | FKToTech_id |
+----+--------------------------------------------------------------------------------------------------------------------------------------------------+--------------------------+
| 1 | website.com/path | 1 |
How can I modify my code below to just python pass if the above is True (==1 ref with same FKToTech_id?
for i in elms:
allcves = {cursor.execute("INSERT INTO TechBooks (ref, FKToTech_id) VALUES (%s, %s) ", (i.attrs["href"], row[1])) for row in cves}
mydb.commit()
Thanks
Make ref a unique column, then use INSERT IGNORE to skip the insert if it would cause a duplicate key error.
ALTER TABLE TechBooks ADD UNIQUE INDEX (ref);
for i in elms:
cursor.executemany("INSERT IGNORE INTO TechBooks (ref, FKToTech_id) VALUES (%s, %s) ", [(i.attrs["href"], row[1]) for row in cves])
mydb.commit()
I'm not sure what your intent was by assigning the results of cursor.execute() to allcves. cursor.execute() doesn't return a value unless you use multi=True. I've replaced the useless set comprehension with use of cursor.executemany() to insert many rows at once.
Say I have the following CSV file
Purchases.csv
+--------+----------+
| Client | Item |
+--------+----------+
| Mark | Computer |
| Mark | Lamp |
| John | Computer |
+--------+----------+
What is the best practice, in Python, to split this table into two separate tables and join them in a bridge table using foreign key, i.e.
Client table
+----------+--------+
| ClientID | Client |
+----------+--------+
| 1 | Mark |
| 2 | John |
+----------+--------+
Item table
+--------+----------+
| ItemID | Item |
+--------+----------+
| 1 | Computer |
| 2 | Lamp |
+--------+----------+
Item Client Bridge Table
+----------+--------+
| ClientID | ItemID |
+----------+--------+
| 1 | 1 |
| 1 | 2 |
| 2 | 1 |
+----------+--------+
I should mention here that it possible for records to already exist in the tables, i.e., if the Client Name in the CSV already has an assigned ID in the Client Table, this ID should be used in the Bridge table. This is because I have to do a one-time batch upload of a million line of data, and then insert a few thousands line of data daily.
I have also already created the tables, they are in the database, just empty at the moment
You would do this in the database (or via database commands in Python). The data never needs to be loaded into Python.
Load the purchases.csv table into a staging table in the database. Then be sure you have your tables defined:
create table clients (
clientId int generated always as identity primary key,
client varchar(255)
);
create table items (
itemId int generated always as identity primary key,
item varchar(255)
);
create table clientItems (
clientItemId int generated always as identity primary key,
clientId int references clients(clientId),
itemId int references items(itemId)
);
Note that the exact syntax for these depends on the database. Then load the tables:
insert into clients (client)
select distinct s.client
from staging s
where not exists (select 1 from clients c where c.client = s.client);
insert into items (item)
select distinct s.item
from staging s
where not exists (select 1 from items i where i.item = s.item);
I'm not sure if you need to take duplicates into account for ClientItems:
insert into ClientItems (clientId, itemId)
select c.clientId, i.itemId
from staging s join
clients c
on s.client = c.client join
items i
on s.item = i.item;
If you need to prevent duplicates here, then:
where not exists (select 1
from clientitems ci join
clients c
on c.clientid = ci.clientid join
items i
on i.itemid = ci.itemid
where c.client = s.client and i.item = s.item
);
I want to make a Python (SQLAlchemy) program that takes the ID from a table according to a value.
For example, I have a table like this :
-------------------------------
| ID | Name |
-------------------------------
| 1 | Paul |
-------------------------------
| 2 | Paul |
-------------------------------
| 3 | John |
-------------------------------
And I want to retrieve the IDs where the name is Paul.
The code I was doing was something like this :
list = session.query(Table).filter_by(Name='Paul')
list_id = []
for tuple in list :
list_id.append(tuple.id)
for id in list_id :
print(id)
Is there any much easier solution?
Thanks!
You don't need the intermediate list to 'hold' the id, if you only use them once, just iterate directly over the query.
for row in session.query(Table).filter_by(Name='Paul'):
print(row.ID)
If you only need the ID, you can arrange for your query to return only that:
for ID in session.query(Table.ID).filter_by(Name='Paul'):
print(ID)
It might be a redundant question, but I have tried previous answers from other related topics and still can't figure it out.
I have a table Board_status looks like this (multiple status and timestamp for each board):
time | board_id | status
-------------------------------
2012-4-5 | 1 | good
2013-6-6 | 1 | not good
2013-6-7 | 1 | alright
2012-6-8 | 2 | good
2012-6-4 | 3 | good
2012-6-10 | 2 | good
Now I want to select all records from Board_status table, group all of them by board_id for distinct board_id, then select the latest status on each board. Basically end up with table like this (only latest status and timestamp for each board):
time | board_id | status
------------------------------
2013-6-7 | 1 | alright
2012-6-4 | 3 | good
2012-6-10 | 2 | good
I have tried:
b = Board_status.objects.values('board_id').annotate(max=Max('time')).values_list('board_id','max','status')
but doesn't seem like it is working. Still give me more than 1 record per board_id.
Which command should I use in Django to do this?
An update, this is the solution I use. Not the best, but it works for now:
b=[]
a = Board_status.objects.values('board_id').distinct()
for i in range(a.count()):
b.append(Board_status.objects.filter(board_id=a[i]['board_id']).latest('time'))
So I got all board_id, store into list a. Then for each board_id, do another query to get the latest time. Any better answer is still welcomed.
How will it work? You neither have filter nor distinct to filter out the duplicates. I am not sure if this can be easily done in a single django query. You should read more on:
https://docs.djangoproject.com/en/dev/ref/models/querysets/#django.db.models.query.QuerySet.distinct
https://docs.djangoproject.com/en/1.4/topics/db/aggregation/
If you can't do it in 1 raw sql query, you can't do it with an OR mapper either as it's built on top of mysql (in your case). Can you tell me how you would do this via raw SQL?