I was creating a database by using SQLite3.
and my python version is 2.7.5.
Before creating a database I simply created example database and tested it whether it will function well or not.
and It failed that the id wasn't incremented even I
declared it as a serial type.
I created simple database:
import sqlite3
con = sqlite3.connect('sample.db')
cur = con.cursor()
cur.execute("""CREATE TABLE sample(id serial,test real)""")
cur.execute("""INSERT INTO sample(test) VALUES(?)""",(3,))
cur.execute("""INSERT INTO sample(test) VALUES(?)""",(6,))
cur.execute("""INSERT INTO sample(test) VALUES(?)""",(8,))
con.commit()
Then I fetched all data:
data = cur.execute("""SELECT * from sample""")
t = data.fetchall()
In [33]: t
Out[33]: [(None, 3.0), (None, 6.0), (None, 8.0)]
I expected this:Out[33]: [(1, 3.0), (2, 6.0), (3, 8.0)]
However, As you can see, all of the id element was None
How can I solve this problem ?
I know I can do by just incrementing a variable and Insert it.
like this:
id += 1
cur.execute("""CREATE TABLE sample(id serial,test real)""")
id += 1
cur.execute("""INSERT INTO sample(test) VALUES(?)""",(3,))
id += 1
cur.execute("""INSERT INTO sample(test) VALUES(?)""",(6,))
However, isn't this awful ? I don't want to do it.
I'd like to make my code clear and smart.
There is no serial data type in SQLite. The correct way to create the table is with AUTOINCREMENT in SQLite:
CREATE TABLE sample ( id INTEGER PRIMARY KEY AUTOINCREMENT NOT NULL,
test REAL);
You'd need to mark a column as INTEGER PRIMARY KEY or INTEGER PRIMARY KEY AUTOINCREMENT to get auto-incrementation behaviour.
The difference between those two types is subtle; the latter form will never reuse IDs. See the ROWID and the INTEGER PRIMARY KEY documentation.
Related
I'm comparing images between thousands of users who can have between 1 and 12 photos. The comparison between two photos returns a score that I need to be stored so I don't make the comparison twice.
What is the best way of storing it?
I thought about storing in a table with one photo per row/column but this can quickly get out of hand
Multi-indexed pandas if you want to work in memory, like so:
df = pd.DataFrame(index=[['Alice/foo.png'], ['Bob/bar.png']], columns=['user1', 'user2', 'score'], data=[['Alice', 'Bob', 42.0]])
df.index.names = ['photo1', 'photo2']
df
user1 user2 score
photo1 photo2
user1/foo.png user2/bar.png Alice Bob 42.0
SQLite if you want to work on disk, like so
import sqlite3
# The important part: defining the table
conn = sqlite3.Connection('photos.sqlite')
c = conn.cursor()
c.execute('CREATE TABLE IF NOT EXISTS photos (photoid INTEGER PRIMARY KEY, photopath TEXT UNIQUE, userid INT)')
c.execute('CREATE TABLE IF NOT EXISTS photoscores (photoid1 INTEGER, photoid2 INTEGER, userid1 INT, userid2 INT, score REAL)')
c.execute('CREATE UNIQUE INDEX photopair on photoscores (photoid1, photoid2)')
conn.commit()
# Example of populating the table
sql = ('INSERT INTO photos(photopath, userid) VALUES ("foo.png", 1)',
'INSERT INTO photos(photopath, userid) VALUES ("bar.png", 2)')
for statement in sql:
c.execute(statement)
conn.commit()
sql = 'SELECT photoid FROM PHOTOS'
c.execute(sql)
values = [v[0] for v in enumerate(c.fetchall())]
# This is more complicated than it needs to be, especially since
# values will always be sorted if this code is run, but I'm just
# emphasizing the need to keep the photo ids and user ids aligned
sql = ('INSERT OR REPLACE INTO photoscores VALUES (%s, %s, %s, %s, 42.0)'
% (tuple(sorted(values)) + ((1, 2) if values == sorted(values) else (2, 1))))
c.execute(sql)
conn.commit()
import pandas as pd
pd.read_sql('SELECT * FROM photoscores', conn)
photoid1 photoid2 userid1 userid2 score
0 0 1 1 2 42.0
If you use the SQL code, I'd suggest always sorting the pair of photo IDs you compare.
The key thing is that you want something with an underlying hash map that will quickly tell you whether an existing pair of photos has already been compared.
Given the schema
CREATE TABLE `test` (
`name` VARCHAR(255) NOT NULL,
`text` TEXT NOT NULL,
PRIMARY KEY(`name`)
)
I would like to insert new data in such a way that if a given name exists, the name I am trying to insert is changed. I've checked the SQLite docs, and all I could find is INSERT OR REPLACE, which would change the text of the existing name instead of creating a new element.
The only solution I can think of is
def merge_or_edit(curr, *data_tuples):
SELECT = """SELECT COUNT(1) FROM `test` WHERE `name`=?"""
INSERT = """INSERT INTO `test` (`name`, `text`) VALUES (?, ?)"""
to_insert = []
for t in data_tuples:
while curr.execute(SELECT, (t[0],)).fetchone()[0] == 1:
t = (t[0] + "_", t[1])
to_insert.append(t)
curr.executemany(INSERT, to_insert)
But this solution is extremely slow for large sets of data (and will crash if the rename takes its name to more than 255 chars.)
What I would like to know is if this functionality is even possible using raw SQLite code.
I am creating one table per user in my database and later storing data specific to that user. Since I have 100+ users, I was looking to automate the table creation process in my Python code.
Much like how I can automate a row insertion in a table, I tried to automate table insertion.
Row insertion code:
PAYLOAD_TEMPLATE = (
"INSERT INTO metadata "
"(to_date, customer_name, subdomain, internal_users)"
"VALUES (%s, %s, %s, %s)"
)
How I use it:
connection = mysql.connector.connect(**config)
cursor = connection.cursor()
# Opening csv table to feed data
with open('/csv-table-path', 'r') as weeklyInsight:
reader = csv.DictReader(weeklyInsight)
for dataDict in reader:
# Changing date to %m/%d/%Y format
to_date = dataDict['To'][:5] + "20" + dataDict['To'][5:]
payload_data = (
datetime.strptime(to_date, '%m/%d/%Y'),
dataDict['CustomerName'],
dataDict['Subdomain'],
dataDict['InternalUsers']
)
cursor.execute(PAYLOAD_TEMPLATE, payload_data)
How can I create a 'TABLE_TEMPLATE' that can be executed in a similar way to create a table?
I wish to create it such that I can execute the template code from my cursor after replacing certain fields with others.
TABLE_TEMPLATE = (
" CREATE TABLE '{customer_name}' (" # Change customer_name for new table
"'To' DATE NOT NULL,"
"'Users' INT(11) NOT NULL,"
"'Valid' VARCHAR(3) NOT NULL"
") ENGINE=InnoDB"
)
There is no technical¹ need to create a separate table for each client. It is simpler and cleaner to have a single table, e.g.
-- A simple users table; you probably already have something like this
create table users (
id integer not null auto_increment,
name varchar(50),
primary key (id)
);
create table weekly_numbers (
id integer not null auto_increment,
-- By referring to the id column of our users table we link each
-- row with a user
user_id integer references users(id),
`date` date not null,
user_count integer(11) not null,
primary key (id)
);
Let's add some sample data:
insert into users (id, name)
values (1, 'Kirk'),
(2, 'Picard');
insert into weekly_numbers (user_id, `date`, user_count)
values (1, '2017-06-13', 5),
(1, '2017-06-20', 7),
(2, '2017-06-13', 3),
(1, '2017-06-27', 10),
(2, '2017-06-27', 9),
(2, '2017-06-20', 12);
Now let's look at Captain Kirk's numbers:
select `date`, user_count
from weekly_numbers
-- By filtering on user_id we can see one user's numbers
where user_id = 1
order by `date` asc;
¹There may be business reasons to keep your users' data separate. A common use case would be isolating your clients' data, but in that case a separate database per client seems like a better fit.
I am trying to add values to a 'pending application table'. This is what I have so far:
appdata = [(ID,UNIQUE_IDENTIFIER,(time.strftime("%d/%m/%Y"),self.amount_input.get(),self.why_input.get())]
self.c.execute('INSERT into Pending VALUES (?,?,?,?,?)', appdata)
self.conn.commit()
I need to set a value for 'UNIQUE_IDENTIFIER', which is a primary key in a sqlite database.
How can I generate a unquie number for this value?
CREATE TABLE Pending (
ID STRING REFERENCES StaffTable (ID),
PendindID STRING PRIMARY KEY,
RequestDate STRING,
Amount TEXT,
Reason TEXT
);
two ways to do that:
1-First
in python you can use uuid module example:
>>> import uuid
>>> str(uuid.uuid4()).replace('-','')
'5f202bf198e24242b6a11a569fd7f028'
note : a small chance to get the same str so check for object exist with the same primary key in the table before saving
this method uuid.uuid4() each time return new random
for example:
>>> ID=str(uuid.uuid4()).replace('-','')
>>>cursor.execute("SELECT * FROM Pending WHERE PendindID = ?", (ID,))
>>>if len(data)==0:
#then save new object as there is no row with the same id
else:
#create new ID
2-second
in sqlite3 make a composite primary key according to sqlite doc
CREATE TABLE Pending (
column1,
column2,
column3,
PRIMARY KEY (column1, column2)
);
Then make sure of uniqueness throw unique(column1, column2)
When ever i try to insert data to my database, it's like it simply just overwrite whole file or not saving it correctly.
My thoughts about this script was that if i ran it multiple times it would output this:
(1, 126)
(2, 127)
(3, 126)
(4, 127)
I also now that if i doesn't create table within the script, it gives me an error that the table doesn't exist, even though the table was created, last time i ran the script.
import sqlite3
# Connecting to the database file
conn = sqlite3.connect("MyDB.db")
c = conn.cursor()
# Check if table exists
c.execute("DROP TABLE IF EXISTS t1")
# Create table
c.execute('''CREATE TABLE t1(
a INTEGER PRIMARY KEY,
b INTEGER);''')
# Insert value
c.execute('''INSERT INTO t1 VALUES(NULL,126);''')
c.execute('''INSERT INTO t1 VALUES(NULL,127);''')
# Save changes
conn.commit()
for row in c.execute('SELECT * FROM t1'):
print(row)
# Output:
# (1, 126)
# (2, 127)
conn.close()
I really want this to work, since i'm working on a school project where my product is to make a functional store where you can add, remove products and sort them by date they were added.
Two issues:
You are dropping the table if exists, this will always deletes the table:
c.execute("DROP TABLE IF EXISTS t1")
You are re-creating the table after dropping:
c.execute('''CREATE TABLE t1(
a INTEGER PRIMARY KEY,
b INTEGER);''')
You could comment above codes and re-run your script to get the output you want. But, in the final version ensure to check db and table exists before writing to it.