Develop Reference

Python-sqlite3 : How to put a python variable in SELECT statement

How can I put a python variable just after the SELECT. The idea is to create a python function with three arguments where you can choose what you what (here, it's the age) from whom (here, it's Mike and James)
conn = sqlite3.connect('test.s3db')
cur = conn.cursor()
cur.execute('''DROP TABLE IF EXISTS people''')
cur.execute('''CREATE TABLE IF NOT EXISTS people
(id INTEGER,
name TEXT,
surname TEXT,
age INTEGER,
alone INTEGER DEFAULT 0);''')
def add_people(id, name, surname, age, alone=0):
cur.executemany('INSERT INTO people (id, name, surname, age, alone) VALUES (?,?,?,?,?)', [(id, name, surname, age, alone)])
conn.commit()
add_people(1, 'SMITH','James',45)
add_people(2,'JOHNSON','Mike',75)
cur.execute('''SELECT (?) FROM people WHERE surname = (?) OR surname = (?)''', ('age','Mike', 'James'))
print(cur.fetchall())
My code return:
[('age',), ('age',)]
instead of :
[(75,), (45,)]
EDIT : I want that what is selected is a variable and not directly written in the query. My goal is to make a function like this one :
def query(what, who_1, who_2):
cur.executemany('''SELECT (?) FROM people WHERE surname = (?) OR surname = (?)''', (what, who_1, who_2))
return cur.fetchall()
Thank you in advance for your answers !

This takes the data you need as argument of select_data_of
import sqlite3
def add_people(id, name, surname, age, alone=0):
cur.executemany('INSERT INTO people (id, name, surname, age, alone) VALUES (?,?,?,?,?)', [(id, name, surname, age, alone)])
conn.commit()
def select_data_of(names, data="age"):
select = []
for name in names:
cur.execute(f'''SELECT [{data}] FROM people WHERE surname = (?)''', (name, ))
select.append(cur.fetchall()[0])
return select
with sqlite3.connect('test.s3db') as conn:
cur = conn.cursor()
cur.execute('''DROP TABLE IF EXISTS people''')
cur.execute('''CREATE TABLE IF NOT EXISTS people
(id INTEGER,
name TEXT,
surname TEXT,
age INTEGER,
alone INTEGER DEFAULT 0);''')
add_people(2,'JOHNSON','Mike',75)
add_people(1, 'SMITH','James',45)
data = select_data_of(("Mike", "James"), data="age")
print(data)
OUT:
[(75,), (45,)]

I think your select query is wrong because of the 'ages' parameter, this new query will work.
Try
cur.execute("SELECT [age] FROM people WHERE surname = 'Mike' OR surname = 'James")

Updating table SQlite

def update_employee_table(self,ids,username, password, first_name, last_name,age, phone_number, department, city, address):
self.connect()
print(self)
self.c.execute("UPDATE employee SET (username,password,first_name,last_name,age,phone_number,department,city,address) VALUES (?,?,?,?,?,?,?,?,?) WHERE id = ?",(username, password, first_name, last_name,age, phone_number, department, city, address),(ids))
self.commit()
i am trying to update a table by the id and insert the values that i have passed to the function , the syntax of the update confuses me a little and i get this error : TypeError: function takes at most 2 arguments (3 given)

You must include the variable ids inside the tuple that you pass as the 2nd argument of execute() and not as a 3d argument.
Also, the UPDATE statement is syntactically wrong as it is written with VALUES.
Write it like this:
sql = """
UPDATE employee
SET (username,password,first_name,last_name,age,phone_number,department,city,address) = (?,?,?,?,?,?,?,?,?)
WHERE id = ?
"""
self.c.execute(sql, (username, password, first_name, last_name, age, phone_number, department, city, address, ids))

SQLite3 upsert always fails when called from python

I’ve a sqlite3 db: phone book (name_id, phone_nb). I want to
insert (“Kyl”, +33661) if Kyl-entry doesn’t exist yet,
or, if Kyl already exists, I want to update his phone number to +33770. This is called upsert.
SQLite upsert is:
INSERT INTO table(...)
VALUES (...)
ON CONFLICT (...) DO UPDATE SET expression
My issue:
The above statement works perfectly when I use sqlite3, but it doesn’t work at all when I call the same from python.
On the other hand, if from python I use pure INSERT INTO table VALUES it works (without ON CONFLICT)
In addition, if from python I use classical UPDATE table SET col WHERE condition, it works too
Using SQLite upsert, I always have the same error: near "ON": syntax error
This is my table:
CREATE TABLE phone_book (
author_id INTEGER PRIMARY KEY
UNIQUE,
short_name TEXT NOT NULL,
join_date DATE NOT NULL,
email TEXT NOT NULL,
phone_nb STRING
);
From SQL Studio, I run
INSERT INTO phone_book(author_id, short_name, join_date, email, phone_nb)
VALUES (13, "kyl", "2020-12-20", "kyl#domain.net", 33670668832)
ON CONFLICT(author_id) DO UPDATE SET phone_nb=excluded.phone_nb;
This insert works. Then as Kyl changed his phone nb, I update his phone nb, using the same:
INSERT INTO phone_book(author_id, short_name, join_date, email, phone_nb)
VALUES (13, "kyl", "2020-12-20", "kyl#domain.net", 33677755231)
ON CONFLICT(author_id) DO UPDATE SET phone_nb=excluded.phone_nb;
This update work too. Everything’s in place! It’s time now to run all that from python. The bad news is that, when called from python, this precise statement doesn’t work at all.
What I’ve tried all the combinations:
cursor.execute(...)
cursor.executemany(...)
With explicit parameters
With ‘?’ placeholder
I always have the same error: near "ON": syntax error.
My non-working code with ‘?’ placeholder:
try:
sqliteConnection = sqlite3.connect('my.db')
cursor = sqliteConnection.cursor()
#print("Connected to SQLite")
author_id = 13
short_name = "mike"
join_date = "2021-01-12"
email = "mike#domain.net"
phone_nb = "00336"
tupple = []
tupple.append((author_id, short_name, join_date, email, phone_nb))
statement_ON_CONF = """INSERT INTO phone_book(author_id, short_name, join_date, email, phone_nb)
VALUES(?,?,?,?,?)
ON CONFLICT(author_id) DO UPDATE SET phone_nb=excluded.phone_nb;"""
print("statement_ON_CONF: " + statement_ON_CONF) # check my statement
cursor.executemany(statement_ON_CONF, tupple)
sqliteConnection.commit()
except sqlite3.Error as error:
print("Failed to insert or update into sqlite table: ", error)
finally:
if (sqliteConnection):
sqliteConnection.close()
#print("The SQLite connection is closed")
On the other hand, using pure INSERT and then UPDATE all's ok: my working code:
try:
sqliteConnection = sqlite3.connect('my.db')
cursor = sqliteConnection.cursor()
author_id = 2
short_name = "mike"
join_date = "2021-01-12"
email = "mike#domain.net"
phone_nb = "00336"
# Insert a new entry: Mike
statement = """INSERT INTO phone_book(author_id, short_name, join_date, email, phone_nb)
VALUES(?,?,?,?,?)"""
print("statement: " + statement)
cursor.execute(statement, (author_id, short_name, join_date, email, phone_nb))
sqliteConnection.commit()
# Update Mike phone nb
phone_nb = "+3310"
statement_ON_CONF = """INSERT INTO phone_book(author_id, short_name, join_date, email, phone_nb)
VALUES(?,?,?,?,?)
ON CONFLICT(author_id) DO UPDATE SET phone_nb=excluded.phone_nb;"""
statement_UPDATE = "UPDATE phone_book SET phone_nb=? WHERE author_id=?;"
cursor.execute(statement_UPDATE, (phone_nb, author_id))
sqliteConnection.commit()
except sqlite3.Error as error:
print("Failed to insert or update into sqlite table: ", error)
finally:
if (sqliteConnection):
sqliteConnection.close()
I use SQLite version 3.34.0 2020-12-01, and python version 3.7.2rc1, on Windows 7 Pro
Does anyone know why upsert always throws an error when called from python? Thanks!

According to a comment:
"SQLite supports UPSERT since version 3.24.0" – forpas.

You try to pass this tupple:
[(author_id, short_name, join_date, email, phone_nb)]
as VALUES in your statement, this results to error as it's not readable from sqlite
Try this instead:
statement_ON_CONF = """INSERT INTO phone_book(author_id, short_name, join_date, email, phone_nb)
VALUES ?
ON CONFLICT(author_id) DO UPDATE SET phone_nb=excluded.phone_nb;"""
print("statement_ON_CONF: " + statement_ON_CONF) # check my statement
cursor.executemany(statement_ON_CONF, tupple[0])

Iterate Over a list, deleting every value that matches a database

I have been working on this function in python. I intend for it to iterate over a list of phone numbers, checking with a database to see whether the number has been used yet or not. If it has been used, it should remove the phone number from the list and choose another and check the new one until an unused one has been found and return the unused one. If it has not been used, it should simply just return the number. However, after one run, it picks a number, checks it, runs, and then enters it into the database. The next run deletes the previously used number, and picks another that hasn't been used. It continues to run and enters this number into the database. The third run does not delete the previously used number from the list, but it still picks a new one regardless. Although this still works, when the numbers run out, since there are no others to pick, it continues using the last number in the list for every following run of the script. Sorry if the code is a bit sloppy right now, I am in a bit of a rush and this is only a script I have been messing around with. I hope this is clear, and not too confusing. If I need to clear any confusion, I will be glad too.
Edit: Sorry, I forgot to mention that these phone numbers are constantly grabbed from a website by another script. These set of numbers listed below is just a dummy set for testing. So in the end, I am needing to see if these recently grabbed numbers have been used by checking with the database tables.
import random
import names
##############################Information Variables##################################
emailAddress = "Fakeemail#mail.com"
titleValues = [0,1] #0 is 'Mr.', 1 is 'Mrs.'
country = 'Schwifty'
title = random.choice(titleValues)
#Generate a random name based on gender
if title == 1:
firstName = names.get_first_name(gender= 'female')
else:
firstName = names.get_first_name(gender= 'male')
lastName = names.get_last_name()
fullName = firstName + ' ' + lastName
print(fullName)
phoneNumber = '111-222-3333'
#########################################################
import sqlite3
import time
import datetime
conn = sqlite3.connect('accounts.db')
c = conn.cursor()
def createTable():
c.execute('CREATE TABLE IF NOT EXISTS accounts(Email TEXT, Name TEXT, Title TEXT, PhoneNumber TEXT, Country TEXT, DateStamp TEXT)')
def dynamic_data_entry(email, name, title, phone, country):
unix = time.time()
date = str(datetime.datetime.fromtimestamp (unix).strftime('%Y-%m-%d %H:%M:%S'))
c.execute('INSERT INTO accounts (Email, Name, Title, PhoneNumber, Country, DateStamp) VALUES (?, ?, ?, ?, ?, ?)', (email, name, title, phone, country, date))
conn.commit()
createTable()
#################################TEST NUMBER CHECK###########################
phoneNumbers = ['111-222-3333', '444-555-6666', '777-888-9999', '123-456-7890', '321-321-321']
def checkNumber(a):
c.execute("SELECT * FROM accounts WHERE PhoneNumber = ?", (a,))
row = c.fetchall()
if row:
print("Phone number has already been used, choosing another and deleting current from list.")
phoneNumbers.remove(a)
a = random.choice(phoneNumbers)
checkNumber(a)
elif row == False:
print("Number is fresh and new, using " + a)
return a
elif row == None:
print('No new phone numbers to use, exiting... ')
exit()
# for num in phoneNumbers:
# checkNumber(num)
# print(num)
checkNumber(phoneNumber)
print(phoneNumbers)
print('working')
##########################################
# INSERT DATA TO DB #
##########################################
#Insert information to database in this order: email, name, title, phone, country
dynamic_data_entry(emailAddress, fullName, title, phoneNumber, country)
conn.commit()
c.close()
conn.close()

Don’t do this. Populate a table with your phone numbers and update each phone number record with a field like ‘used’ once used.
Always keep state and data modeling in the database where possible. It is made for it.
Update in response to OP:
Create a separate table for phone numbers and replace your number field in the accounts table with a foreign key id to the primary key of the phone number table. This is called maintaining an object model or data model, so that if you want to query accounts, you have the data you need via foreign key, and if you just want phone numbers you can query the phone numbers table directly.
This way your phone number ‘objects’ can have their own attributes like ‘already called’ or ‘on do not call list’ without muddying up your accounts ‘object’.
If you want to insert a new account, you should first insert your new phone number 'object' into the phone number table and return the id, and then use that in your account insert.

python sqlite3 query with AND

I am trying to build a simple Address book GUI that has a wx.listbox, that holds all the names in the book, first and last. Once clicked, it will return the information attached to the name from a database file. Right now I have it working by just the last name, I am trying to match first and last names. I am not, really, familiar with the SQLite 3 commands and syntax.
The function is below, this works fine now, but I want to change the query to something like:
select * from AddressBook where Last like names[0] and First like names[1]
Any help would be great!
def onListBox(self, event):
name = event.GetEventObject().GetStringSelection()
names = name.split(',')###names[0]=Last name, names[1] = first name
cursor = db.cursor()
cursor.execute("select * from AddressBook where Last like ?",('%'+names[0]+'%',) )
result = cursor.fetchall()
return result

The query from your comment should work.
Here is a small working example:
import sqlite3
conn = sqlite3.connect("test.sql")
cursor = conn.cursor()
cursor.execute("create table address_book (first_name text, last_name text)")
names = [["John", "Smith"], ["Jane", "Smith"]]
for first_name, last_name in names:
cursor.execute("insert into address_book (first_name, last_name) values (?, ?)", (first_name, last_name))
cursor.execute("select * from address_book where first_name like ? and last_name like ?", ("%" + names[0][0] + "%", "%" + names[0][1] + "%"))
print(cursor.fetchall())
It prints:
[('John', 'Smith')]