I've been working most of today on a function that creates a record in an sqlite table. I believe I am finally making some progress and it currently looks like this:
shopping = True
while shopping:
itemToAdd = input("Please enter the ID of the item to add to the basket: ")
basket.append(itemToAdd)
print(basket)
continueShop = input("Continue shopping?(y/n): ")
if continueShop == "n":
conn.execute("INSERT INTO Orders (UserID) VALUES (?)", (results[0][0],))
lastID = conn.execute("SELECT last_insert_rowid()")
conn.commit()
counter = 0
for items in basket:
createOrderItems = "INSERT INTO OrderItems (OrderID, ProductID) VALUES (?,?)"
conn.execute(createOrderItems, (lastID, basket[counter]))
counter = +1
conn.commit()
However, I am now encountering this error to do with lastID if I am reading the error correctly.
line 107, in
conn.execute(createOrderItems, (lastID, basket[counter])) sqlite3.InterfaceError: Error binding parameter 0 - probably unsupported type.
I have 2 tables that I am currently attempting to use:
Orders - contains only an orderID and userID
OrderItems- contains OrderItemsID, OrderID (which i need to be the same as the OrderID Just added) and ProductID(which needs to be taken from the list created and looped to make a record for each item in the "basket".
When you do
lastID = conn.execute("SELECT last_insert_rowid()")
lastID gets bound to an sqlite3.Cursor:
To retrieve data after executing a SELECT statement, you can either treat the cursor as an iterator, call the cursor’s fetchone() method to retrieve a single matching row, or call fetchall() to get a list of the matching rows.
The cursor cannot be converted automatically to the raw ID. You'll have to pull it out yourself, e.g. by doing something like
row = lastID.fetchone()
row_id = row[0]
Then use row_id in your query instead of lastID.
Related
I believe that I need to to expand the sql statement as the current one I have:
SELECT OrderID FROM Orders WHERE UserID = (?)
is likely to cause problems in the future when there are users with multiple orders. Is there a way that I can select and use the autoincrementing orderID in my orders table in combination?
For reference, my DB looks like this:
And the python I currently have is like this however I feel that it may have to be rewritten entirely.
results[0][0] is the ID of the current user from the Users table
elif userinput == "5":
basket = []
print(pd.read_sql_query('SELECT * FROM Products', conn))
shopping = True
while shopping:
itemToAdd = input("Please enter the ID of the item to add to the basket: ")
basket.append(itemToAdd)
print(basket)
continueShop = input("Continue shopping?(y/n): ")
if continueShop == "n":
conn.execute("INSERT INTO Orders (UserID) VALUES (?)", (results[0][0],))
conn.commit()
counter = 0
for items in basket:
createOrderItems = "INSERT INTO OrderItems (OrderID, ProductID) VALUES (?,?)"
currentOrder = ("SELECT OrderID FROM Orders WHERE UserID = (?) ", (results[0][0]))
conn.execute(createOrderItems, (currentOrder[0], basket[counter]))
counter = +1
conn.commit()
Because the database is built using the DB Browser application, I really have no clue how to access the parts of it that I need to ensure I am selecting the right record. Any help is appreciated, thanks.
Also, since the primary key of each table is autoincrementing I need a way to select only the record that was created just now using
conn.execute("INSERT INTO Orders (UserID) VALUES (?)",
However, I cannot think of a way to do so even though the record is unique in the orders table, as there is no way of knowing which order is the one that needs to be looked at as far as I can tell
You can use the lastrowid attribute of the cursor object to find the id of the inserted order. Then use a compound WHERE in the select statement (ie WHERE userid = something and orderid = something_else).
cHandler = myDB.cursor()
cHandler.execute('select UserId,C1,LogDate from DeviceLogs_12_2019') // data from remote sql server database
curs = connection.cursor()
curs.execute("""select * from biometric""") //data from my database table
lst = []
result= cHandler.fetchall()
for row in result:
lst.append(row)
lst2 = []
result2= curs.fetchall()
for row in result2:
lst2.append(row)
t = []
r = [elem for elem in lst if not elem in lst2]
for i in r:
print(i)
t.append(i)
for i in t:
frappe.db.sql("""Insert into biometric(UserId,C1,LogDate) select '%s','%s','%s' where not exists(select * from biometric where UserID='%s' and LogDate='%s')""",(i[0],i[1],i[2],i[0],i[2]),as_dict=1)
I am trying above code to insert data into my table if record not exists but getting error :
pymysql.err.ProgrammingError: (1064, "You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near '1111'',''in'',''2019-12-03 06:37:15'' where not exists(select * from biometric ' at line 1")
Is there anything I am doing wrong or any other way to achieve this?
It appears you have potentially four problems:
There is a from clause missing between select and where not exists.
When using a prepared statement you do not enclose your placeholder arguments, %s, within quotes. Your SQL should be:
Your loop:
Loop:
t = []
r = [elem for elem in lst if not elem in lst2]
for i in r:
print(i)
t.append(i)
If you are trying to only include rows from the remote site that will not be duplicates, then you should explicitly check the two fields that matter, i.e. UserId and LogDate. But what is the point since your SQL is taking care of making sure that you are excluding these duplicate rows? Also, what is the point of copying everything form r to t?
SQL:
Insert into biometric(UserId,C1,LogDate) select %s,%s,%s from DUAL where not exists(select * from biometric where UserID=%s and LogDate=%s
But here is the problem even with the above SQL:
If the not exists clause is false, then the select %s,%s,%s from DUAL ... returns no columns and the column count will not match the number of columns you are trying to insert, namely three.
If your concern is getting an error due to duplicate keys because (UserId, LogDate) is either a UNIQUE or PRIMARY KEY, then add the IGNORE keyword on the INSERT statement and then if a row with the key already exists, the insertion will be ignored. But there is no way of knowing since you have not provided this information:
for i in t:
frappe.db.sql("Insert IGNORE into biometric(UserId,C1,LogDate) values(%s,%s,%s)",(i[0],i[1],i[2]))
If you do not want multiple rows with the same (UserId, LogDate) combination, then you should define a UNIQUE KEY on these two columns and then the above SQL should be sufficient. There is also an ON DUPLICATE KEY SET ... variation of the INSERT statement where if the key exists you can do an update instead (look this up).
If you don't have a UNIQUE KEY defined on these two columns or you need to print out those rows which are being updated, then you do need to test for the presence of the existing keys. But this would be the way to do it:
cHandler = myDB.cursor()
cHandler.execute('select UserId,C1,LogDate from DeviceLogs_12_2019') // data from remote sql server database
rows = cHandler.fetchall()
curs = connection.cursor()
for row in rows:
curs.execute("select UserId from biometric where UserId=%s and LogDate=%s", (ros[0], row[2])) # row already in biometric table?
biometric_row = curs.fetchone()
if biometric_row is None: # no, it is not
print(row)
frappe.db.sql("Insert into biometric(UserId,C1,LogDate) values(%s, %s, %s)", (row[0],row[1],row[2]))
I am retrieving a value from a SQLITE table and then using this value to retrieve data from another table using a SELECT FROM WHERE statement. I cannot use the retrieved value to query the other table even though the values appear to match when I retrieve this value independently from the table I am querying. I get Error binding parameter 0 - probably unsupported type. I am passing the value I believe correctly from what I've read in the docs, but there is obviously something wrong.
Edit: The ulitmate goal is to select the Name and EndTime then insert the EndTime value in another table in the same db if a column value = Name in that table. Added Update code below that gives an idea of what I'm attempting to accomplish.
When I print nameItem it appears this is a unicode string, (u'Something') is how it appears. I don't know if this is an encoding issue, but I have used similar queries before and not run into this issue.
I have tried to use the text I am searching for directly in the SELECT query and this works, but when passing it as a variable I still get unsupported type.
c.execute('SELECT Name FROM Expected WHERE Timing = 1')
timeList = c.fetchall()
for i in range(len(timeList)):
nameItem = timeList[i]
c.execute('SELECT "EndTime" FROM Expected WHERE "Name" = ?', (nameItem,))
end = c.fetchone()
conn = sqlite3.connect(otherDb)
c = conn.cursor()
c.execute('UPDATE Individuals SET Ending = end WHERE NAME = NameItem')
I expect this to retrieve a time associated with the current value of nameItem.
The first fix to try is to use string instead of tuple (note timeList is list of tuples):
nameItem = timeList[i][0]
But better is to not use len, something like:
c.execute('SELECT Name FROM Expected WHERE Timing = 1')
timeList = c.fetchall()
for elem in timeList:
nameItem = elem[0]
c.execute('SELECT "EndTime" FROM Expected WHERE "Name" = ?', (nameItem,))
end = c.fetchone()
And even better is to fetch EndTime in the first query (looks acceptable in given context):
c.execute('SELECT Name, EndTime FROM Expected WHERE Timing = 1')
timeList = c.fetchall()
for elem in timeList:
name = elem[0]
end_time = elem[1]
I'm using Postgresql. I have code as follows:
vehicles = get_vehicle_references()
for vehicle in vehicles:
insert_into_db(vehicle[0], vehicle[1])
And get_vehicle_references() is as follows:
def get_vehicle_references():
conn = db_connection()
cur = conn.cursor()
try:
s = "SELECT reference, reference_url FROM vehicles v WHERE NOT EXISTS (select reference from daily_run_vehicle rv WHERE handled = %s AND retries < %s AND rv.timestamp::timestamp::date = %s AND v.reference=reference) ORDER BY id DESC"
cur.execute(s, (False, 5, today))
return cur.fetchall()
except Exception as e:
capture_error(str(e))
conn.close()
Thus the query in that function is as follows:
SELECT reference, reference_url
FROM vehicles v
WHERE NOT EXISTS
(select reference from daily_run_vehicle rv
WHERE handled = False
AND retries < 5
AND rv.timestamp::timestamp::date = str(date.today())
AND v.reference=reference)
ORDER BY id DESC
I get the reference and the reference_url and then I insert them in the database.
I repeat this more than 500 times every day. What I want is as follows:
When it executes the first time, if the reference from vehicles in daily_run_vehicle has handled field = False and retries < 5, I want to insert that reference in database.
When it goes to that query again that same day, I want to check if that
reference in daily_run_vehicle has handled field = False and retries < 5 has already been added to daily_run_vehicle table that day.
If its already been added I want to skip that record.
I'm lost here. Can anybody help me please.
Add a UNIQUE INDEX to the table you are inserting records into:
CREATE UNIQUE INDEX yourtable_idx on yourtable (date, reference, reference_url);
Notice that the date is part of the unique index. So each unique (date, reference, reference_url) tuple can only be inserted once.
Then you can insert records into yourtable using the SQL:
INSERT INTO table (date, reference, reference_url)
VALUES (%s, %s, %s)
ON CONFLICT (date, reference, reference_url) DO NOTHING
The ON CONFLICT ... DO NOTHING clause tells Postgresql to ignore the insert if the
(date, reference, reference_url) tuple is already in yourtable.
Once you've set up your Postgresql table with a unique index, you can insert with impunity and not have to worry about creating duplicates in your Python code. It's much easier and more reliable to enforce this at the database level than it is to code this consistently at the application level.
I've managed to query and get results for values from a list. But say i have the genres ["Action", "Comedy", "Romance"] in the list I am using to query. My code below returns all the records which are action, romance and comedy whereas I want it to return only the records that meet all three genres. How can I do this?
The database has a field called genre in which there are multiple genres for each record.
Code:
with sqlite3.connect("animeData.db") as db: #anime database
c = db.cursor()
c.execute("select * from t where genre in %s" % repr(self.genreResults).replace('[','(').replace(']',')') ) #self.genreResults is the list containing the genres i want to query
results = c.fetchall()
print(results)
db.commit()
db.close()
Output:
Database:
It's returning all animes which have genres of action and adventure whereas I only want it to return animes which have both. Any suggestions?
If I understand correctly, 'like' clauses meet your requirements but you don't know how to convert list to 'like' clauses.
So the codes will be like below:
import sqlite3
with sqlite3.connect("animeData.db") as db: #anime database
c = db.cursor()
#self.genreResults is the list containing the genres i want to query
user_input = self.genreResults # for example: ['Action','Drama']
convert_to_like_conditions = ' and '.join(list(map(lambda item : "genre like '%{0}%'".format(item), user_input)))
print(convert_to_like_conditions) #Output: genre like '%Action%' and genre like '%Drama%'
c.execute("select * from t where %s" % convert_to_like_conditions )
results = c.fetchall()
print(results)
db.commit()
You can refer to the documents on map, string join.
And you probably notice I removed the last line (db.close) of your codes, you can refer to Understanding Python's "with" statement.