I have a plugin to a game server that writes down changes made to a map.The database contains entries formatted like this - id INTEGER PRIMARY KEY,matbefore INTEGER, matafter INTEGER, name VARCHAR(50), date DATE. I am trying to create a function that, when given a column name, an integer, string, or tuple of an integer or string, and a keyword, will find the selected entries. so far, this is the code that I have come to -
def readdb(self,keyword,column,returncolumn = "*"):
self.memwrite
if isinstance(keyword, int) or isinstance(keyword,str):
entry = [keyword]
qmarks = ("? OR " * len(entry))[:-4]
statement = 'SELECT all {0} FROM main WHERE {1} is {2}'.format(returncolumn,column,qmarks)
print(qmarks)
self.memcursor.execute(statement, entry)
return(self.memcursor.fetchall())
keyword is a keyword to search for, column is teh column to search in, and returncolumn is the column to return So I was wondering why this code always fetches no rows, EG - Returns None, no matter what I put for the function. It seems to work fine if I do these things in the console, but not if I wrap them in a function
If entry is a list (like in yesterday's question) , it's not going to work.
>>> returncolumn = "*"
>>> column = "name"
>>> entry = ["Able", "Baker", "Charlie"]
>>> qmarks = ("? OR " * len(entry))[:-4]
>>> statement = 'SELECT all {0} FROM main WHERE {1} is {2}'.format(returncolumn,
column,qmarks)
>>> print statement
SELECT all * FROM main WHERE name is ? OR ? OR ?
and what SQLite will see is:
SELECT all * FROM main WHERE name is 'Able' OR 'Baker' OR 'Charlie'
which is not valid syntax because you need =, not is.
Even if you fix that then (using an integer query for example):
SELECT all * FROM main WHERE id = 1 or 2 or 3
you will get mysterious results because that means WHERE ((id = 1) or 2) or 3), not what you think it does ... you need WHERE id = 1 or id = 2 or id = 3 or (reverting to yesterday's question) WHERE id IN (1,2,3)
def readdb(self,keyword,column,returncolumn = "*"):
self.memwrite # 1.
if isinstance(keyword, int) or isinstance(keyword,str): # 2.
entry = [keyword] # 3.
qmarks = ("? OR " * len(entry))[:-4] # 4.
statement = 'SELECT all {0} FROM main WHERE {1} is {2}'.format(returncolumn,column,qmarks)
print(qmarks) # 5.
self.memcursor.execute(statement, entry)
return(self.memcursor.fetchall())
Not sure what this is supposed to
do. Did you mean self.memwrite()?
Can be changed to if
isinstance(keyword, (int,str))
Or better yet, don't test the type.
As you've written it, keyword can
not be a unicode string. Why
restrict like this? In this case I
think it would be better to use
a try...except... block to catch the subsequent
error than to restrict type.
So at best len(entry) = 1. Notice
also, its possible to never reach
this line if keyword is not of type
int or str. In that case, you would
get an error on line (4) since entry
would not be defined..
This could also be written as
qmarks = ' OR '.join(['?']*len(entry))
It avoids the need for the somewhat magic number 4 in ("? OR " *1)[:-4].
What are you seeing here? If it was
empty, that should have been a clue. Also it would be worth running
print(statement) to get a full picture of what is being sent to .execute().
Perhaps try
statement = 'SELECT {0} FROM main WHERE {1} = ?'.format(
returncolumn,column)
In particular, the is should be changed to =.
Related
I have this little code
cquery = "SELECT * FROM `workers` WHERE `Username` = (%s)"
cvalue = (usernameR,)
flash(cquery)
flash(cvalue)
x = c1.execute(cquery, cvalue)
flash(x)
usernameR is a string variable I got it's value from a form
x supposed to be the number of rows or some value but it returns none I need it's value for one if.
I tested it with a value that is in the table in one row so thats not the case the the value is not there or something. But if it's not there in that case the x should return 0 or something.
I cant work out what's the problem after several hours.
value of cvalue:
('Csabatron99',)
Edit for solution:
I needed to add the rowcount and fetchall to the code like this:
cquery = "SELECT * FROM `workers` WHERE `Username` = (%s)"
cvalue = (usernameR,)
flash(cquery)
flash(cvalue)
c1.execute(cquery, cvalue)
c1.fetchall()
a = c1.rowcount
cursor.execute() doesn't return anything in the normal case. If you use the multi=True argument, it returns an iterator used to get results from each of the multiple queries.
To get the number of rows returned by the query, use the rowcount attribute.
c1.execute(cquery, cvalue)
flash(c1.rowcount)
So i am trying to work out how to use sql lite with python by setting up a simple database full of different students and their courseworks and scores etc....
I am writing a function that updates my database for the score a student got given their id, unit id and the mark. However I have a column in my assignments table that is called mark!
This is what I thought would work....
def mark_assignment(database, studentid, assessmentid, mark):
mark = score
access_database(database,"UPDATE assignments SET mark = score, marked = 1 WHERE assignments.studentid = studentid AND assignments.assessmentid = assessmentid ")
All you have to do is add placeholders in your SQL line while executing.
str.format() is one of the string formatting methods in Python3, which
allows multiple substitutions and value formatting. This method lets
us concatenate elements within a string through positional formatting.
Each parameter in the format function replaces the corresponding placeholder in the string.
Here is a small example for the same:
#Code
first_name = "Heeth"
last_name = "Jain"
print("Hello {} {}".format(first_name, last_name))
#Output
Hello Heeth Jain
SQL Statement would look like:
"UPDATE assignments SET mark = {}, marked = 1 WHERE assignments.studentid = {} AND assignments.assessmentid = {}".format(mark, studentid, assessmentid)
Since you are using a single table in this command, you can also change the statement as follows (I just changed assignments.studentid to studentid and so on:
"UPDATE assignments SET mark = {}, marked = 1 WHERE studentid = {} AND assessmentid = {}".format(mark, studentid, assessmentid)
So Finally, after the changes, it should be something like this:
def mark_assignment(database, studentid, assessmentid, mark):
statement = "UPDATE assignments SET mark = {}, marked = 1 WHERE studentid = {} AND assessmentid = {}".format(mark, studentid, assessmentid)
access_database(database, statement)
Note:
In real-life situations, using str.format() is a bad idea as it can be misused using SQL Injection. But since we only know about this single function and not the entire code, formatting can help here. Adding placeholders like ? are the better options in those cases. Reference: https://realpython.com/prevent-python-sql-injection/
I want to query a sqlite database using a f-string for the query, it is supposed to return a small description given a specific id that i stored in a local variable var. However some fields will be empty for some values of the variable. I'm specially interested in the value 0 has it will always return an empty description and is an isolated case that needs to be addressed differently from other values.
I want to iterate through a range until I find a not empty description field.
Here is an example:
var = str(self.some_global_variable) # In this case consider var = '0'
query = f'my sql query goes here WHERE this_id={var}'
description = self.function_to_fetch_db(query)
while not description and var == '0':
for i in range (1, 31):
var = str(i)
description = self.function_to_fetch_db(query)
print(description, var)
print(query)
The output of this will be something like:
[] 1
my sql query goes here WHERE this_id=0
[] 2
my sql query goes here WHERE this_id=0
[] 3
my sql query goes here WHERE this_id=0
.
.
.
The local variable is updated but the query always keeps the original value from outside the while loop.
I also tried an if...else instead of the while loop but the result is the same.
I don't think the SQLite part of the problem is relevant, it's just to illustrate my specific case, the query works for other values. I'm just having trouble to figure out this local variable and f-string relationship.
There are two answers to your questions: the first one is formal, the second one is correct.
The formal answer: the string is computed once before the loop in your case. If you want it to alter it for every value, move it inside:
for i in range(31):
query = f'my sql query goes here WHERE this_id={i}'
description = self.function_to_fetch_db(query)
The correct answer: use parameterized queries instead, that will look like:
conn.execute('my sql query goes here WHERE this_id=?', (i,))
(The substitution syntax may vary depending on your database / driver.)
you could use a named variable in a string:
var = str(self.some_global_variable) # In this case consider var = '0'
fquery = 'my sql query goes here WHERE this_id={var}'
description = self.function_to_fetch_db(fquery.format(var=var))
while not description and var == '0':
for i in range(1, 31):
query = fquery.format(var=i)
description = self.function_to_fetch_db(query)
print(description, i)
print(query)
I want to display a data in QTableWidget according to QComboBoxes. In case of select all gender or select all ages, I want apply select all in the column in sqlite3 query
I want gender to be all
gender = "select all both male and female"
connection.execute("SELECT * FROM child where region=? and hospital=? and ageInMonths=? and gender=?", (region,hospital,ageInMonths,gender))
Welcome to Stackoverflow.
While it's a little tedious, the most sensible way to attack this problem is to build a list of the conditions you want to apply, and another of the data values that need to be inserted. Something like the following (untested) code, in which I assume that the variables are set to None if they aren't required in the search.
conditions = []
values = []
if region is not None:
conditions.append('region=?')
values.append(region)
# And similar logic for each other value ...
if gender is not None:
conditions.append('gender=?')
values.append(gender)
query = 'SELECT * FROM child'
if conditions:
query = query + ' WHERE ' + ' AND '.join(conditions)
connection.execute(query, values)
This way, if you want to include all values of a column you simply exclude if from the conditions by setting it to None.
You can build your where clause and your parameter list conditionally.
Below I am assuming that the ageInMonths and gender variables actually contain the value 'all' when this is selected on your form. You can change this to whichever value is actually passed to your code, if it is something different.
When it comes to your actual query, the best way to get all values for a field is to simply exclude it from the where clause of your query entirely.
So something like:
query_parameters = []
query_string = "SELECT * FROM child where region=? and hospital=?"
query_parameters.append(region)
query_parameters.append(hospital)
if ageInMonths != 'all':
query_string += " and ageInMonths=?"
query_parameters.append(ageInMonths)
if gender != 'all':
query_string += " and gender=?"
query_parameters.append(gender)
connection.execute(query_string, query_parameters)
Basically, at the same time we are testing and building the dynamic parts of the SQL statement (in query_string), we are also dynamically defining the list of variables to pass to the query in query_parameters, which is a list object.
I am new to python (as I am sure you can tell from my code) and have basically written a sample script to test some core commands so I can better understand how things work. I have gotten everything running as designed except the last "insert" command -- after hours of googling and experimenting I cannot figure out what's wrong, so I would appreciate it if anyone can show me what needs to be changed and helping me understand why (I'm sure it's basic, but am stumped!).
Below is the line that is giving me trouble:
c.execute("INSERT OR IGNORE INTO {tn} ({cn1}, {cn2}, {cn3}, {cn4}) VALUES ({VID}, {VSnu}, {VIN}, {VName})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, VID=ID, VSnu=Snu, VIN=IN, VName=Name))
Here's the entire script for context, if that helps:
import sqlite3
sqlite_file = '/test_database.sqlite' # name of the sqlite database file
table_name = 'test_table'
column1 = 'my_1st_column'
column2 = 'my_2nd_column'
column3 = 'my_3rd_column'
column4 = 'my_4th_column'
ID = int(123456)
Base = 'Arnold'
Snu = 'test'
conn = sqlite3.connect(sqlite_file)
c = conn.cursor()
c.execute("UPDATE {tn} SET {cn2}=('Snu'), {cn3}=('Muh'), {cn4}=('Arnold_A') WHERE {cn1}=({NID})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, NID=ID))
i = 1
while(i<15):
if i == 1: IN = 'B'
if i == 2: IN = 'C'
if i == 3: IN = 'D'
if i == 4: IN = 'E'
if i == 5: IN = 'F'
if i == 6: IN = 'G'
if i == 7: IN = 'H'
if i == 8: IN = 'I'
if i == 9: IN = 'J'
ID = ID+1
i = i+1
Name = Base + '_' + IN
params = (Snu, IN, Name)
c.execute("INSERT OR IGNORE INTO {tn} ({cn1}, {cn2}, {cn3}, {cn4}) VALUES ({VID}, {VSnu}, {VIN}, {VName})".\
format(tn=table_name, cn1=column1, cn2=column2, cn3=column3, cn4=column4, VID=ID, VSnu=Snu, VIN=IN, VName=Name))
if(i == 10): break
conn.commit()
conn.close()
It will write the "VID" item just fine if isolated (which is an integer column and the primary key of the table), but everything after that it interprets as a column and I get the "no such column: [value]" error.
I get this error regardless of what comes after "VID" -- be it any variable, as demonstrated with the command above, or if I try to insert a direct string value. These other columns are simply text columns, if that helps.
I know you are just learning, so don't be scared by what I'm going to say next :-)
You are improperly forming SQL commands. There's a special feature known as parameter substitution that is present in many database APIs, including that of SQLite in Python.
You don't want to concatenate values like you are doing, because it opens up room for something called SQL injection. In simple scenarios like yours it may not a problem, but when people do it in services exposed on the internet, it can cause great damage (data theft, data loss, data corruption etc). This comic illustrates the problem: https://xkcd.com/327/
In your case, writing the SQL command with parameter substitution is not only going to make your code safer and easier to read, it will also solve the problem you are seeing.
See the example below:
import sqlite3
conn = sqlite3.connect(":memory:")
c = conn.cursor()
# Here we don't need parameter substitution
c.execute("CREATE TABLE mytable (name text, quantity int)")
# We set the values somewhere else. For example, this could come from a web
# form
entry = ("myname", 2)
# Now we execute the query, with the proper parameter substitution.
# Note the `?` characters. Python will automatically replace those with the
# values in `entry`, making sure that things are right when forming the
# command before passing to SQLite.
c.execute("INSERT OR IGNORE INTO mytable VALUES (?, ?)", entry)
print(c.execute("SELECT * from mytable").fetchall())
The example above assumes you don't need to parametrize the column names. In your case, you are apparently reading all columns, so you don't really need to pass all the names, just do like I did in the example above and it will read all columns. If you really need column names to be parameters to read a subset of data, then you'll have to resort to a concatenation mechanism similar to what you were using. But in that case, as always, be really careful with user input to make sure it's not forming bad queries and commands.
More details here: https://docs.python.org/3/library/sqlite3.html