I am trying to create a program where a user can enter an operator i.e. <> or = and then a number for a database in pymysql. I have tried a number of different ways of doing this but unfortunately unsuccessful. I have two documents with display being one and importing display into the other document.
Docuemnt 1
def get_pop(op, pop):
if (not conn):
connect();
query = "SELECT * FROM city WHERE Population %s %s"
with conn:
cursor = conn.cursor()
cursor.execute(query, (op, pop))
x = cursor.fetchall()
return x
Document two
def city():
op = input("Enter < > or =: ")
population = input("Enter population: ")
pop = display.get_pop(op, population)
for p in pop:
print(pop)
I am getting the following error.
pymysql.err.ProgrammingError: (1064,......
Please help thanks
You can't do this. Parameterization works for values only, not operators or table names, or column names. You'll need to format the operator into the string. Do not confuse the %s placeholder here with Python string formatting; MySQL is awkward in that it uses %s for binding parameters, which clashes with regular Python string formatting.
The MySQL %s in a query string escapes the user input to protect against SQL Injection. In this case, I set up a basic test to see if the operation part submitted by the user was in a list of accepted operations.
def get_pop(op, pop):
query = "SELECT * FROM city WHERE Population {} %s" # Add a placeholder for format
with conn: # Where does this come from?
cursor = conn.cursor()
if op in ['=', '!=']:
cursor.execute(query.format(op), (pop,))
x = cursor.fetchall()
return x
You'll want to come up with some reasonable return value in the case that if op in ['=', '!='] is not True but that depends entirely on how you want this to behave.
After checking that op indeed contains either "<>" or "=" and that pop indeed contains a number you could try:
query = "SELECT * FROM city WHERE Population " + op + " %s";
Beware of SQL injection.
Then
cursor.execute(query, (pop))
Related
This question already has answers here:
How to use variables in SQL statement in Python?
(5 answers)
Closed 1 year ago.
nameEnt = nameEntered.get()
print(nameEnt)
sql = "SELECT * FROM attendance WHERE name="%s"
val = (nameEnt)
print(mycursor.execute(sql, val))
myresult = mycursor.fetchall()
for x in myresult:
print(x)
I would like to pass the string in 'nameEnt' into the SQL query using python. I'm currently using the mysql-connector package. The program keeps telling me that my syntax is incorrect. I can execute the query in SQL directly without any problem.
I have also tried
sql = "SELECT * FROM attendance WHERE name= "+nameENt
Do not combine SQL strings with data unless you know what you are doing. Doing so is a sure way to get yourself an SQL injection vulnerability.
Your original code was almost correct. First, as the comments noted, you don't need the quotes around %s:
sql = "SELECT * FROM attendance WHERE name=%s"
Then, your second parameter to cursor.execute() is a tuple, but in Python, to make a single-element tuple, wrapping it in brackets isn't enough:
my_element = 12345
not_a_tuple = (my_element)
type(not_a_tuple) == int
real_tuple = (my_element,) # note the comma at the end
type(real_tuple) == tuple
Applying these to your code, you get:
nameEnt = nameEntered.get()
print(nameEnt)
sql = "SELECT * FROM attendance WHERE name=%s"
val = (nameEnt,)
print(mycursor.execute(sql, val))
myresult = mycursor.fetchall()
for x in myresult:
print(x)
you can try:
sql = "SELECT * FROM attendance WHERE name = %s",(nameEnt)
Or:
sql = "SELECT * FROM attendance WHERE name = {}".format(nameEnt)
I have the following code, using pscyopg2:
sql = 'select %s from %s where utctime > %s and utctime < %s order by utctime asc;'
data = (dataItems, voyage, dateRangeLower, dateRangeUpper)
rows = cur.mogrify(sql, data)
This outputs:
select 'waterTemp, airTemp, utctime' from 'ss2012_t02' where utctime > '2012-05-03T17:01:35+00:00'::timestamptz and utctime < '2012-05-01T17:01:35+00:00'::timestamptz order by utctime asc;
When I execute this, it falls over - this is understandable, as the quotes around the table name are illegal.
Is there a way to legally pass the table name as a parameter, or do I need to do a (explicitly warned against) string concatenation, ie:
voyage = 'ss2012_t02'
sql = 'select %s from ' + voyage + ' where utctime > %s and utctime < %s order by utctime asc;'
Cheers for any insights.
According to the official documentation:
If you need to generate dynamically an SQL query (for instance
choosing dynamically a table name) you can use the facilities
provided by the psycopg2.sql module.
The sql module is new in psycopg2 version 2.7. It has the following syntax:
from psycopg2 import sql
cur.execute(
sql.SQL("insert into {table} values (%s, %s)")
.format(table=sql.Identifier('my_table')),
[10, 20])
More on: https://www.psycopg.org/docs/sql.html#module-usage
[Update 2017-03-24: AsIs should NOT be used to represent table or fields names, the new sql module should be used instead: https://stackoverflow.com/a/42980069/5285608 ]
Also, according to psycopg2 documentation:
Warning: Never, never, NEVER use Python string concatenation (+) or string parameters interpolation (%) to pass variables to a SQL query string. Not even at gunpoint.
Per this answer you can do it as so:
import psycopg2
from psycopg2.extensions import AsIs
#Create your connection and cursor...
cursor.execute("SELECT * FROM %(table)s", {"table": AsIs("my_awesome_table")})
The table name cannot be passed as a parameter, but everything else can. Thus, the table name should be hard coded in your app (Don't take inputs or use anything outside of the program as a name). The code you have should work for this.
On the slight chance that you have a legitimate reason to take an outside table name, make sure that you don't allow the user to directly input it. Perhaps an index could be passed to select a table, or the table name could be looked up in some other way. You are right to be wary of doing this, however. This works, because there are relatively few table names around. Find a way to validate the table name, and you should be fine.
It would be possible to do something like this, to see if the table name exists. This is a parameterised version. Just make sure that you do this and verify the output prior to running the SQL code. Part of the idea for this comes from this answer.
SELECT 1 FROM information_schema.tables WHERE table_schema = 'public' and table_name=%s LIMIT 1
This is a workaround I have used in the past
query = "INSERT INTO %s (col_1, col_2) VALUES (%%s, %%s)" % table_name
cur.execute(query, (col_1_var, col_2_var))
Hope it help :)
This is a small addition to #Antoine Dusséaux's answer. If you want to pass two (unquoted) parameters in a SQL query, you can do it as follows: -
query = sql.SQL("select {field} from {table} where {pkey} = %s").format(
field=sql.Identifier('my_name'),
table=sql.Identifier('some_table'),
pkey=sql.Identifier('id'))
As per the documentation,
Usually you should express the template of your query as an SQL
instance with {}-style placeholders and use format() to merge the
variable parts into them, all of which must be Composable subclasses.
You can still have %s-style placeholders in your query and pass values
to execute(): such value placeholders will be untouched by format()
Source: https://www.psycopg.org/docs/sql.html#module-usage
Also, please keep this in mind while writing queries.
I have created a little utility for preprocessing of SQL statements with variable table (...) names:
from string import letters
NAMECHARS = frozenset(set(letters).union('.'))
def replace_names(sql, **kwargs):
"""
Preprocess an SQL statement: securely replace table ... names
before handing the result over to the database adapter,
which will take care of the values.
There will be no quoting of names, because this would make them
case sensitive; instead it is ensured that no dangerous chars
are contained.
>>> replace_names('SELECT * FROM %(table)s WHERE val=%(val)s;',
... table='fozzie')
'SELECT * FROM fozzie WHERE val=%(val)s;'
"""
for v in kwargs.values():
check_name(v)
dic = SmartDict(kwargs)
return sql % dic
def check_name(tablename):
"""
Check the given name for being syntactically valid,
and usable without quoting
"""
if not isinstance(tablename, basestring):
raise TypeError('%r is not a string' % (tablename,))
invalid = set(tablename).difference(NAMECHARS)
if invalid:
raise ValueError('Invalid chars: %s' % (tuple(invalid),))
for s in tablename.split('.'):
if not s:
raise ValueError('Empty segment in %r' % tablename)
class SmartDict(dict):
def __getitem__(self, key):
try:
return dict.__getitem__(self, key)
except KeyError:
check_name(key)
return key.join(('%(', ')s'))
The SmartDict object returns %(key)s for every unknown key, preserving them for the value handling. The function could check for the absence of any quote characters, since all quoting now should be taken care of ...
If you want to pass the table name as a parameter, you can use this wrapper:
class Literal(str):
def __conform__(self, quote):
return self
#classmethod
def mro(cls):
return (object, )
def getquoted(self):
return str(self)
Usage: cursor.execute("CREATE TABLE %s ...", (Literal(name), ))
You can just use the module format for the table name and then use the regular paramaterization for the execute:
xlist = (column, table)
sql = 'select {0} from {1} where utctime > %s and utctime < %s order by utctime asc;'.format(xlist)
Keep in mind if this is exposed to the end user, you will not be protected from SQL injection unless you write for it.
Surprised no one has mentioned doing this:
sql = 'select {} from {} where utctime > {} and utctime < {} order by utctime asc;'.format(dataItems, voyage, dateRangeLower, dateRangeUpper)
rows = cur.mogrify(sql)
format puts in the string without quotations.
Whenever I try the code below, I get near "?": syntax error
I tried multiple things including prepping it into a variable
Is this possible in python? Or am I thinking in the wrong direction?
import sqlite3
sqlite_file = 'DATABASE.db'
conn = sqlite3.connect(sqlite_file)
c = conn.cursor()
word = ()
question = int(input("What would you like to see?"))
if question == 1:
word = ("Batchnumbers")
if question == 2:
word = ("Worker IDs")
c.execute('SELECT * FROM ?', (word))
data = c.fetchall()
for x in range(len(data)):
print(data[x])
Query parameters can only be used to supply column values, not table or column names. If you need to supply a table name you will have to use dynamic SQL, e.g.,
c.execute('SELECT * FROM "{}"'.format(word))
Note that this approach is vulnerable to SQL injection issues, so you really should consider mitigating those, e.g., ensuring that word does not contain double-quote characters that would cause errors (or worse).
Indeed use this line of code
word =
c.execute('SELECT * FROM "{}"'.format(word))
I am trying to query a mysql db from python but having troubles generating the query ebcasue of the wildcard % and python's %s. As a solution I find using ?, but when I run the following,
query = '''select * from db where name like'Al%' and date = '%s' ''', myDateString
I get an error
cursor.execute(s %'2015_05_21')
ValueError: unsupported format character ''' (0x27) at index 36 (the position of %)
How can i combine python 2.7 string bulding and sql wildcards? (The actual query is a lot longer and involves more variables)
First of all, you need to escape the percent sign near the Al:
'''select * from db where name like 'Al%%' and date = '%s''''
Also, follow the best practices and pass the query parameters in the second argument to execute(). This way your query parameters would be escaped and you would avoid sql injections:
query = """select * from db where name like 'Al%%' and date = %s"""
cursor.execute(query, ('2015_05_21', ))
Two things:
Don't use string formatting ('%s' % some_var) in SQL queries. Instead, pass the string as a sequence (like a list or a tuple) to the execute method.
You can escape your % so Python will not expect a format specifier:
q = 'SELECT foo FROM bar WHERE zoo LIKE 'abc%%' and id = %s'
cursor.execute(q, (some_var,))
Use the format syntax for Python string building, and %s for SQL interpolation. That way they don't conflict with each other.
You are not using the ? correctly.
Here's an example:
command = '''SELECT M.name, M.year
FROM Movie M, Person P, Director D
WHERE M.id = D.movie_id
AND P.id = D.director_id
AND P.name = ?
AND M.year BETWEEN ? AND ?;'''
*Execute the command, replacing the placeholders with the values of
the variables in the list [dirName, start, end]. *
cursor.execute(command, [dirName, start, end])
So, you want to try:
cursor.execute(query,'2015_05_21')
I am trying to get the numbers of rows returned from an sqlite3 database in python but it seems the feature isn't available:
Think of php mysqli_num_rows() in mysql
Although I devised a means but it is a awkward: assuming a class execute sql and give me the results:
# Query Execution returning a result
data = sql.sqlExec("select * from user")
# run another query for number of row checking, not very good workaround
dataCopy = sql.sqlExec("select * from user")
# Try to cast dataCopy to list and get the length, I did this because i notice as soon
# as I perform any action of the data, data becomes null
# This is not too good as someone else can perform another transaction on the database
# In the nick of time
if len(list(dataCopy)) :
for m in data :
print("Name = {}, Password = {}".format(m["username"], m["password"]));
else :
print("Query return nothing")
Is there a function or property that can do this without stress.
Normally, cursor.rowcount would give you the number of results of a query.
However, for SQLite, that property is often set to -1 due to the nature of how SQLite produces results. Short of a COUNT() query first you often won't know the number of results returned.
This is because SQLite produces rows as it finds them in the database, and won't itself know how many rows are produced until the end of the database is reached.
From the documentation of cursor.rowcount:
Although the Cursor class of the sqlite3 module implements this attribute, the database engine’s own support for the determination of “rows affected”/”rows selected” is quirky.
For executemany() statements, the number of modifications are summed up into rowcount.
As required by the Python DB API Spec, the rowcount attribute “is -1 in case no executeXX() has been performed on the cursor or the rowcount of the last operation is not determinable by the interface”. This includes SELECT statements because we cannot determine the number of rows a query produced until all rows were fetched.
Emphasis mine.
For your specific query, you can add a sub-select to add a column:
data = sql.sqlExec("select (select count() from user) as count, * from user")
This is not all that efficient for large tables, however.
If all you need is one row, use cursor.fetchone() instead:
cursor.execute('SELECT * FROM user WHERE userid=?', (userid,))
row = cursor.fetchone()
if row is None:
raise ValueError('No such user found')
result = "Name = {}, Password = {}".format(row["username"], row["password"])
import sqlite3
conn = sqlite3.connect(path/to/db)
cursor = conn.cursor()
cursor.execute("select * from user")
results = cursor.fetchall()
print len(results)
len(results) is just what you want
Use following:
dataCopy = sql.sqlExec("select count(*) from user")
values = dataCopy.fetchone()
print values[0]
When you just want an estimate beforehand, then simple use COUNT():
n_estimate = cursor.execute("SELECT COUNT() FROM user").fetchone()[0]
To get the exact number before fetching, use a locked "Read transaction", during which the table won't be changed from outside, like this:
cursor.execute("BEGIN") # start transaction
n = cursor.execute("SELECT COUNT() FROM user").fetchone()[0]
# if n > big: be_prepared()
allrows=cursor.execute("SELECT * FROM user").fetchall()
cursor.connection.commit() # end transaction
assert n == len(allrows)
Note: A normal SELECT also locks - but just until it itself is completely fetched or the cursor closes or commit() / END or other actions implicitely end the transaction ...
I've found the select statement with count() to be slow on a very large DB. Moreover, using fetch all() can be very memory-intensive.
Unless you explicitly design your database so that it does not have a rowid, you can always try a quick solution
cur.execute("SELECT max(rowid) from Table")
n = cur.fetchone()[0]
This will tell you how many rows your database has.
I did it like
cursor.execute("select count(*) from my_table")
results = cursor.fetchone()
print(results[0])
this code worked for me:
import sqlite3
con = sqlite3.connect(your_db_file)
cursor = con.cursor()
result = cursor.execute("select count(*) from your_table").fetchall() #returns array of tupples
num_of_rows = result[0][0]
A simple alternative approach here is to use fetchall to pull a column into a python list, then count the length of the list. I don't know if this is pythonic or especially efficient but it seems to work:
rowlist = []
c.execute("SELECT {rowid} from {whichTable}".\
format (rowid = "rowid", whichTable = whichTable))
rowlist = c.fetchall ()
rowlistcount = len(rowlist)
print (rowlistcount)
The following script works:
def say():
global s #make s global decleration
vt = sqlite3.connect('kur_kel.db') #connecting db.file
bilgi = vt.cursor()
bilgi.execute(' select count (*) from kuke ') #execute sql command
say_01=bilgi.fetchone() #catch one query from executed sql
print (say_01[0]) #catch a tuple first item
s=say_01[0] # assign variable to sql query result
bilgi.close() #close query
vt.close() #close db file