How to pass variable parameters dynamically
order = 10100
status = 'Shipped'
df1 = pd.read_sql_query("SELECT * from orders where orderNumber =""" +
str(10100) + """ and status = """ + 'status' +""" order by orderNumber """,cnx)
TypeError: must be str, not int
getting above error although i converted to strings any idea?
is there any alternative wy to pass the parameters?
Use parametrized sql by supplying the arguments via the params keyword argument. The proper quotation of arguments will be done for you by the database adapter and the code will be less vulnerable to SQL injection attacks. (See Little Bobby Tables for an example of the kind of trouble improperly quoted, non-parametrized sql can get you into.)
order = 10100
status = 'Shipped'
sql = """SELECT * from orders where orderNumber = ?
and status = ? order by orderNumber"""
df1 = pd.read_sql_query(sql, cnx, params=[order, status])
The ?s in sql are parameter markers. They get replaced with properly quoted values from params. Note that the proper parameter marker depends on the database adapter you are using. For example, MySQLdb and psycopg2 uses %s, while sqlite3, and oursql uses ?.
IN MYSQL ::
order = 10100
status = 'Shipped'
sql = """SELECT * from orders where orderNumber = %s
and status = %s order by orderNumber"""
df1 = pd.read_sql_query(sql, cnx, params=[order, status])
Related
I'm aware that the best way to prevent sql injection is to write Python queries of this form (or similar):
query = 'SELECT %s %s from TABLE'
fields = ['ID', 'NAME']
cur.execute(query, fields)
The above will work for a single query, but what if we want to do a UNION of 2 SQL commands? I've set this up via sqlite3 for sake of repeatability, though technically I'm using pymysql. Looks as follows:
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT, VALUE varchar(255))'
query2 = 'CREATE TABLE DUMMy2(ID int AUTO INCREMENT, VALUE varchar(255)'
try:
cur.execute(query)
cur.execute(query2)
except:
print('Already made table!')
tnames = ['DUMMY1', 'DUMMY2']
sqlcmds = []
for i in range(0,2):
query = 'SELECT %s FROM {}'.format(tnames[i])
sqlcmds.append(query)
fields = ['VALUE', 'VALUE']
sqlcmd = ' UNION '.join(sqlcmds)
cur.execute(sqlcmd, valid_fields)
When I run this, I get a sqlite Operational Error:
sqlite3.OperationalError: near "%": syntax error
I've validated the query prints as expected with this output:
INSERT INTO DUMMY VALUES(%s) UNION INSERT INTO DUMMY VALUES(%s)
All looks good there. What is the issue with the string substitutions here? I can confirm that running a query with direct string substitution works fine. I've tried it with both selects and inserts.
EDIT: I'm aware there are multiple ways to do this with executemany and a few other. I need to do this with UNION for the purposes I'm using this for because this is a very, very simplified example fo the operational code I'm using
The code below executes few INSERTS at once
import sqlite3
conn = sqlite3.connect('dummy.db')
cur = conn.cursor()
query = 'CREATE TABLE DUMMY(ID int AUTO INCREMENT NOT NULL, VALUE varchar(255))'
try:
cur.execute(query)
except:
print('Already made table!')
valid_fields = [('ya dummy',), ('stupid test example',)]
cur.executemany('INSERT INTO DUMMY (VALUE) VALUES (?)',valid_fields)
Ultimately I'm trying to use pandas read_sql which takes in a raw SQL query.
I'm trying to convert something like;
sql = str(session.query(Post).filter(Post.user_id=1))
That generates something like
select * from Post where user_id = %(user_id_1)
Is there any way to generate that query with the parameter already interpolated?
As you have found, if we str() an ORM query we get the SQL command text with parameter placeholders using the paramstyle for our dialect:
qry = session.query(Parent).filter(Parent.id == 1)
sql = str(qry)
print(sql)
"""console output:
SELECT parent.id AS parent_id, parent.lastname AS parent_lastname, parent.firstname AS parent_firstname
FROM parent
WHERE parent.id = %(id_1)s
"""
If we want to have the parameter values embedded in the SQL statement then we need to .compile() it:
sql_literal = qry.statement.compile(
compile_kwargs={"literal_binds": True},
)
print(sql_literal)
"""console output:
SELECT parent.id, parent.lastname, parent.firstname
FROM parent
WHERE parent.id = 1
"""
(Standard disclaimers regarding SQL Injection apply.)
I am making a website in django where I want the user to put in a table id and group id and then return the table and group that the put in. However, I have only found statements that are prone to SQL injection. Does anybody know how to fix this?
mycursor = mydb.cursor()
qry = "SELECT * from %s WHERE group_id = %i;" % (assembly_name, group_id)
mycursor.execute(qry)
return mycursor.fetchall()
Or do something that achieves the same thing?
I have tried doing something like this:
assembly_id = 'peptides_proteins_000005'
group_id = 5
mycursor = mydb.cursor()
mycursor.execute("SELECT * FROM %s WHERE group_id = %s", [assembly_id, group_id])
myresult = mycursor.fetchall()
but I get this error:
1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''peptides_proteins_000005' WHERE group_id = 5' at line 1
It's typically not possible to bind table names. For SELECT statements, the easiest way is to sanitize table name candidates by whitelisting.
Check whether the overhead of using abstraction or some way of constraining user input to the finite set of valid names as part of the user interface may be justified.
I am trying to update some values into a database. The user can give the row that should be changed. The input from the user, however is a string. When I try to parse this into the MySQL connector with python it gives an error because of the apostrophes. The code I have so far is:
import mysql.connector
conn = mysql.connector
conn = connector.connect(user=dbUser, password=dbPasswd, host=dbHost, database=dbName)
cursor = conn.cursor()
cursor.execute("""UPDATE Search SET %s = %s WHERE searchID = %s""", ('maxPrice', 300, 10,))
I get this error
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''maxPrice' = 300 WHERE searchID = 10' at line 1
How do I get rid of the apostrophes? Because I think they are causing problems.
As noted, you can't prepare it using a field.
Perhaps the safest way is to allow only those fields that are expected, e.g.
#!/usr/bin/python
import os
import mysql.connector
conn = mysql.connector.connect(user=os.environ.get('USER'),
host='localhost',
database='sandbox',
unix_socket='/var/run/mysqld/mysqld.sock')
cur = conn.cursor(dictionary=True)
query = """SELECT column_name
FROM information_schema.columns
WHERE table_schema = DATABASE()
AND table_name = 'Search'
"""
cur.execute(query)
fields = [x['column_name'] for x in cur.fetchall()]
user_input = ['maxPrice', 300, 10]
if user_input[0] in fields:
cur.execute("""UPDATE Search SET {0} = {1} WHERE id = {1}""".format(user_input[0], '%s'),
tuple(user_input[1:]))
print cur.statement
Prints:
UPDATE Search SET maxPrice = 300 WHERE id = 10
Where:
mysql> show create table Search\G
*************************** 1. row ***************************
Search
CREATE TABLE `Search` (
`id` int(11) DEFAULT NULL,
`maxPrice` float DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1
A column name is not a parameter. Put the column name maxPrice directly into your SQL.
cursor.execute("""UPDATE Search SET maxPrice = %s WHERE searchID = %s""", (300, 10))
If you want to use the same code with different column names, you would have to modify the string itself.
sql = "UPDATE Search SET {} = %s WHERE searchID = %s".format('maxPrice')
cursor.execute(sql, (300,10))
But bear in mind that this is not safe from injection the way parameters are, so make sure your column name is not a user-input string or anything like that.
You cannot do it like that. You need to place the column name in the string before you call cursor.execute. Column names cannot be used when transforming variables in cursor.execute.
Something like this would work:
sql = "UPDATE Search SET {} = %s WHERE searchID = %s".format('maxPrice')
cursor.execute(sql, (300, 10,))
You cannot dynamically bind object (e.g., column) names, only values. If that's the logic you're trying to achieve, you'd have to resort to string manipulation/formatting (with all the risks of SQL-injection attacks that come with it). E.g.:
sql = """UPDATE Search SET {} = %s WHERE searchID = %s""".format('maxPrice')
cursor.execute(sql, (300, 10,))
When I try to pass a tuple to the IN argument of a WHERE clause, it gets double-quoted so my query fails. For example, if I do this,
# Connect to DB
import MySQLdb
cnxn = MySQLdb.connect(connectString)
curs = cnxn.cursor()
# Setup query
accounts = ('Hyvaco','TLC')
truck_type = 'fullsize'
query_args = (truck_type, accounts)
sql ='SELECT * FROM archive.incoming WHERE LastCapacity=%s AND Account IN %s'
# Run query and print
curs.execute(sql, query_args)
print(curs._executed)
then I get zero rows back, and the query prints out as
SELECT * FROM archive.incoming WHERE LastCapacity='fullsize'
AND Account IN ("'Hyvaco'", "'TLC'")
Switching accounts from a tuple to a list does not affect the result. How should I be passing these arguments?
How about you create the accounts as a string and then do this:
accounts = "('Hyvaco','TLC')"
sql ='SELECT * FROM archive.incoming WHERE LastCapacity=%s AND Account IN '+ accounts