I've build a simple Python API which receives JSON dictionaries with strings and numeric values from the clients.
In order to protect my system and avoid MySQL-Injections I escape the strings when I'm using it f.ex.:
jsonInput = request.get_json(force=True, silent=True) # INPUT FROM HTTP POST REQUEST
# ESCAPING STRING BY STRING
flowerColor = MySQLdb.escape_string(jsonInput['flowerColor'])
flowerSize = MySQLdb.escape_string(jsonInput['flowerSize'])
query = "SELECT * FROM flowers WHERE color='" + flowerColor + "' AND size='" + flowerSize + "'"
try:
conn = // CONNECT TO DB
cursor = conn.cursor()
cursor.execute(query)
return cursor.fetchall()
except MySQLdb.IntegrityError:
return False
In my opinion it would be way easier, if there is a method to just escape the whole input at once, so every string in the JSON can be used directly after the escape method in a MySQL query. For example:
jsonInput = request.get_json(force=True, silent=True) # INPUT FROM HTTP POST REQUEST
# ESCAPING WHOLE JSON AT ONCE
escapedJSON = escape_json_at_once(jsonInput)
query = "SELECT * FROM flowers WHERE color='" + escapedJSON['flowerColor'] + "' AND size='" + escapedJSON['flowerSize'] + "'"
try:
// EXECUTE THE QUERY
except MySQLdb.IntegrityError:
return False
// DO OTHER QUERIES WITH ESCAPED JSON DATA
I already found the method MySQLdb.escape_dict() but couldn't get it to work also with reading the docs.
Does somebody know a way to escape the whole input at once?
yes, there is indeed a better way
jsonInput = request.get_json(force=True, silent=True) # INPUT FROM HTTP POST REQUEST
query = "SELECT * FROM flowers WHERE color=%s AND size=%s"
try:
conn = // CONNECT TO DB
cursor = conn.cursor()
cursor.execute(query, ( jsonInput['flowerColor'],jsonInput['flowerSize']) )
return cursor.fetchall()
except MySQLdb.IntegrityError:
return False
With this approach you don't need to escape anything at all and it's shorter and neater.
Related
As it is said in asyncpg Usage, I can use $n pattern for the arguments and execute a query this way, for example:
result = await conn.fetchval("SELECT $1", 42)
In this case the raw SQL would be SELECT 42. How do I get this raw text with an asyncpg function before execution? I am asking this is because I want to log queries in my project before they are applied.
query_tpl = "SELECT $1"
values = (42,)
sql = what_is_here(query_tpl, *values) # <- ???
print(sql) # Must be "SELECT 42"
result = await conn.fetchval(query_tpl, *values)
I'm learning programming with python and trying to implement the safest possible MySQL queries starting with the simple SELECT ones. The problem is whenever I use coma in a query I got the following error:
cursor.execute(query)
File "C:\Users\username\AppData\Local\Programs\Python\Python37-32\lib\site-packages\mysql\connector\cursor.py", line 536, in execute
stmt = operation.encode(self._connection.python_charset)
AttributeError: 'tuple' object has no attribute 'encode'
I am aware of the fact that coma itself isn't a source of a problem but I tried many different MySQL syntax and everytime I use a come I got this "AttributeError: 'tuple' object has no attribute 'encode'" error.
I also tried to change MySQL database encoding - nothing changes. The code is below.
import mysql.connector
conn = mysql.connector.connect(
charset='utf8',
# init_command='SET NAMES UTF8',
host="10.0.0.234",
user="x",
passwd="x>",
database="x",
)
print(conn.is_connected())
param = "test"
cursor = conn.cursor()
# =========== query below does work ========
# query = ("SELECT * from list WHERE username LIKE '%test%'")
# ============ query below does work =======
# query = ("SELECT * from list HAVING username = '%s'" % param)
# ============ query below doesn't work =====
# query = ("SELECT * from list HAVING username = %s", (param,))
# ============= query below doesn't work =====
query = "SELECT * from list WHERE username = :name", {'name': param}
cursor.execute(query)
result = cursor.fetchall()
for x in result:
print(x)
conn.close()
Any ideas what am I doing wrong?
The answer is a little bit tricky, but it is in essence because of what the actual value of the 'query' variable is...
For example:
# 1.
query = ("SELECT * from list WHERE username LIKE '%test%'")
# when you do this, query is a string variable,
# NB: the parentheses are not necessary here
# so when you call
cursor.execute(query)
# the value passed into the execute call is the string "SELECT * from list WHERE username LIKE '%test%'"
# 2.
query = ("SELECT * from list HAVING username = '%s'" % param)
# when you do this, query is the result of a string formatting operation
# This is a Python 2 form of string formatting
# The discussion here probably makes it more clear:
# https://stackoverflow.com/questions/13945749/string-formatting-in-python-3
# it is almost the same as doing this:
query = "SELECT * from list HAVING username = 'test'"
# so when you call
cursor.execute(query)
# the value passed into the execute call is the string "SELECT * from list HAVING username = 'test'"
# 3.
query = ("SELECT * from list HAVING username = %s", (param,))
# This operation is assigning a 2-value tuple into the query variable
# The first value in the tuple is the string "SELECT * from list HAVING username = %s"
# The second value in the tuple is a 1-value, with 'test' as its first value
# 4.
query = "SELECT * from list WHERE username = :name", {'name': param}
# This is similar to #3, but the values in the tuple are instead
# query[0] == "SELECT * from list WHERE username = :name"
# query[1] is a dictionary: {'name': param}
Both 3 and 4 above are not calling the MySQL execute with the parameters you are expecting (see API here). You probably need to do one of:
unpack the query tuple into separate variables, and call the function with them
operation, params = query # unpack the first elem into operation, and second into params
cursor.execute(operation, params)
just index into the query tuple
cursor.execute(query[0], query[1])
# NB: you could also use the 'named parameters' feature in Python
cursor.execute(query[0], params=query[1])
Use the 'unpacking arguments list' (SPLAT operator)
cursor.execute(*query)
I have a problem with my python code which I want to use for a REST API server.
The current problem is that my database query is returning null when I know that the value is there
The code for the specific path:
#app.route('/data/active_predicted/<int:ticketId>', methods=['GET'])
def search_db_tickId_act(ticketId):
cursor = db.cursor()
db_query = cursor.execute("select * from active_predicted where ticketId=" + str(ticketId))
json_output = json.dumps(dict(cursor.fetchall()))
cursor.close()
if not cursor.fetchall():
return "Nothing found \n SQL Query: " + "select * from active_predicted where ticketId=" + str(ticketId)
else:
return str(cursor.fetchall())
When I access this URL I get returned the following:
Nothing found SQL Query: select * from active_predicted where ticketId=1324
When I plug this SQL query I get the result I want, 1 row with 2 columns but it seems as though the program cannot locate the row?
The problems:
As #pvg mentioned, you need to escape your input values when querying database;
If you want to fetch a dictionary-like result, passing dictionary=True when you initialize the cursor;
In your original code, you didn't return the variable json_output;
To fetch only one result, use fetchone instead fetchall;
After cursor.close() got called, you can obtain nothing from that cursor no matter you fetched before or not;
Use try-finally to ensure that cursor always get closed (at last).
Here's the fixed code:
#app.route('/data/active_predicted/<int:ticketId>', methods=['GET'])
def search_db_tickId_act(ticketId):
try:
cursor = db.cursor(dictionary=True)
db_query = cursor.execute("select * from active_predicted where ticketId=%s LIMIT 1", ticketId)
row = cursor.fetchone()
if row:
return json.dumps(row)
else:
return "Nothing found \n SQL Query: " + "select * from active_predicted where ticketId=" + str(ticketId)
finally:
cursor.close()
This question already has an answer here:
How do I escape % from python mysql query
(1 answer)
Closed 6 years ago.
I use PyMySQL to query from a MySQL database in python:
filter = "Pe"
connection = pymysql.connect(host="X", user="X", password="X", db="X", port=3306, cursorclass=pymysql.cursors.SSCursor)
cursor = connection.cursor()
sqlquery = "SELECT * FROM usertable WHERE name LIKE '%%s%'"
cursor.execute(sql, (filter))
response = cursor.fetchall()
connection.close()
This returns nothing.
I could write:
sqlquery = "SELECT * FROM usertable WHERE name LIKE '%" + filter +"%'"
and execute: cursor.execute(sql), but then I lose the escaping, which makes the program vulnerable for injection attacks, right?
Is there way I could insert the value into the LIKE without losing the escape?
...WHERE name LIKE '%%%s%%'" does not work. I think %s adds ' on both sides of the replaced escaped string as a part of its function within PyMySQL.
You need to pass the whole pattern as a query parameter, and use a tuple:
filter = "%Pe%"
sql = "SELECT * FROM usertable WHERE name LIKE %s"
cursor.execute(sql, (filter,))
Double the % you want to keep.
sqlquery = "SELECT * FROM usertable WHERE name LIKE '%%%s%%'"
I know that this question has been asked in the past, but thorough searching hasn't seemed to fix my issue. I'm probably just missing something simple, as I'm new to the Python-mysql connector supplied by mysql.
I have a Python script which accesses a mysql database, but I'm having issues with removing quotes from my query. Here is my code:
import mysql.connector
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
query = "select * from tags where %s = %s"
a = 'tag_id'
b = '0'
cursor.execute(query, (a, b))
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
My database is set up properly (as I'll prove shortly).
My output for this program is:
MySQLCursor: select * from tags where 'tag_id' = '0'
[]
Yet when I change my query to not use variables, for example:
cursor.execute("select * from tags where tag_id = 0")
Then my output becomes:
MySQLCursor: select * from tags where tag_id = 0
[(0, u'192.168.1.110')]
To me, this means that the only difference between my Cursor queries are the quotes.
How do I remove them from the query?
Thanks in advance.
I personally believe this code is correct and safe, but you should be extremely skeptical of using code like this without carefully reviewing it yourself or (better yet) with the help of a security expert. I am not qualified to be such an expert.
Two important things I changed:
I changed b = '0' to b = 0 so it ends up as a number rather than a quoted string. (This part was an easy fix.)
I skipped the built-in parameterization for the column name and replaced it with my own slight modification to the escaping/quoting built in to mysql-connector. This is the scary part that should give you pause.
Full code below, but again, be careful with this if the column name is user input!
import mysql.connector
def escape_column_name(name):
# This is meant to mostly do the same thing as the _process_params method
# of mysql.connector.MySQLCursor, but instead of the final quoting step,
# we escape any previously existing backticks and quote with backticks.
converter = mysql.connector.conversion.MySQLConverter()
return "`" + converter.escape(converter.to_mysql(name)).replace('`', '``') + "`"
try:
db = mysql.connector.connect(user='root', password='somePassword', host='127.0.0.1', database='dbName')
cursor = db.cursor()
a = 'tag_id'
b = 0
cursor.execute(
'select * from tags where {} = %s'.format(escape_column_name(a)),
(b,)
)
print cursor
data = cursor.fetchall()
print data
except mysql.connector.Error as err:
print "Exception tripped..."
print "--------------------------------------"
print err
cursor.close()
db.close()
I encountered a similar problem using pymysql and have shown my working code here, hope this will help.
What I did is overwrite the escape method in class 'pymysql.connections.Connection', which obviously adds "'" arround your string.
better have shown my code:
from pymysql.connections import Connection, converters
class MyConnect(Connection):
def escape(self, obj, mapping=None):
"""Escape whatever value you pass to it.
Non-standard, for internal use; do not use this in your applications.
"""
if isinstance(obj, str):
return self.escape_string(obj) # by default, it is :return "'" + self.escape_string(obj) + "'"
if isinstance(obj, (bytes, bytearray)):
ret = self._quote_bytes(obj)
if self._binary_prefix:
ret = "_binary" + ret
return ret
return converters.escape_item(obj, self.charset, mapping=mapping)
config = {'host':'', 'user':'', ...}
conn = MyConnect(**config)
cur = conn.cursor()