Develop Reference

How can I know if my SQL query not expose to SQL injection [duplicate]

This question already has answers here:
How can I know if my website is vulnerable to SQL Injection?
(3 answers)
Closed 8 months ago.
I have an originally SQL query:
f"SELECT FIELDS(ALL) from xxxx WHERE CreatedDate >= {start_time}"
I wanted to make that query safe from sql injection attack but I could not see how can I know that I did it right.
This is the new version that should be safe:
f"SELECT FIELDS(ALL) from xxxx WHERE CreatedDate >= %s" % (start_time,)
I'm using it in an API call. The query itself will be excecated in the other side (third party). I want to send the query as parameter in the api call
I would like to get some tips regarding this issue
Thank you!

Anytime you are directly creating the string in your code you are exposing yourself to SQL injection. You want to pass the handling of data off to the DBMS. Using an ORM like SQLAlchemy will handle a lot of that (if you use the ORM and don't pass your SQL in directly). Most libraries for connecting to a database follow python's DB api standard. Since you haven't mentioned what you're using I'll use pyodbc as an example.
Copied from the docs:
Inserting Data
To insert data, pass the insert SQL to Cursor execute(), along with any parameters > necessary:
cursor.execute("insert into products(id, name) values ('pyodbc', 'awesome library')")
cnxn.commit()
or, parameterized:
cursor.execute("insert into products(id, name) values (?, ?)", 'pyodbc', 'awesome library')
cnxn.commit()
Notice the parameterized version. This is what you want. Here pyodbc is handing both your query and your data to the DBMS. The DBMS will handle sanitizing the data. This form is called qmark notation (notice the question marks). There are a few other notations but the important part is you are using parameterization and passing the data as separate from your query. With most libraries this looks something like:
cursor.execute(query_string_with_qmark_notation, data_or_tuple_of_data)

Get executemany to accept Unicode strings

How can one get executemany to format a prepared SQL to write unicode data?
I have a prepared query that is similar to this:
insert into foobar select(select baz.somestring from baz where baz.whatever = %s ), %s
Changing the %s directly into something like N'%s' in the prepared query won't work, how do i get pymssql to encase my inputs with N'mystring' instead of a simple 'mystring'?
The current behaviour results in questionmarks in the table when something like a \u5000 comes around, for example, while it should save the \u5000 or any other unicode character.

how do i get pymssql to encase my inputs with N'mystring' instead of a simple 'mystring'?
pymssql will do that if you create a proper parameterized query:
print(sys.version) # 3.7.2 ...
print(pymssql.__full_version__) # 2.1.4
#
sql = "SELECT COUNT(*) AS n FROM #tmp WHERE whatever = %s"
params = ('Ώπα',)
crsr.execute(sql, params)
print(crsr.fetchone())
SQL Profiler shows that the query is sent as
SELECT COUNT(*) AS n FROM #tmp WHERE whatever = N'Ώπα'

Error "not all arguments converted during string formatting" while inserting in Database MySQL

I have an SQL query which is giving an error:
cur.execute("INSERT INTO `DB` (`ban`, `dntr`, `usrnm`, `id`, `dis`) VALUES (1,0,?,?,?)",(param1,param2,param3,))
I don't want to use %s in query because it is prone to SQL injection and I am taking input from users.

mysqlclient uses %s as the placeholder (see the example in the docs).
Change your code to the following:
cur.execute("INSERT INTO `DB` (`ban`, `dntr`, `usrnm`, `id`, `dis`) VALUES (1,0,%s,%s,%s)", (param1,param2,param3,))
You're right to be concerned about SQL injection, but the above is OK. You are still using execute with parameters, so they will be escaped.
The thing you shouldn't do is cur.execute(query % parameters, []). This is vulnerable to SQL injection.

Use of '.format()' vs. '%s' in cursor.execute() for mysql JSON field, with Python mysql.connector,

My objective is to store a JSON object into a MySQL database field of type json, using the mysql.connector library.
import mysql.connector
import json
jsonData = json.dumps(origin_of_jsonData)
cnx = mysql.connector.connect(**config_defined_elsewhere)
cursor = cnx.cursor()
cursor.execute('CREATE DATABASE dataBase')
cnx.database = 'dataBase'
cursor = cnx.cursor()
cursor.execute('CREATE TABLE table (id_field INT NOT NULL, json_data_field JSON NOT NULL, PRIMARY KEY (id_field))')
Now, the code below WORKS just fine, the focus of my question is the use of '%s':
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES (%s, %s)"
values_to_insert = (1, jsonData)
cursor.execute(insert_statement, values_to_insert)
My problem with that: I am very strictly adhering to the use of '...{}'.format(aValue) (or f'...{aValue}') when combining variable aValue(s) into a string, thus avoiding the use of %s (whatever my reasons for that, let's not debate them here - but it is how I would like to keep it wherever possible, hence my question).
In any case, I am simply unable, whichever way I try, to create something that stores the jsonData into the mySql dataBase using something that resembles the above structure and uses '...{}'.format() (in whatever shape or form) instead of %s. For example, I have (among many iterations) tried
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES ({}, {})".format(1, jsonData)
cursor.execute(insert_statement)
but no matter how I turn and twist it, I keep getting the following error:
ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[some_content_from_jsonData})]' at line 1
Now my question(s):
1) Is there a way to avoid the use of %s here that I am missing?
2) If not, why? What is it that makes this impossible? Is it the cursor.execute() function, or is it the fact that it is a JSON object, or is it something completely different? Shouldn't {}.format() be able to do everything that %s could do, and more?

First of all: NEVER DIRECTLY INSERT YOUR DATA INTO YOUR QUERY STRING!
Using %s in a MySQL query string is not the same as using it in a python string.
In python, you just format the string and 'hello %s!' % 'world' becomes 'hello world!'. In SQL, the %s signals parameter insertion. This sends your query and data to the server separately. You are also not bound to this syntax. The python DB-API specification specifies more styles for this: DB-API parameter styles (PEP 249). This has several advantages over inserting your data directly into the query string:
Prevents SQL injection
Say you have a query to authenticate users by password. You would do that with the following query (of course you would normally salt and hash the password, but that is not the topic of this question):
SELECT 1 FROM users WHERE username='foo' AND password='bar'
The naive way to construct this query would be:
"SELECT 1 FROM users WHERE username='{}' AND password='{}'".format(username, password)
However, what would happen if someone inputs ' OR 1=1 as password. The formatted query would then become
SELECT 1 FROM users WHERE username='foo' AND password='' OR 1=1
which will allways return 1. When using parameter insertion:
execute('SELECT 1 FROM users WHERE username=%s AND password=%s', username, password)
this will never happen, as the query will be interpreted by the server separately.
Performance
If you run the same query many times with different data, the performance difference between using a formatted query and parameter insertion can be significant. With parameter insertion, the server only has to compile the query once (as it is the same every time) and execute it with different data, but with string formatting, it will have to compile it over and over again.

In addition to what was said above, I would like to add some details that I did not immediately understand, and that other (newbies like me ;)) may also find helpful:
1) "parameter insertion" is meant for only for values, it will not work for table names, column names, etc. - for those, the Python string substitution works fine in the sql syntax defintion
2) the cursor.execute function requires a tuple to work (as specified here, albeit not immediately clear, at least to me: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html)
EXAMPLE for both in one function:
def checkIfRecordExists(column, table, condition_name, condition_value):
...
sqlSyntax = 'SELECT {} FROM {} WHERE {} = %s'.format(column, table, condition_name)
cursor.execute(sqlSyntax, (condition_value,))
Note both the use of .format in the initial sql syntax definition and the use of (condition_value,) in the execute function.

How to insert strings with quotes and newlines into sqlite db with Python?

I'm trying to insert strings read from a file into an sqlite database in Python. The strings have whitespace (newline, tab characters, and spaces) and also have appearances of single or double quotes. Here's how I try to do it:
import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()
# Create table
c.execute('''CREATE TABLE test
(a text, b text)''')
f = open("foo", "w")
f.write("hello\n\'world\'\n")
f.close()
testfield = open("foo").read()
# Insert a row of data
c.execute("INSERT INTO test VALUES ('%s', 'bar')" %(testfield))
# Save (commit) the changes
conn.commit()
I find that this fails with the error:
c.execute("INSERT INTO test VALUES ('%s', 'bar')" %(testfield))
sqlite3.OperationalError: near "world": syntax error
How can I achieve this? Do the strings need to be escaped before insertion in the db, and if so how? thanks.

You use SQL parameters instead of string formatting:
c.execute("INSERT INTO test VALUES (?, 'bar')", (testfield,))
When using SQL parameters you let the database library handle the quoting, and even better, give the database to optimize the query and reuse the optimized query plan for multiple executions of the same basic query (with different parameters).
Last but not least, you are much better defended against SQL injection attacks as the database library knows best how to escape dangerous SQL-like values.
To quote the sqlite3 documentation:
Usually your SQL operations will need to use values from Python variables. You shouldn’t assemble your query using Python’s string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack (see http://xkcd.com/327/ for humorous example of what can go wrong).
Instead, use the DB-API’s parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method.