Develop Reference

How to solve second order SQL injection attacks vulnerability?

I'm facing a challenge while deploying a backed application built using Python Flask. The codegate scan is catching up some of the code integrated in the CI/CD process as potential vulnerabilities. Below are 2 types of vulnerabilities.
Second order SQL injection.
SQL injection
I'm using raw SQL queries to get the data from the DB in conjunction with Flask SQLAlchemy and Pandas.
Below is a sample code where codegate is pointing the issues for Second order SQL injection.
def get_user_data(user_id: str):
query: str = USER_ID_QUERY.format(user_id=user_id)
result = db.session.execute(query)
result = result.fetchall()
if len(result) < 1:
raise Exception("User not Found")
return result[0][0]
Query
USER_ID_QUERY = """SELECT USER_ID FROM USER WHERE USER_ID = '{user_id}'"""
Vulnerability description - Method get_user_data at line 236 of
src\utils.py gets database data from the execute element. This
element’s value then flows through the code without being properly
sanitized or validated, and is eventually used in a database query in
method check_access at line 50 of src\service.py. This may enable an
Second-Order SQL Injection attack.
I have tried below solution after digging out the internet by still it is giving the same error.
Binding the parameters using text and bindparams of sqlalchemy
query = text(USER_ID_QUERY).bindparams(user_id=user_id)
Could someone please help me in highlighting what is wrong here or what can be done to resolve these painfull issues?

PyMySQL query string builder

So I'm using the PyMySQL package to query a remote database and I am finding it quite annoying to have to use the .execute() method.
I am looking for a wrapper that would be able to construct the raw MySQL queries in a more friendly fashion. Does anyone know of a package that would do this?
I have tried using pypika but the queries that it's building ('SELECT "id","username" FROM "users"') throw an error
pymysql.err.ProgrammingError: (1064, 'You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near \'"users"\' at line 1')
EDIT: this is a remote DB that I have no control over structure-wise and it is likely that the structure could change so I don't think I could use SQLAlchemy
EDIT 2: Here is the code that I used when I got that error above
from pypika import Query, Table, Field
import pymysql
users = Table('users')
q = Query.from_(users).select(users.id, users.username)
db = pymysql.connect(host=host,
port=3306,
user=user,
passwd=password,
db=db,
cursorclass=pymysql.cursors.DictCursor)
db.execute(str(q))

It looks like it's generating a query using double-quotes as identifier delimiters.
But MySQL uses back-ticks as the default identifier delimiter, so the query should look like this:
SELECT `id`,`username` FROM `users`
You can configure MySQL to use double-quotes, which is the proper identifier delimiter to comply with the ANSI SQL standard. To do this, you have to change the sql_mode to include the modes ANSI or ANSI_QUOTES.
Read https://dev.mysql.com/doc/refman/8.0/en/sql-mode.html for full documentation on sql_mode behavior, and https://dev.mysql.com/doc/refman/8.0/en/identifiers.html for documentation on identifier delimiters.
I searched the pypika site and the docs are pretty sparse. But there's apparently a class for MySQLQuery that sets its own QUOTE_CHAR to ` which is what I would expect.
Are you using the proper query builder for MySQL?

You can use peewee a a simple and small ORM.
http://docs.peewee-orm.com/en/latest/peewee/quickstart.html#quickstart

PYODBC Insert Into Database - Error: Optional Feature Not Implemented (0) (SQLBindParameter)

I am currently trying to use pyodbc to select data from a table within Database A and insert it into a table within Database B. I was able to establish connections with both databases, so I know there is no error there. Additionally, my first cursor.execute command (line #9) works as I was able to print all the data.
The issue I am running into is when I try and insert the data from the first cursor.execute command into Database B. There are a few questions on SO regarding this same error, however I have checked to ensure I am not committing on of those errors. All the data types are accepted within SQL Server, I have the correct number of parameters and parameter markers, and I have ensured that the columns within my Python code match both the input and output tables. I am completely stuck and would greatly appreciate any help.
The specific error I am getting is:
('HYC00', '[HYC00] [Microsoft][ODBC SQL Server Driver]Optional feature
not implemented (0) (SQLBindParameter)')
Please see my code below:
import pyodbc
import time
cnxn1 = pyodbc.connect(r"DRIVER={SQL Server Native Client 11.0};SERVER='Server';" + \
"DATABASE='DatabaseA';Trusted_Connection=Yes")
cursor1 = cnxn1.cursor()
cnxn2 = pyodbc.connect(r"DRIVER={SQL Server};SERVER='Server'," + \
"user='Username', password='Password', database='DatabaseB'")
cursor2 = cnxn2.cursor()
SQL = cursor1.execute("select * from table.DatabaseA")
SQL2 = """insert into table.DatabaseB([col1], [col2], [col3], [col4],[col5], [col6], [col7],
[col8], [col9], [col10], [col11], [col12], [col13], [col14],
[col15], [col16],[col17], [col18], [col19], [col20], [col21],
[col22], [col23], [col24], [col25], [col26], [col27], [col28],
[col29], [col30], [col31])
values (?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?,?)"""
for row in cursor1.fetchall():
cursor2.execute(SQL2,row)
In regard to the last two lines of code, I have also tried the following with no success:
for row in SQL:
cursor2.execute(SQL2,row)

pypyodbc: OPENJSON incorrect syntax near keyword "WITH"

I'm trying to use OPENJSON in a Python script to import some basic JSON into a SQL database. I initially tried with a more complex JSON file, but simplified it for the sake of this post. Here's what I have:
sql_statement = "declare #json nvarchar(max) = '{\"name\":\"James\"}'; SELECT * FROM OPENJSON(#json) WITH (name nvarchar(20))"
cursor.execute(sql_statement)
cursor.commit()
connection.close()
The error I receive:
pypyodbc.ProgrammingError: (u'42000', u"[42000] [Microsoft][ODBC SQL
Server Driver][SQL Server]Incorrect syntax near the keyword 'with'. If
this statement is a common table expression, an xmlnamespaces clause
or a change tracking context clause, the previous statement must be
terminated with a semicolon.")
Any thoughts on why I'm seeing this error? I was successfully able to execute other SQL queries with the same pypyodbc / database configuration.

The problem could be that your database is running in an older compatibility level, where OPEN JSON is not available.
To find the compatibility level of your database, run following SQL statement:
SELECT compatibility_level FROM sys.databases WHERE name = 'your_db_name';
If the result is 120 or lower, you'll need to update your compatibility level to 130, by running:
ALTER DATABASE your_db_name SET COMPATIBILITY_LEVEL = 130;
Note: In case your database is actually Azure SQL DB, you should check the version as well, as OPEN JSON is not available for versions prior to 12.x

pyodbc & MS SQL Server - "No results. Previous SQL was not a query."

I am using pyodbc to retrieve data from a Microsoft SQL Server. The query is of the following form
SET NOCOUNT ON --Ignore count statements
CREATE TABLE mytable ( ... )
EXEC some_stored_procedure
INSERT mytable
--Perform some processing...
SELECT *
FROM mytable
The stored procedure performs some aggregation over values that contain NULLs such that warnings of the form Warning: Null value is eliminated by an aggregate or other SET operation. are issued. This results in pyodbc failing to retrieve data with the error message No results. Previous SQL was not a query.
I have tried to disable the warnings by setting SET ANSI_WARNINGS OFF. However, the query then fails with the error message Heterogeneous queries require the ANSI_NULLS and ANSI_WARNINGS options to be set for the connection. This ensures consistent query semantics. Enable these options and then reissue your query..
Is it possible to
disable the warnings
or have pyodbc ignore the warnings?
Note that I do not have permissions to change the stored procedure.

Store the results of the query in a temporary table and execute the statement as two queries:
with pyodbc.connect(connection_string) as connection:
connection.execute(query1) #Do the work
result = connection.execute(query2) #Select the data
data = result.fetchall() #Retrieve the data
The first query does the heavy lifting and is of the form
--Do some work and execute complicated queries that issue warning messages
--Store the results in a temporary table
SELECT some, column, names
INTO #datastore
FROM some_table
The second query retrieves the data and is of the form
SELECT * FROM #datastore
Thus, all warning messages are issued upon execution of the first query. They do not interfere with data retrieval during the execution of the second query.

I have had some luck mitigating this error by flipping ansi_warnings on and off just around the offending view or stored proc.
/* vw_someView aggregates away some nulls and presents warnings that blow up pyodbc */
set ANSI_WARNINGS off
select *
into #my_temp
from vw_someView
set ANSI_WARNINGS on
/* rest of query follows */
This assumes that the entity that produces the aggregate warning doesn't also require warnings to be turned on. If it complains, it probably means that the entity itself has a portion of code like this that requires a toggle of the ansi_warnings (or a rewrite to eliminate the aggregation.)
One caviat is that I've found that this toggle still returns the "heterogeneous" warning if I try to run it as a cross-server query. Also, while debugging, it's pretty easy to get into a state where the ansi_warnings are flipped off when you don't realize it and you start getting heterogeneous errors for seemingly no reason. Just run the "set ANSI_WARNINGS on" line by itself to get yourself back into a good state.

Best thing is to add try: except: block
sql="sp_help stored_procedure;"
print(">>>>>executing {}".format(sql))
next_cursor=cursor.execute(sql)
while next_cursor:
try:
row = cursor.fetchone()
while row:
print(row)
row = cursor.fetchone()
except Exception as my_ex:
print("stored procedure returning non-row {}".format(my_ex))
next_cursor=cursor.nextset()