I'm calling extremely simple query from Python program using pymsqsql library.
with self.conn.cursor() as cursor:
cursor.execute('select extra_id from mytable where id = %d', id)
extra_id = cursor.fetchone()[0]
Note that parameter binding is used as described in pymssql documentation.
One of the main goals of parameter binding is providing ability for DBMS engine to cache the query plan. I connected to MS SQL with Profiler and checked what queries are actually executed. It turned out that each time a unique statement gets executed (with its own bound ID). I also checked query usage with such query:
select * from sys.dm_exec_cached_plans ec
cross apply
sys.dm_exec_sql_text(ec.plan_handle) txt
where txt.text like '%select extra_id from mytable where id%'
And it shown that the plan is not reused (which is expectable of course due to unique text of each query). This differs much from parameter binding when querying from C#, when we can see that queries are the same but the supplied parameters are different.
So I wonder if I am using pymssql correctly and whether this lib is appropriate for using with MS SQL DBMS.
P.S. I know that MS SQL has a feature of auto-parameterization which works for basic queries, but it is not guarantied, and may not work for complex queries.
You are using pymssql correctly. It is true that pymssql actually does substitute the parameter values into the SQL text before sending the query to the server. For example:
pymssql:
SELECT * FROM tablename WHERE id=1
pyodbc with Microsoft's ODBC Driver for SQL Server (not the FreeTDS ODBC driver):
exec sp_prepexec #p1 output,N'#P1 int',N'SELECT * FROM tablename WHERE id=#P1',1
However, bear in mind that pymssql is based on FreeTDS and the above behaviour appears to be a function of the way FreeTDS handles parameterized queries, rather than a specific feature of pymssql per se.
And yes, it can have implications for the re-use of execution plans (and hence performance) as illustrated in this answer.
Related
Using Sybase as the main transactional DB and SQLite as the in memory DB for Integration tests.
Issue I am facing is conflicting behavior of the two implementations.
I need to execute a query similar to
select dbo.get_name(id), id from some_table
This runs perfectly fine in sybase (I understand the importance of schema prefix for user defined functions). However SQL lite throws error saying SQLite.Operational error near "("
Tried to add dbo as schema while creating SQLite connections but no luck.
Using Python for all the implementation.
You could make dbo a group, and make all your users member of that group. Then you could avoid using the schema prefix at all.
Or, you could have a SQLite database named dbo where you put the function get_name.
I searched the web and the Stack Overflow site in particular, and I couldn't find any simple explanation as to the role a cursor plays in PyMySQL. Why is it required? what function does it fulfill? Can I have multiple cursors? Can I pass it as an argument to a class or a function?
Looking at tutorials with examples I wrote code that uses cursors and does work. But so far the use of cursors is counter intuitive to me without really understanding their role and function.
Please help...
The cursor in MySQL is used in most cases to retrieve rows from your resultset and then perform operations on that data. The cursor enables you to iterate over returned rows from an SQL query.
Here is an example.
1) First we declare a cursor:
DECLARE cursor_name CURSOR FOR SELECT_statement;
2) Let's open the cursor.
OPEN cursor_name;
3) Now we can use the FETCH statement to retrieve the next row in the result set.
(Recall the syntax for the FETCH statement: FETCH [ NEXT [ FROM ] ] cursor_name INTO variable_list;. As you can see, cursor is within the syntax, so it is a vital part of the FETCH statement).
FETCH cursor_name INTO variable_list;
4) Summary: Okay, so we have used our cursor_name to FETCH the next row, and we store that in variable_list (a list of variables, comma-separated, where the cursor result should be stored).
This should illustrate the following:
FETCH use MySQL cursors to fetch the next row in a resultset.
The cursor is a tool to iterate over your rows in a resultset, one row at a time.
The pymysql cursor
PyMySQL is used to "interact" with the database. However, take a look at PEP 249 which defines the Python Database API Specification.
PyMySQL is based on the PEP 249 specification, so the cursor is derived from the PEP 249 specification.
And in PEP 249 we see this:
https://www.python.org/dev/peps/pep-0249/#cursor-objects
"Cursor Objects
These objects represent a database cursor, which is used to manage the context of a fetch operation. Cursors created from the same connection are not isolated, i.e., any changes done to the database by a cursor are immediately visible by the other cursors. Cursors created from different connections can or can not be isolated, depending on how the transaction support is implemented (see also the connection's .rollback() and .commit() methods)."
Sorry for ask here but I cannot found much reference about pymysql's security guide about how do we prevent sql injection,
When I do PHP develope I know use mysql preparedstatement(or called Parameterized Query or stmt),but I cannot found reference about this in pymysql
simple code use pymysql like
sqls="select id from tables where name=%s"
attack="jason' and 1=1"
cursor.execute(sqls,attack)
How do I know this will prevent sql injection attack or not?if prevent succeed,how do pymysql prevent?Is cursor.execute already use preparedstatement by default?
Python drivers do not use real query parameters. In python, the argument (the variable attack in your example) is interpolated into the SQL string before sending the SQL to the database server.
This is not the same as using a query parameter. In a real parameterized query, the SQL string is sent to the database server with the parameter placeholder intact.
But the Python driver does properly escape the argument as it interpolates, which protects against SQL injection.
I can prove it when I turn on the query log:
mysql> SET GLOBAL general_log=ON;
And tail the log while I run the Python script:
$ tail -f /usr/local/var/mysql/bkarwin.log
...
180802 8:50:47 14 Connect root#localhost on test
14 Query SET ##session.autocommit = OFF
14 Query select id from tables where name='jason\' and 1=1'
14 Quit
You can see that the query has had the value interpolated into it, and the embedded quote character is preceded by a backslash, which prevents it from becoming an SQL injection vector.
I'm actually testing MySQL's Connector/Python, but pymysql does the same thing.
I disagree with this design decision for the Python connectors to avoid using real query parameters (i.e. real parameters work by sending the SQL query to the database with parameter placeholders, and sending the values for those parameters separately). The risk is that programmers will think that any string interpolation of parameters into the query string will work the same as it does when you let the driver do it.
Example of SQL injection vulnerability:
attack="jason' and '1'='1"
sqls="select id from tables where name='%s'" % attack
cursor.execute(sqls)
The log shows this has resulted in SQL injection:
180802 8:59:30 16 Connect root#localhost on test
16 Query SET ##session.autocommit = OFF
16 Query select id from tables where name='jason' and '1'='1'
16 Quit
I'm trying to understand what this code is doing behind the scenes:
import psycopg2
c = psycopg2.connect('db=some_db user=me').cursor()
c.execute('select * from some_table')
for row in c:
pass
Per PEP 249 my understanding was that this was repeatedly calling Cursor.next() which is the equivalent of calling Cursor.fetchone(). However, the psycopg2 docs say the following:
When a database query is executed, the Psycopg cursor usually fetches
all the records returned by the backend, transferring them to the
client process.
So I'm confused -- when I run the code above, is it storing the results on the server and fetching them one by one, or is it bringing over everything at once?
It depends on how you configure psycopg2. See itersize and server side cursors.
By default it fetches all rows into client memory, then just iterates over the fetched rows with the cursor. But per the above docs, you can configure batch fetches from a server-side cursor instead.
I have the following python code:
row = conn.execute('''SELECT admin FROM account WHERE password = ?''',
(request.headers.get('X-Admin-Pass'),)).fetchone()
My question is whether this code is secure for SQL injection? Since I use parameterized query it should be. However, since I am passing user information straight from the header, I am a little worried :)
Any thoughts about the issue?
The way that you are inserting the data into the database will ensure that an SQL attack will not work, the execute method will automatically escape the parameters that you passed as a tuple as its second parameter to the query.
You are doing that correctly.
If your module uses the DBI specs, then you're parameterizing fine. Unless you want to do research into preventing specific SQL attacks, paramterizing your queries is a good umbrella against SQL injection.