Develop Reference

Sql Alchemy Insert Statement failing to insert, but no error

I am attempting to execute a raw sql insert statement in Sqlalchemy, SQL Alchemy throws no errors when the constructed insert statement is executed but the lines do not appear in the database.
As far as I can tell, it isn't a syntax error (see no 2), it isn't an engine error as the ORM can execute an equivalent write properly (see no 1), it's finding the table it's supposed to write too (see no 3). I think it's a problem with a transaction not being commited and have attempted to address this (see no 4) but this hasn't solved the issue. Is it possible to create a nested transaction and what would start the 'first' so to speak?
Thankyou for any answers.
Some background:
I know that the ORM facilitates this and have used this feature and it works, but is too slow for our application. We decided to try using raw sql for this particular write function due to how often it's called and the ORM for everything else. An equivalent method using the ORM works perfectly, and the same engine is used for both, so it can't be an engine problem right?
I've issued an example of the SQL that the method using raw sql constructs to the database directly and that reads in fine, so I don't think it's a syntax error.
it's communicating with the database properly and can find the table as any syntax errors with table and column names throw a programmatic error so it's not just throwing stuff into the 'void' so to speak.
My first thought after reading around was that it was transaction error and that a transaction was being created and not closed, and so constructed the execute statement as such to ensure a transaction was properly created and commited.
with self.Engine.connect() as connection:
connection.execute(Insert_Statement)
connection.commit
The so called 'Insert Statement' has been converted to text using the sqlalchemy 'text' function, I don't quite understand why it won't execute if I pass the constructed string directly to the execute statement but mention it in case it's relevant.
Other things that may be relevant:
Python3 is running on an individual ec2 instance the postgres database on another. The table in particular is a timescaledb hypertable taking realtime data, hence the need for very fast writes, but probably not relevant.
Currently using pg8000 as dialect for no particular reason other than psycopg2 was throwing errors when trying the execute an equivalent method using the ORM.

Just so this question is answered in case anyone else ends up here:
The issue was a failure to call commit as a method, as #snakecharmerb pointed out. Gord Thompson also provided an alternate method using 'begin' which automatically commits rather than connection which is a 'commit as you go' style transaction.

Alternatives for Double Apostrophe for Postgresql queries?

I noticed a lot of pythonic replace functions being used prior to using psycopg2 to insert data into a database. I understand that it is done to prevent the user from creating malformed sql queries.
Example:
insert into users (name) values ('Hello''World');
Which will store: Hello'World in the column.
My thought process is that they are only really doing this before inserting and I feel uneasy about it.
Example:
s = "Hello'World"
q = "insert into users (name) values ('{}');".format(s.replace("'", "''"))
cursor.execute(q)
It feels off to me. Part of me is thinking that I can still mess with s in such a way to get around this replace as it is the only thing stopping injection. Yes, the arent using psycopg2 correctly because the second param is a list or dict of values correlating to the database entry.
It should say:
cursor.execute("insert into users (name) values (%s);", ["Hello'World"])
So i am trying to find a Proof of concept to do sql injection here as I dont think just double ticks isnt good enough.
Since this was python I am thinking of options, but i dont know how postgres works, if there is some sort of preprocessor used in strings etc.
Is there a way to pass a string which will get processed as s ' without replace touching it? I was looking into things like attempting to pass in unicode or something. See if I can do something with unicode or chr to get it to process differently?
Has anyone else encountered this? I am sitting in dbeaver as well as psycopg2/python to see if i can get a working sample working.

Is this Python code vulnerable to SQL injection? (SQLite3)

As the title suggests, I would like to know if this code is vulnerable to SQL Injection? And if so, is there a better, more secure, way of achieving the same thing?
def add(table,*args):
statement="INSERT INTO %s VALUES %s" % (table,args)
cursor.execute(statement)

Yes, it is. Use something like this to prevent it:
cursor.execute("INSERT INTO table VALUES ?", args)
Note that you cannot enter the table in like this. Ideally the table should be hard coded, in no circumstance should it come from a user input of any kind. You can use a string similar to what you did for the table, but you'd better make 100% certain that a user can't change it somehow... See Can I use parameters for the table name in sqlite3? for more details.
Essentially, you want to put the parameters in the cursor command, because it will make sure to make the data database safe. With your first command, it would be relatively easy to make a special table or args that put something into your SQL code that wasn't safe. See the python pages, and the referenced http://xkcd.com/327/ . Specifically, the python pages quote:
Usually your SQL operations will need to use values from Python
variables. You shouldn’t assemble your query using Python’s string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack (see http://xkcd.com/327/ for
humorous example of what can go wrong).
Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor’s execute() method.
(Other database modules may use a different placeholder, such as %s or
:1.)
Basically, someone could set an args that executed another command, something like this:
args="name; DELETE table"
Using cursor.execute will stuff the value given, so that the argument could be as listed, and when you do a query on it, that is exactly what you will get out. XKCD explains this humorously as well.

SQL Injection Prevention in Python - is using parameterized query enough?

I have the following python code:
row = conn.execute('''SELECT admin FROM account WHERE password = ?''',
(request.headers.get('X-Admin-Pass'),)).fetchone()
My question is whether this code is secure for SQL injection? Since I use parameterized query it should be. However, since I am passing user information straight from the header, I am a little worried :)
Any thoughts about the issue?

The way that you are inserting the data into the database will ensure that an SQL attack will not work, the execute method will automatically escape the parameters that you passed as a tuple as its second parameter to the query.
You are doing that correctly.

If your module uses the DBI specs, then you're parameterizing fine. Unless you want to do research into preventing specific SQL attacks, paramterizing your queries is a good umbrella against SQL injection.

Django getting executable raw sql for a QuerySet

I know that you can get the SQL of a given QuerySet using
print query.query
but as we know from a previous question ( Potential Django Bug In QuerySet.query? ) the returned SQL is not properly quoted. See http://code.djangoproject.com/browser/django/trunk/django/db/models/sql/query.py
Is there any way that is it possible to get the raw, executable SQL (quoted) for a given QuerySet without actually executing it?

Django never creates the raw sql, so no. To prevent SQL injection, django passes the parameters separately to the database drivers at the last step. The best way to get the actual SQL is to look at your query log, which you cannot do before you execute the query.