I noticed a lot of pythonic replace functions being used prior to using psycopg2 to insert data into a database. I understand that it is done to prevent the user from creating malformed sql queries.
Example:
insert into users (name) values ('Hello''World');
Which will store: Hello'World in the column.
My thought process is that they are only really doing this before inserting and I feel uneasy about it.
Example:
s = "Hello'World"
q = "insert into users (name) values ('{}');".format(s.replace("'", "''"))
cursor.execute(q)
It feels off to me. Part of me is thinking that I can still mess with s in such a way to get around this replace as it is the only thing stopping injection. Yes, the arent using psycopg2 correctly because the second param is a list or dict of values correlating to the database entry.
It should say:
cursor.execute("insert into users (name) values (%s);", ["Hello'World"])
So i am trying to find a Proof of concept to do sql injection here as I dont think just double ticks isnt good enough.
Since this was python I am thinking of options, but i dont know how postgres works, if there is some sort of preprocessor used in strings etc.
Is there a way to pass a string which will get processed as s ' without replace touching it? I was looking into things like attempting to pass in unicode or something. See if I can do something with unicode or chr to get it to process differently?
Has anyone else encountered this? I am sitting in dbeaver as well as psycopg2/python to see if i can get a working sample working.
Related
I need to move data from one database to another.
I can use python my counterpart can't.
How can select all data from a table and save it as insert statements.
Using SQLalchemy.
Is there a way to create a back up like this?
As others have suggested in comments, using the database backup program (mysqldump, pg_dump, etc) is your best bet; that will make sure that the data is transferred correctly for the underlying database.
Outputting INSERT statements will be risky; even the built-in SQLAlchemy facility for doing this comes with a big red warning, complete with a picture of a dragon, indicating that it can be dangerous.
If you nevertheless need to do this, and the data is generally trusted and doesn't contain much in the way of odd types, you can use:
Create (but do not execute) an insert expression as though you were inserting the rows back into the database.
Use the .compile() method with the relevant dialect parameter and literal_binds set to True.
Manually double-check that the output is, in fact, valid for the database; as per the warning in the SQLAlchemy FAQ, this method is not very dependable and may expose you to attacks if it's part of any production system.
I wouldn't recommend formatting up INSERT statements by hand; you're unlikely to do a better job than SQLAlchemy...
I am building a little interface where I would like users to be able to write out their entire sql statement and then see the data that is returned. However, I don't want a user to be able to do anything funny ie delete from user_table;. Actually, the only thing I would like users to be able to do is to run select statements. I know there aren't specific users for SQLite, so I am thinking what I am going to have to do, is have a set of rules that reject certain queries. Maybe a regex string or something (regex scares me a little bit). Any ideas on how to accomplish this?
def input_is_safe(input):
input = input.lower()
if "select" not in input:
return False
#more stuff
return True
I can suggest a different approach to your problem. You can restrict the access to your database as read-only. That way even when the users try to execute delete/update queries they will not be able to damage your data.
Here is the answer for Python on how to open a read-only connection:
db = sqlite3.connect('file:/path/to/database?mode=ro', uri=True)
Python's sqlite3 execute() method will only execute a single SQL statement, so if you ensure that all statements start with the SELECT keyword, you are reasonably protected from dumb stuff like SELECT 1; DROP TABLE USERS. But you should check sqlite's SQL syntax to ensure there is no way to embed a data definition or data modification statement as a subquery.
My personal opinion is that if "regex scares you a little bit", you might as well just put your computer in a box and mail it off to <stereotypical country of hackers>. Letting untrusted users write SQL code is playing with fire, and you need to know what you're doing or you'll get fried.
Open the database as read only, to prevent any changes.
Many statements, such as PRAGMA or ATTACH, can be dangerous. Use an authorizer callback (C docs) to allow only SELECTs.
Queries can run for a long time, or generate a large amount of data. Use a progress handler to abort queries that run for too long.
I am new to using SQLite with python and we have been code in which there is this statement
c.execute('INSERT INTO users VALUES (?,?)', user)
I am not sure what the question marks (?,?) mean, I have tried reading the documentation on sqlite3 website but was not able to get anywhere. Would be a great help if someone can tell me or direct me to the right link.
Thank you
They are placeholders for literal values that can be bound to a prepared SQL statement. Essentially it allows you to supply literal values in the SQL program without putting them into the SQL string. This both prevents SQL injection attacks and improves performance if you're running the same query with different parameter values - the SQL has to be compiled only once.
Documentation (C API)
As the title suggests, I would like to know if this code is vulnerable to SQL Injection? And if so, is there a better, more secure, way of achieving the same thing?
def add(table,*args):
statement="INSERT INTO %s VALUES %s" % (table,args)
cursor.execute(statement)
Yes, it is. Use something like this to prevent it:
cursor.execute("INSERT INTO table VALUES ?", args)
Note that you cannot enter the table in like this. Ideally the table should be hard coded, in no circumstance should it come from a user input of any kind. You can use a string similar to what you did for the table, but you'd better make 100% certain that a user can't change it somehow... See Can I use parameters for the table name in sqlite3? for more details.
Essentially, you want to put the parameters in the cursor command, because it will make sure to make the data database safe. With your first command, it would be relatively easy to make a special table or args that put something into your SQL code that wasn't safe. See the python pages, and the referenced http://xkcd.com/327/ . Specifically, the python pages quote:
Usually your SQL operations will need to use values from Python
variables. You shouldn’t assemble your query using Python’s string
operations because doing so is insecure; it makes your program
vulnerable to an SQL injection attack (see http://xkcd.com/327/ for
humorous example of what can go wrong).
Instead, use the DB-API’s parameter substitution. Put ? as a
placeholder wherever you want to use a value, and then provide a tuple
of values as the second argument to the cursor’s execute() method.
(Other database modules may use a different placeholder, such as %s or
:1.)
Basically, someone could set an args that executed another command, something like this:
args="name; DELETE table"
Using cursor.execute will stuff the value given, so that the argument could be as listed, and when you do a query on it, that is exactly what you will get out. XKCD explains this humorously as well.
I'm in the processing of moving over a mysql database to a postgres database. I have read all of the articles presented here, as well as reading over some of the solutions presented on stackoverflow. The tools recommended don't seem to work for me. Both databases were generated by Django's syncdb, although the postgres db is more or less empty at the moment. I tried to migrate the tables over using Django's built in dumpdata / loaddata functions and its serializers, but it doesn't seem to like a lot of my tables, leading me to believe that writing a manual solution might be best in this case. I have code to verify that the column headers are the same for each table in the database and that the matching tables exist- that works fine. I was thinking it would be best to just grab the mysql data row by row and then insert it into the respective postgres table row by row (I'm not concerned with speed atm). The one thing is, I don't know what's the proper way to construct the insert statement. I have something like:
table_name = retrieve_table()
column_headers = get_headers(table_name) #can return a tuple or a list
postgres_cursor = postgres_con.cursor()
rows = mysql_cursor.fetchall()
for row in rows: #row is a tuple
postgres_cursor.execute(????)
Where ??? would be the insert statement. I just don't know what the proper way is to construct it. I have the table name that I would like to insert into as a string, I have the column headers that I can treat as a list, tuple, or string, and I have the respective values that I'd like to insert. What would be the recommended way to construct the statement? I have read the documentation on psycopg's documentation page and I didn't quite see the way that would satisfy my needs. I don't know (or think) this is the entirely correct way to properly migrate, so if someone could steer me in the correct way or offer any advice I'd really appreciate it.