Develop Reference

sql data formatting and sql injections

I have a database with 2 tables: students, employees and I want to update one of those tables:
import sqlite3
db_file = "school.db"
def update_address(identifier, user_address, user_id):
with sqlite3.connect(db_file) as conn:
c = conn.cursor()
c.execute(f"""
UPDATE {identifier}
SET address = ?
WHERE id = ?;
""",
(user_address, user_id))
update_address("students", "204 Sycamore Street", 2)
The above code works, the problem is I know that using python string formatting in an sql operation can lead to vulnerabilities per sqlite3 docs:
Usually your SQL operations will need to use values from Python variables. You shouldn’t assemble your query using Python’s string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack (see https://xkcd.com/327/ for humorous example of what can go wrong).
Instead, use the DB-API’s parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method.
The placeholder '?' works when it comes to inserting values but not for sql identifiers. Output:
sqlite3.OperationalError: near "?": syntax error
So the question here is: can an sql injection occur if I use python string formatting on an sql identifier or does it only occur on values ?
If it also occurs on identifiers is there a way to format the string in a safe manner?

Yes, if you interpolate any content into an SQL query unsafely, it is an SQL injection vulnerability. It doesn't matter if the content is supposed to be used as a value in the SQL expression, or an identifier, SQL keyword, or anything else.
It's pretty common to format queries from fragments of SQL expressions, if you want to write a query with a variable set of conditions. These are also possible SQL injection risks.
The way to mitigate the SQL injection risk is: don't interpolate untrusted input into your SQL query.
For identifiers, you should make sure the content matches a legitimate name of a table (or column, or other element, if that's what you're trying to make dynamic). I.e. create an "allowlist" of tables known to exist in your database that are permitted to update using your function. If the input doesn't match one of these, then don't run the query.
It's also a good idea to use back-ticks to delimit identifiers, because if one of the table names happens to be a reserved keyword in SQLite, that will allow the table to be used in the SQL query.
if identifier not in ["table1", "table2", "table3"]:
raise Exception("Unknown table name: '{identifier}'")
c.execute(f"""
UPDATE `{identifier}`
SET address = ?
WHERE id = ?;
""",
(user_address, user_id))

SQLite - How to use VARIABLE for TABLE NAME + VARIABLE FOR INSERT VALUES

im new in programing, starting with python
I found this script in a post (link below)
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
How to use variable for SQLite table name
it worked, but now i have to add some complexity and dont know how...
i had this befour:
def add(order, theme):
parameters = [order, theme]
c.execute("INSERT INTO TABLE_NAME VALUES(NULL, ?,?)", parameters)
so im trying have the same, but with de name of the table free to choose.
trying things like this, but dont really know the syntax for it:
def add(order, theme):
table = input("with what table do you want to work with? ")
parameters = [order, theme]
insert = 'INSERT INTO {} VALUES(NULL, ?,?)'.format(table, parameters)
c.execute(insert)
i suspected it woudnt work Hope you can help! :)

The following line substitutes {} with the contents of the variable table and stores the result in the variable query. format works with any string:
query = 'SELECT * FROM {}'.format(table)
On the other hand the kind of substitution done by c.execute replaces the ? in the string with the values in the list parameters:
c.execute("INSERT INTO TABLE_NAME VALUES(NULL, ?,?)", parameters)
You could combine them both to achieve the effect you want:
table = input("with what table do you want to work with? ")
query = 'INSERT INTO {} VALUES(NULL, ?, ?)'.format(table)
parameters = [order, theme]
c.execute(query, parameters)

You are mixing things.
Format is a method of Python string object used to generate dynamically your string based on variables from your code. In your first example is correct to define the name of the database.
Questions marks as a placeholder for the values of your query is proper of SQL and must be used without format as you did in the second example.
To fix your code, first build the string template of the query using format to define the databasename. Then, execute the resultant string with execute and passing the arguments targeting the question marks.

Pyodbc - loading data inside a table causing error

I am trying to load the data inside the table trial and it says Invalid Column name - Name.
I am passing values inside Name and Area dynamically.
cursor.execute("insert into trial (NameofTheProperty, AreaofTheProperty)
values (Name, Area)")
cnxn.commit()

You need to have quotes around the column values so that they are not gonna be interpreted as column names instead:
insert into
trial (NameofTheProperty, AreaofTheProperty)
values
("Name", "Area")
Now, since you mentioned that you dynamically insert these values into the query, you can just let your database driver handle the quotes and other things like type conversions:
property_name = "Name"
property_area = "Area"
cursor.execute("""
insert into
trial (NameofTheProperty, AreaofTheProperty)
values
(?, ?)""", (property_name, property_area))
cnxn.commit()
This is called query parameterization and is considered the safest and the most robust way to insert values into the SQL queries. These ? values are called "placeholders".
Note that the database driver is gonna put quotes around the string values automatically - no need to do it manually.

How to insert strings with quotes and newlines into sqlite db with Python?

I'm trying to insert strings read from a file into an sqlite database in Python. The strings have whitespace (newline, tab characters, and spaces) and also have appearances of single or double quotes. Here's how I try to do it:
import sqlite3
conn = sqlite3.connect('example.db')
c = conn.cursor()
# Create table
c.execute('''CREATE TABLE test
(a text, b text)''')
f = open("foo", "w")
f.write("hello\n\'world\'\n")
f.close()
testfield = open("foo").read()
# Insert a row of data
c.execute("INSERT INTO test VALUES ('%s', 'bar')" %(testfield))
# Save (commit) the changes
conn.commit()
I find that this fails with the error:
c.execute("INSERT INTO test VALUES ('%s', 'bar')" %(testfield))
sqlite3.OperationalError: near "world": syntax error
How can I achieve this? Do the strings need to be escaped before insertion in the db, and if so how? thanks.

You use SQL parameters instead of string formatting:
c.execute("INSERT INTO test VALUES (?, 'bar')", (testfield,))
When using SQL parameters you let the database library handle the quoting, and even better, give the database to optimize the query and reuse the optimized query plan for multiple executions of the same basic query (with different parameters).
Last but not least, you are much better defended against SQL injection attacks as the database library knows best how to escape dangerous SQL-like values.
To quote the sqlite3 documentation:
Usually your SQL operations will need to use values from Python variables. You shouldn’t assemble your query using Python’s string operations because doing so is insecure; it makes your program vulnerable to an SQL injection attack (see http://xkcd.com/327/ for humorous example of what can go wrong).
Instead, use the DB-API’s parameter substitution. Put ? as a placeholder wherever you want to use a value, and then provide a tuple of values as the second argument to the cursor’s execute() method.

python, mysql, inserting string into table, error 1054

I am having the problem
OperationalError: (1054, "Unknown column 'Ellie' in 'field list'")
With the code below, I'm trying to insert data from json into a my sql database. The problem happens whenever I try to insert a string in this case "Ellie" This is something do to with string interpolation I think but I cant get it to work despite trying some other solutions I have seen here..
CREATE TABLE
con = MySQLdb.connect('localhost','root','','tweetsdb01')
cursor = con.cursor()
cursor.execute("CREATE TABLE IF NOT EXISTS User(user_id BIGINT NOT NULL PRIMARY KEY, username varchar(25) NOT NULL,user varchar(25) NOT NULL) CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE=InnoDB")
con.commit()
INSERT INTO
def populate_user(a,b,c):
con = MySQLdb.connect('localhost','root','','tweetsdb01')
cursor = con.cursor()
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(%s,%s,%s)"%(a,b,c))
con.commit()
cursor.close()
READ FILE- this calls the populate method above
def read(file):
json_data=open(file)
tweets = []
for i in range(10):
tweet = json.loads(json_data.readline())
populate_user(tweet['from_user_id'],tweet['from_user_name'],tweet['from_user'])

Use parametrized SQL:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES (%s,%s,%s)", (a,b,c))
(Notice the values (a,b,c) are passed to the function execute as a second argument, not as part of the first argument through string interpolation). MySQLdb will properly quote the arguments for you.
PS. As Vajk Hermecz notes, the problem occurs because the string 'Ellie' is not being properly quoted.
When you do the string interpolation with "(%s,)" % (a,) you get
(Ellie,) whereas what you really want is ('Ellie',). But don't bother doing the quoting yourself. It is safer and easier to use parametrized SQL.

Your problem is that you are adding the values into the query without any escaping.... Now it is just broken. You could do something like:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(\"%s\",\"%s\",\"%s\")"%(a,b,c))
But that would just introduce SQL INJECTION into your code.
NEVER construct SQL statements with concatenating query and data. Your parametrized queries...
The proper solution here would be:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(%s,%s,%s)", (a,b,c))
So, the problem with your code was that you have used the % operator which does string formatting, and finally you just gave one parameter to cursor.execute. Now the proper solution, is that instead of doing the string formatting yourself, you give the query part to cursor.execute in the first parameter, and provide the tuple with arguments in the second parameter.