Develop Reference

Python MySQL statement to prevent SQL Injection [duplicate]

I am using pymyql/mysql-connector to write the messages to mysql database. The messages are processed on callback (paho.mqtt callback) from mqtt broker.I have 4 different tables and based on the message type, I am inserting messages into database. I have written the insert queries as below. this way of writing leads to sql injections it seems.Any suggestions how can I improve the insert query statements?
# callback attached to paho.mqtt.client
def on_message(self, client, userdata, msg):
if msg.topic.startswith("topic1/"):
self.bulkpayload += "(" + msg.payload.decode("utf-8") + "," + datetime + "),"
elif msg.topic.startswith("topic2/"):
self.insertStatement += "INSERT INTO mydatabase.table1 VALUES (" + msg.payload.decode("utf-8") + "," + datetime + ");"
elif msg.topic.startswith("topic3/")
self.insertStatement += "INSERT INTO mydatabase.table2 VALUES (" +msg.payload.decode("utf-8") + "," + datetime + ");"
elif msg.topic.startswith("messages"):
self.insertStatement += "INSERT INTO mydatabase.table3 VALUES ('" + msg.topic + "'," + msg.payload.decode("utf-8") + "," + datetime + ");"
else:
return # do not store in DB
cursor.execute(self.insertStatement)
cursor.commit()

Make your query use parameters. Much less chance of injection:
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
credit (and more info) here: How to use variables in SQL statement in Python?
Also, Dan Bracuk is correct - make sure you validate your params before executing the SQL if you aren't already

pymysql.err.IntegrityError: (1062, "Duplicate entry '8207' for key 'agentId'")

"insert into agentattribute(agentid,attributeRefURL,attributeName,attributeValue,Operation,attributeOldValue) values (" + live_agent_id + ",'" + live_entry + "','" +live_name_value_array[0] + "','" + live_name_value_array[1] + "', 'Added' ,',')")
Can someone help me out please?

As the error message explains, agentId column has primary key/unique constraint on it and value 8207 already exists.
In this case, you can go ahead with one of the below options:
Update the record if the value already exists
Propogate the error message back to User Interface/application

How do I set Python None to db3 Null

I have the following values In Python values:
_sfwBIPMKEeCTfuMZpaDEBQ Atomic Warehouse Model None
These values are sent through a function and is being put into a db3 table like this:
INSERT INTO [PackageRoster] (PkgID, Package, PkgParentID) VALUES ('_sfwBIPMKEeCTfuMZpaDEBQ', 'Atomic Warehouse Model', 'None');
I do not want the string 'None' put into the table. I would like NULL.
How do I tell Python to inject NULL instead of None?
Here's the function that works the statement (includes debug feature):
def StorePackage(PkgID, Package, PkgParentID):
dbutils.ResetTable()
conn = sqlite3.connect(dbutils.db)
try:
conn.execute("INSERT INTO " + dbutils.table + " (PkgID, Package, PkgParentID) VALUES ('" + PkgID + "', '" + Package + "', '" + PkgParentID + "');")
except:
print("INSERT INTO " + dbutils.table + " (PkgID, Package, PkgParentID) VALUES ('" + PkgID + "', '" + Package + "', '" + str(PkgParentID) + "');")
conn.commit()
Thanks

I initially commented a recommendation to use parameter substitution, assuming it wasn't actually related. On further inspection, though, I think it's exactly what you're looking for.
Your code currently inserts all strings because you're explicitly converting everything to a string in Python. Parameter substitution enables Python to handle how to get its values into the database, and also protects you from SQL injection attacks.
conn.execute("INSERT INTO " + dbutils.table + " (PkgID, Package, PkgParentID) VALUES (?,?,?)", (PkgID, Package, PkgParentID))
A Python None object will be inserted as an SQLite NULL using this method. You can read more about parameter substitution in the docs, as well as how Python values are converted to SQLite values and vice versa.

You are converting all of your values to strings, because you add quotes arround them. I.e, because you do '" + str(PkgParentID) + "' it will end up as 'None' instead of None.
You should for example convert your values to a string before the query, and replace them with NULL when they are None:
def StorePackage(PkgID, Package, PkgParentID):
PkgParentID = "'{}'".format(PkgParentID) if PkgParentID is not None else "NULL"
# Same for other parameters if desired
dbutils.ResetTable()
conn = sqlite3.connect(dbutils.db)
query = "INSERT INTO " + dbutils.table + " (PkgID, Package, PkgParentID) VALUES ('" + PkgID + "', '" + Package + "', " + PkgParentID + ");"
try:
conn.execute(query)
except:
print(query)
conn.commit()
Also you should use parameter substitution, although thats not what is causing your issue in the first place

How to trim white space in MySQLdb using python

I have a web form taking user data and putting it into a mysql database. I need to trim leading and trailing blanks/spaces. I currently am utilizing the strip() method, but it only trims them for the fields, NOT for the mysql database.
My code is:
first_name = first_name.strip()
last_name = last_name.strip()
so on and so forth. It strips it perfectly fine for the webpage, but not when it is entered into the SQL database. The spaces still exist. How do I remove them?
EDIT:
db = MySQLdb.connect("localhost","user","pass","db_name")
cursor = db.cursor()
cursor.execute("Select * FROM registrants")
cursor.execute("INSERT INTO registrants VALUES( " + "'" + first_name + "'" + ", " + "'" + last_name + "'" + ");")
db.commit()
db.close()

It could be a scope issue.
If the stripping occurs in a different scope (ex: first_name is a global variable and the strip() occurs in a function) then you will not benefit from it in another scope (if the insert is happening in another function for example).
Have you tried this for a test:
cursor.execute("INSERT INTO registrants VALUES( " + "'" + first_name.strip() + "'" + ", " + "'" + last_name.strip() + "'" + ");")
btw, how do you know there's a space in the db? There could be an issue with the way the data is retrieved or displayed..

I think you should be passing the values into the INSERT differently
cursor.execute("INSERT INTO registrants (fname, lname) VALUES (%s, %s)", (first_name, last_name)
I'm not sure if that's where you're getting the whitespace, but it opens you up to sql injection so it's bad form to put the variables straight into the query.

cant resolve cryptic error in Python/sqlite program

Im running into this error that I can't work out
Im writing some code in Python using tkinter interface to transfer data from a text file to sqlite.
first, here is the relevant code:
def submit_data(self):
self.new_filename = self.file_entry.get()
self.new_tablename = self.table_entry.get()
self.new_fieldname = self.field_entry.get().split(',')
#print(self.new_fieldname)
self.create_new.destroy()
from sqlite3 import dbapi2 as sqlite
con = sqlite.connect(self.new_filename)
cur = con.cursor()
cur.execute('CREATE TABLE ' + self.new_tablename + '(id INTEGER PRIMARY KEY)')
for field in self.new_fieldname:
cur.execute('ALTER TABLE ' + self.new_tablename + ' ADD ' + field)
with open(self.filename, 'r', encoding='latin-1') as self.the_file:
status = True
#_keynumber=1
while status:
_row = self._next_line()
if _row:
_entry_list = _row.split(',')
# add space after text line comma for formatting
_entry_list = ', '.join(_entry_list)
#print(_entry_list)
#entries = {'row': _keynumber, 'entry': _entry_list}
#row_entry = "INSERT INTO " + self.new_tablename + " VALUES(" + _entry_list + ")"
cur.execute("INSERT INTO " + self.new_tablename + " VALUES(" + _entry_list + ")")
#_colrange = range(_colamount)
#_keynumber+=1
else:
status = False
con.commit()
At the cur.execute("INSERT INTO " ... line (about 6 lines up) I get this error:
** cur.execute("INSERT INTO " + self.new_tablename + " VALUES(" + _entry_list + ")")
sqlite3.OperationalError: near ".": syntax error**
I have changed this around in many different ways. At one time I had the whole "INSERT INTO ... VALUES ...." string as a variable and used
cur.execute(*variable*)
when I did it this way the error was the same except "OperationalError: near "." was "OperationalError: near "of" ... and there was no 'of' anywhere.
Im really confused and frustrated. Someone break this down for my please??
Thanks
F
the text file lines its reading are set up like this:
A Big Star In Hollywood,Sandra Dickinson
so I had figured that if I use .join() to put a space after the comma then the string would be the equivalent of two VALUES for the INSERT INTO statement.

Remove
_entry_list = ', '.join(_entry_list)
and use
cur.execute("INSERT INTO " + self.new_tablename + "(" + ",".join(self.new_fieldname) +") VALUES(" + ",".join(("?" for i in xrange(len(_entry_list)))) + ")", _entry_list)
This will parameterize your query and automatically quote all value in _entry_list.
You still have to manually quote self.new_tablename and self.new_fieldname. This should be before you use them in any sql statements.

You need to quote your strings.
As written, your SQL statement is:
INSERT INTO foo VALUES(Hello there, world, I am, unquoted, string, not good)
You should use:
INSERT INTO foo VALUES("Hello there","world","I am","quoted","string","hooray")

I suggest you do the following:
a) Instead of executing the statement, print it to the console. Change the line:
cur.execute("INSERT INTO " + self.new_tablename + ...)
to:
print "INSERT INTO " + self.new_tablename + ...
b) Run your program after making this change. Take a look at the SQL statements that you print to the console. Are they valid SQL statements? Start a SQLite command-line, and copy/paste the statements produced by your program. Does SQLite give any errors when you try to execute the pasted statements?