I'm creating a change-password page for a website, which requests the new password and the current password. The old password is hashed and salted using the scrypt library then compared to the password stored in the sqlite3 database, and if these are a match, the new password is hashed and the database is updated. However I am having difficulty executing the update command, as it throws a sqlite3.OperationalError: unrecognised token: "\" error. The execute statement currently has the following code:
c.execute("UPDATE users SET password = \'{0}\' WHERE memberID = \'{1}\'".format(newPas, memID))
Initially we believed this error to have been caused by the use of ' in the string formatting due to the presence of ' within the new password itself, so this was run again as:
c.execute("UPDATE users SET password = \"{0}\" WHERE memberID = \"{1}\"".format(newPas, memID))
This successfully runs, but doesn't actually change anything in the database. We also attempted to create a query string and then execute the string.
query = "UPDATE users SET password = {0} WHERE memberID = {1}".format(newPas, memID)
c.execute(query)
This caused a sqlite3.OperationalError: near "'\xa1\x91\x9f\x88\xfb\x81\x12\xd4\xc2\xf9\xce\x91y\xf0/\xe1*#\x8aj\xc7\x1d\xd3\x91\x14\xcb\xa4\xabaP[\x02\x1d\x1b\xabr\xc7\xe4\xee\x19\x80c\x8e|\xc0S\xaaX\xc6\x04\xab\x08\x9b\x8e\xd7zB\xc6\x84[\xfb\xbc\x8d\xfc'": syntax error. I believe that this is caused by the presence of ' and " characters within the password, but I am unsure how to get around this issue as these are added by the hashing process and thus removing them would change the password.
The password I would like to add is:
b'\xa1\x91\x9f\x88\xfb\x81\x12\xd4\xc2\xf9\xce\x91y\xf0/\xe1*#\x8aj\xc7\x1d\xd3\x91\x14\xcb\xa4\xabaP[\x02\x1d\x1b\xabr\xc7\xe4\xee\x19\x80c\x8e|\xc0S\xaaX\xc6\x04\xab\x08\x9b\x8e\xd7zB\xc6\x84[\xfb\xbc\x8d\xfc'
I was wondering if anyone could share some insights into why it isn't liking the "\" character or why it isn't updating the database, and point me in the right direction to making it work. If you need more information or code snippets or just want to yell at me, please don't hesitate to! Thank you in advance :)
A couple of things with your code:
You should not use format to build your queries like this. This leaves you liable to SQL injection and, whilst you might sanitise your inputs in this case, it's a bad habit that will bite you.
All changes need to be committed to the database to actually take effect. This is why your second query did not throw an error but equally did not make any changes to the database.
The correct formatting of this query would be:
conn = sqlite3.connect('my_db.db')
c = conn.cursor()
query = "UPDATE users SET password = ? WHERE memberID = ?"
c.execute(query, (newPas, memID))
conn.commit() # To finalise the alteration
As a side note, the cursor expects a tuple in this case, so a common stumbling block comes when passing single values:
query = "UPDATE users SET password = ? WHERE memberID = 'abc'"
c.execute(query, (newPas)) # Throws "incorrect number of bindings" error
# Use this instead i.e. pass single value as a tuple
c.execute(query, (newPas,))
You could use format to create variable field names in a query, since placeholders are not allowed in this case:
fields = ['a', 'b', 'c']
query = "UPDATE users SET {} = ?".format(random.choice(fields))
in addition to using it to help you build big queries where it would be tedious to manually type all the placeholders, and difficult to ensure that you had the correct number if your code changed:
my_list = ['a', 'b',...., n]
placeholders = ', '.join(['?' for item in my_list])
query = "INSERT .... VALUES = ({})".format(placeholders)
You should use parametrized queries something like this:
c.execute("""UPDATE users SET password = ? WHERE memberID = ?;""", (newPas, memID))
It will allow to avoid nasty things like SQL-injections.
Related
def overwriteFlat(top, curTable, rawEntrylist, columns):
rawEntrylist = rawEntrylist
entryList = list()
for value in rawEntrylist:
entryList.append(value.get())
conn = sqlite3.connect('data.db')
c = conn.cursor()
for i in range(len(columns)):
if entryList[i] != '':
c.execute("""UPDATE """+curTable+""" SET """+columns[i]+""" = :"""+columns[i]+""" WHERE """+columns[0]+""" = """ + str(entryList[0]), {columns[i]: entryList[i]})
print(curTable,columns[i],entryList[i])
conn.commit()
c.close()
conn.close()
closeWin(top)
Output:
Flat ID 23
Flat Street Test
Flat Street_Number 100
I put in "Test" and "100" so that works. I provide a window for input, the input gets put into here and everything provided gets overwritten in provided ID. Because of print() I see it goes into the right table, it also selects the right column and doesn't throw any exception. But it doesn't update database.
Database not locked.
Variables all valid and work.
No exception is thrown.
Vulnerable to injection, soon as it works I'll change it.
Thanks to #JohnGordon i found the solution
But just so if someone wants to use Variables in Sqlite i will explain how as this is hardly explained anywhere on the Internet (at least at my Beginner-Programmer-Level)
Usually Sql commands work like this and are pretty static:
"UPDATE Your_Table SET Your_Column = :Your_Column WHERE IndexColumn = Your_Index), {Your_Column: Your_Value}"
But by using +Variable+ you can use Variables in there
so its the same thing but with whatever Variable you want:
"UPDATE "+curTable+" SET "+columns[i]+" = :"+columns[i]+" WHERE "+columns[i]+" = " + str(entryList[0]), {columns[i]: entryList[i]}
You can now have the Variables "curTable", "columns", "entryList" set to whatever you want and dont need a static line for everything
The same works with INSERT and the other things too
Edit: (its now 3 hours later, 1 AM and i got the safer way)
NOW THAT YOU GOT THAT READ THIS
you will be vulnerable to SQL Injection, and you need to still change that code to this:
query = " UPDATE "+curTable+" SET "+columns[i]+" = ? WHERE "+columns[0]+" = ?"
c.execute(query, (entryList[i], entryList[0], ))
this makes it safer, but as i am not a pro yet maybe someone can confirm
Edit: Removed triple-quotes as they are only needed in multiple-sentence sql stuff thanks for the hint #Tim Roberts
I have a python script that interfaces with a mysql database i have. Each time i run it, it gives me a different error every time. I am becoming so confused as to what to do to fix it.
Basically, the code is meant for an user to input a search parameter for an account database, and the script searches a MySQL table and brings up data about the account.
As of now i get no errors, but it returns absolutely no search results.
I used to be able to make it search by an EXACT username, but i wanted it so you can search for a term within that username. Every attempt at the latter always results in some sort of error or i get no results back from MySQL.
import mysql.connector
users1 = mysql.connector.connect(
host="localhost",
user="python",
passwd="HaHaYou'reNotGettingMyPassword",
database="accounts"
)
cursor=users1.cursor()
usersearch = input("Please input the search term: ")
sql = ("SELECT user_id, username, date, status, description, gen FROM users1 WHERE username LIKE %s")
cursor.execute(sql, ('"' + usersearch + '"'))
result = cursor.fetchall()
for x in result:
print(x)
print("In Order: User ID, Username, Account Creation Date, bla bla bla")
EDIT: i figured out i think my SQL syntax is incorrect. i'll try an fix that and see if that was my only problem.
Try :
sql = ("SELECT user_id, username, date, status, description, gen FROM users1 WHERE username LIKE %s")
cursor.execute(sql, [usersearch])
If you get results when specifying a full username with the above, you probably need to add "wildcards" to your search term to "find within" existing strings. The wildcard "%" matches 0 or more characters.
E.g. change your string concatenation to:
cursor.execute(sql, ('"%' + usersearch + '%"'))
WARNING: This input style is wide open to SQL Injection Attack, as user input is directly used to create part of the SQL statement that is sent to the DB engine. Please see this answer for mitigation methods.
The statement is set-up so that when a record already exists, it doesn't add a record, else, it does.
I've tried changing the query, even though I don't see anything wrong with it.
I've let the script run on python, and print the query it executed. Then I pasted that query in phpmyadmin, where it executed succesfully.
I have also double checked all parameters.
Query (blank params):
INSERT INTO users (uname,pass) SELECT * FROM (SELECT '{}','{}') AS tmp WHERE NOT EXISTS(SELECT uname FROM users WHERE uname = '{}') LIMIT 1;
Query (filled in parameters):
INSERT INTO users (uname,pass) SELECT * FROM (SELECT 'john_doe','password') AS tmp WHERE NOT EXISTS(SELECT uname FROM users WHERE uname = 'john_doe') LIMIT 1;
Python script (the important part)
if action == "add_user":
username = form.getvalue('username')
password = form.getvalue('password')
query = """
INSERT INTO users (uname,pass) SELECT * FROM
(SELECT '{}','{}') AS tmp WHERE NOT EXISTS(SELECT uname FROM users WHERE uname = '{}') LIMIT 1;
""".format(username, password, username)
mycursor.execute(query)
I know a couple of things.
There is nothing wrong with the database connection.
The parameters are not empty (ex. username="john_doe" & password="secret")
The query actually executes in that specific table.
The query seems to add a record and delete it directly afterwards (as AUTO_INCREMENT increases each time, even when the python script executes and doesn't add anything)
A try except doesn't do anything, as mysql.connector.Error doesn't report any error (which is obvious, since the query actually executes succesfully)
phpMyAdmin practical example:
(Removed INSERT INTO part in order to be able to show the resulting tables)
The first time you enter the query (above query as example), it will result in a table with both values as both column names as column values.
Screenshot of table output: http://prntscr.com/nkgaka
Once that result is entered once, next time you will try to insert it, it will simply result in only column names, no values. This means it will insert nothing, as there is nothing to insert as there are no actual values.
Screenshot of table output: http://prntscr.com/nkgbp3
Help is greatly appreciated.
If you want to ensure any field is unique in a table, make that field a PRIMARY KEY or UNIQUE KEY. In this case you want name to be unique.
CREATE UNIQUE INDEX name ON users(name)
With this in place you only need to INSERT. If a duplicate key error occurs the name already exists.
To avoid SQL injection, don't use """SELECT {}""".format. It will be SQL injection vulnerable.
Also don't store plain text passwords, ever. Salted hashes at least. There's plenty of frameworks that do this well already so you don't need to invent your own.
My objective is to store a JSON object into a MySQL database field of type json, using the mysql.connector library.
import mysql.connector
import json
jsonData = json.dumps(origin_of_jsonData)
cnx = mysql.connector.connect(**config_defined_elsewhere)
cursor = cnx.cursor()
cursor.execute('CREATE DATABASE dataBase')
cnx.database = 'dataBase'
cursor = cnx.cursor()
cursor.execute('CREATE TABLE table (id_field INT NOT NULL, json_data_field JSON NOT NULL, PRIMARY KEY (id_field))')
Now, the code below WORKS just fine, the focus of my question is the use of '%s':
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES (%s, %s)"
values_to_insert = (1, jsonData)
cursor.execute(insert_statement, values_to_insert)
My problem with that: I am very strictly adhering to the use of '...{}'.format(aValue) (or f'...{aValue}') when combining variable aValue(s) into a string, thus avoiding the use of %s (whatever my reasons for that, let's not debate them here - but it is how I would like to keep it wherever possible, hence my question).
In any case, I am simply unable, whichever way I try, to create something that stores the jsonData into the mySql dataBase using something that resembles the above structure and uses '...{}'.format() (in whatever shape or form) instead of %s. For example, I have (among many iterations) tried
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES ({}, {})".format(1, jsonData)
cursor.execute(insert_statement)
but no matter how I turn and twist it, I keep getting the following error:
ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[some_content_from_jsonData})]' at line 1
Now my question(s):
1) Is there a way to avoid the use of %s here that I am missing?
2) If not, why? What is it that makes this impossible? Is it the cursor.execute() function, or is it the fact that it is a JSON object, or is it something completely different? Shouldn't {}.format() be able to do everything that %s could do, and more?
First of all: NEVER DIRECTLY INSERT YOUR DATA INTO YOUR QUERY STRING!
Using %s in a MySQL query string is not the same as using it in a python string.
In python, you just format the string and 'hello %s!' % 'world' becomes 'hello world!'. In SQL, the %s signals parameter insertion. This sends your query and data to the server separately. You are also not bound to this syntax. The python DB-API specification specifies more styles for this: DB-API parameter styles (PEP 249). This has several advantages over inserting your data directly into the query string:
Prevents SQL injection
Say you have a query to authenticate users by password. You would do that with the following query (of course you would normally salt and hash the password, but that is not the topic of this question):
SELECT 1 FROM users WHERE username='foo' AND password='bar'
The naive way to construct this query would be:
"SELECT 1 FROM users WHERE username='{}' AND password='{}'".format(username, password)
However, what would happen if someone inputs ' OR 1=1 as password. The formatted query would then become
SELECT 1 FROM users WHERE username='foo' AND password='' OR 1=1
which will allways return 1. When using parameter insertion:
execute('SELECT 1 FROM users WHERE username=%s AND password=%s', username, password)
this will never happen, as the query will be interpreted by the server separately.
Performance
If you run the same query many times with different data, the performance difference between using a formatted query and parameter insertion can be significant. With parameter insertion, the server only has to compile the query once (as it is the same every time) and execute it with different data, but with string formatting, it will have to compile it over and over again.
In addition to what was said above, I would like to add some details that I did not immediately understand, and that other (newbies like me ;)) may also find helpful:
1) "parameter insertion" is meant for only for values, it will not work for table names, column names, etc. - for those, the Python string substitution works fine in the sql syntax defintion
2) the cursor.execute function requires a tuple to work (as specified here, albeit not immediately clear, at least to me: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html)
EXAMPLE for both in one function:
def checkIfRecordExists(column, table, condition_name, condition_value):
...
sqlSyntax = 'SELECT {} FROM {} WHERE {} = %s'.format(column, table, condition_name)
cursor.execute(sqlSyntax, (condition_value,))
Note both the use of .format in the initial sql syntax definition and the use of (condition_value,) in the execute function.
I am learning myself some programming with python and make use of SQlite3
I keep running into the same problem, and I can't figure out what goes wrong.
My table setup
def user_table():
data = lite.connect(database)
dat = data.cursor()
with data:
dat.executescript("""CREATE TABLE IF NOT EXISTS Users(Id INTEGER PRIMARY KEY AUTOINCREMENT,
'Username' UNIQUE,
'Password',
'Email',
'UserCharacters_Id' INTEGER
)""");
Now my code to select a Username (the username 123 exists and tables seem right (checked with SQLite studio)
database = 'test.db'
data = lite.connect(database)
dat = data.cursor()
with data:
dat.execute("SELECT * FROM Users WHERE 'Username'='123'")
user = dat.fetchone()
print user
I tried a lot of different ways, but it keeps returning None.
The python part seems to be working, just the select part of SQL goes wrong (checked with prints)
Please help me out
In SQL, single quotes are used for strings, while table/columns names are quoted with double quotes.
(SQLite supports single quotes for the latter for compatibility with MySQL in some places.)
Your query compares the string Username against the string 123, and this comparison fails for every record.
Use this:
dat.execute('SELECT * FROM Users WHERE "Username" = \'123\'')
But to prevent string formatting problems and SQL injection attacks, you should use parameters:
username = "123"
dat.execute('SELECT * FROM Users WHERE "Username" = ?', (username,))