I am having the problem
OperationalError: (1054, "Unknown column 'Ellie' in 'field list'")
With the code below, I'm trying to insert data from json into a my sql database. The problem happens whenever I try to insert a string in this case "Ellie" This is something do to with string interpolation I think but I cant get it to work despite trying some other solutions I have seen here..
CREATE TABLE
con = MySQLdb.connect('localhost','root','','tweetsdb01')
cursor = con.cursor()
cursor.execute("CREATE TABLE IF NOT EXISTS User(user_id BIGINT NOT NULL PRIMARY KEY, username varchar(25) NOT NULL,user varchar(25) NOT NULL) CHARACTER SET utf8 COLLATE utf8_unicode_ci ENGINE=InnoDB")
con.commit()
INSERT INTO
def populate_user(a,b,c):
con = MySQLdb.connect('localhost','root','','tweetsdb01')
cursor = con.cursor()
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(%s,%s,%s)"%(a,b,c))
con.commit()
cursor.close()
READ FILE- this calls the populate method above
def read(file):
json_data=open(file)
tweets = []
for i in range(10):
tweet = json.loads(json_data.readline())
populate_user(tweet['from_user_id'],tweet['from_user_name'],tweet['from_user'])
Use parametrized SQL:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES (%s,%s,%s)", (a,b,c))
(Notice the values (a,b,c) are passed to the function execute as a second argument, not as part of the first argument through string interpolation). MySQLdb will properly quote the arguments for you.
PS. As Vajk Hermecz notes, the problem occurs because the string 'Ellie' is not being properly quoted.
When you do the string interpolation with "(%s,)" % (a,) you get
(Ellie,) whereas what you really want is ('Ellie',). But don't bother doing the quoting yourself. It is safer and easier to use parametrized SQL.
Your problem is that you are adding the values into the query without any escaping.... Now it is just broken. You could do something like:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(\"%s\",\"%s\",\"%s\")"%(a,b,c))
But that would just introduce SQL INJECTION into your code.
NEVER construct SQL statements with concatenating query and data. Your parametrized queries...
The proper solution here would be:
cursor.execute("INSERT INTO User(user_id,username,user) VALUES(%s,%s,%s)", (a,b,c))
So, the problem with your code was that you have used the % operator which does string formatting, and finally you just gave one parameter to cursor.execute. Now the proper solution, is that instead of doing the string formatting yourself, you give the query part to cursor.execute in the first parameter, and provide the tuple with arguments in the second parameter.
Related
I don't know how to make this SQL Injection work in SQLite. I'm using a function in Python that connects to a database and inserts a string.
I have "database.db" that has two tables: "feedback" and "users".
The feedback table has 1 column: message.
The users table has 2 columns: username and password.
def send_feedback(feedback):
conn = sqlite3.connect("database.db")
curs = conn.cursor()
curs.execute("INSERT INTO feedback VALUES ('%s')" % (feedback))
print(curs.fetchall())
conn.close()
I know that the execute function allows me to make a single query to the database, so I can't use ";" to
make multiple queries.
What I have tried, is to make the string look like this:
a') SELECT password FROM users --
feedback = "INSERT INTO feedback VALUES ('a') SELECT password FROM users --')"
But this gives me the following error:
sqlite3.OperationalError: near "SELECT": syntax error
So I've tried to use the UNION command:
a') UNION SELECT password FROM users --
feedback = "INSERT INTO feedback VALUES ('a') UNION SELECT password FROM users --')"
This one works but the fetchall function returns an empty list.
Most SQL injections result in nothing useful to the perpetrator, just a syntax error.
For example, pass the string "I'm not satisfied" to this feedback function and the extra ' character would cause the quotes to be imbalanced, and this would result in an error, causing the INSERT to fail.
sqlite3.OperationalError: near "m": syntax error
That's technically SQL injection. The content interpolated into the query has affected the syntax of the SQL statement. That's all. It doesn't necessarily result in a successful "Mission: Impossible" kind of infiltration.
I can't think of a way to exploit the INSERT statement you show to make it do something clever, besides causing an error.
You can't change an INSERT into a SELECT that produces a result set. Even if you try to inject a semicolon followed by a second SQL query, you just get sqlite3.Warning: You can only execute one statement at a time
Your first try above resulted in a syntax error because you had both a VALUES clause and a SELECT as a source for the data to insert. You can use either one but not both in SQL syntax. See https://www.sqlite.org/lang_insert.html
You probably already know how to make the code safe, so unsafe content cannot even cause a syntax error. But I'll include it for other readers:
curs.execute("INSERT INTO feedback VALUES (?)", (feedback,))
You can do it, for example to get table name
a' || (SELECT tbl_name FROM sqlite_master WHERE type='table' and tbl_name NOT like 'sqlite_%'))-- -
My objective is to store a JSON object into a MySQL database field of type json, using the mysql.connector library.
import mysql.connector
import json
jsonData = json.dumps(origin_of_jsonData)
cnx = mysql.connector.connect(**config_defined_elsewhere)
cursor = cnx.cursor()
cursor.execute('CREATE DATABASE dataBase')
cnx.database = 'dataBase'
cursor = cnx.cursor()
cursor.execute('CREATE TABLE table (id_field INT NOT NULL, json_data_field JSON NOT NULL, PRIMARY KEY (id_field))')
Now, the code below WORKS just fine, the focus of my question is the use of '%s':
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES (%s, %s)"
values_to_insert = (1, jsonData)
cursor.execute(insert_statement, values_to_insert)
My problem with that: I am very strictly adhering to the use of '...{}'.format(aValue) (or f'...{aValue}') when combining variable aValue(s) into a string, thus avoiding the use of %s (whatever my reasons for that, let's not debate them here - but it is how I would like to keep it wherever possible, hence my question).
In any case, I am simply unable, whichever way I try, to create something that stores the jsonData into the mySql dataBase using something that resembles the above structure and uses '...{}'.format() (in whatever shape or form) instead of %s. For example, I have (among many iterations) tried
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES ({}, {})".format(1, jsonData)
cursor.execute(insert_statement)
but no matter how I turn and twist it, I keep getting the following error:
ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[some_content_from_jsonData})]' at line 1
Now my question(s):
1) Is there a way to avoid the use of %s here that I am missing?
2) If not, why? What is it that makes this impossible? Is it the cursor.execute() function, or is it the fact that it is a JSON object, or is it something completely different? Shouldn't {}.format() be able to do everything that %s could do, and more?
First of all: NEVER DIRECTLY INSERT YOUR DATA INTO YOUR QUERY STRING!
Using %s in a MySQL query string is not the same as using it in a python string.
In python, you just format the string and 'hello %s!' % 'world' becomes 'hello world!'. In SQL, the %s signals parameter insertion. This sends your query and data to the server separately. You are also not bound to this syntax. The python DB-API specification specifies more styles for this: DB-API parameter styles (PEP 249). This has several advantages over inserting your data directly into the query string:
Prevents SQL injection
Say you have a query to authenticate users by password. You would do that with the following query (of course you would normally salt and hash the password, but that is not the topic of this question):
SELECT 1 FROM users WHERE username='foo' AND password='bar'
The naive way to construct this query would be:
"SELECT 1 FROM users WHERE username='{}' AND password='{}'".format(username, password)
However, what would happen if someone inputs ' OR 1=1 as password. The formatted query would then become
SELECT 1 FROM users WHERE username='foo' AND password='' OR 1=1
which will allways return 1. When using parameter insertion:
execute('SELECT 1 FROM users WHERE username=%s AND password=%s', username, password)
this will never happen, as the query will be interpreted by the server separately.
Performance
If you run the same query many times with different data, the performance difference between using a formatted query and parameter insertion can be significant. With parameter insertion, the server only has to compile the query once (as it is the same every time) and execute it with different data, but with string formatting, it will have to compile it over and over again.
In addition to what was said above, I would like to add some details that I did not immediately understand, and that other (newbies like me ;)) may also find helpful:
1) "parameter insertion" is meant for only for values, it will not work for table names, column names, etc. - for those, the Python string substitution works fine in the sql syntax defintion
2) the cursor.execute function requires a tuple to work (as specified here, albeit not immediately clear, at least to me: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html)
EXAMPLE for both in one function:
def checkIfRecordExists(column, table, condition_name, condition_value):
...
sqlSyntax = 'SELECT {} FROM {} WHERE {} = %s'.format(column, table, condition_name)
cursor.execute(sqlSyntax, (condition_value,))
Note both the use of .format in the initial sql syntax definition and the use of (condition_value,) in the execute function.
I have seen some posts that suggesting using a ? as a place holder when inserting python variables into a SQL Query but all of these examples show the question mark at the end of the query followed by the python variable. What if you want to insert a python variable in the middle of a query and want to avoid SQL injection? I am using Python 3.6 and SQLite.
Update* - This code is working:
id='13'
text='YES'
db=sqlite3.connect('NEW_Inventory.sqlite')
cursor=db.cursor()
query=('''
INSERT
OR REPLACE
INTO
text (id, text)
VALUES
(?,
(SELECT
CASE
WHEN exists(SELECT 1 FROM text WHERE id=?)
THEN 'good'
ELSE 'Hello'
END
)
)''')
cursor.execute(query, (id, id))
db.commit()
You need to pass the parameters to execute() as a tuple. In your case you need to call it like this:
cursor.execute(query, (id, id))
where query is your parameterised SQL query string.
I assume that your code defines id somewhere, otherwise, execute() will try to use the builtin function id() to construct the query, resulting in another error.
It also worth mentioning that if you have only one parameter it must also be passed as a tuple like this (id,). Avoid the common mistake of this: (id) which is not a tuple.
Could you please help me figure out what I am doing wrong?
I am trying to insert a new player into the players table.
this is the python code:
def registerPlayer(name):
code for connecting to db and cursor
c.execute("INSERT INTO players(player_name) VALUES({name});".format(name=name))
code for committing to db and closing the connection
here is my table schema:
CREATE TABLE players(
player_id serial PRIMARY KEY,
player_name varchar(50) NOT NULL
);
below is error:
psycopg2.ProgrammingError: syntax error at or near "Nalaar" LINE 1:
INSERT INTO players(player_name) VALUES(Chandra Nalaar);
You should never use string formatting for placing values into sql query. Instead, you should use %s and pass the name in the vars parameter. The reason behind doing it this way is because it helps you convert the parameters into the appropriate data types.
Btw, having a ; at the end in the sql string is redundant when called by cursor.execute, since it does it for you automatically.
c.execute("INSERT INTO players(player_name) VALUES(%(name)s)", {"name":name})
See this page for further details:
http://initd.org/psycopg/docs/usage.html#query-parameters
I'm trying execute a simple query, but getting this error no matter how I pass the parameters.
Here is the query (I'm using Trac db object to connect to a DB):
cursor.execute("""SELECT name FROM "%s".customer WHERE firm_id='%s'""" % (schema, each['id']))
schema and each['id'] both are simple strings
print("""SELECT name FROM "%s".customer WHERE firm_id='%s'""" % (schema, each['id']))
Result:
SELECT name FROM "Planing".customer WHERE firm_id='135'
There is on error is a remove quote after firm_id=, but that way parameter is treated a an integer and ::text leads to the very same error.
In my case I didn't realize that you had to pass a tuple to cursor.execute. I had this:
cursor.execute(query, (id))
But I needed to pass a tuple instead
cursor.execute(query, (id,))
I got this same error and couldn't for the life of me work out how to fix, in the end it was my mistake because I didn't have enough parameters matching the number of elements in the tuple:
con.execute("INSERT INTO table VALUES (%s,%s,%s,%s,%s)",(1,2,3,4,5,6))
Note that I have 5 elements in the values to be inserted into the table, but 6 in the tuple.
It is recommended to not use string interpolation for passing variables in database queries, but using string interpolation to set the table name is fine as long as it's not an external input or you restrict the allowed value. Try:
cursor.execute("""
SELECT name FROM %s.customer WHERE firm_id=%%s
""" % schema, (each['id'],))
Rules for DB API usage provides guidance for programming against the database.
Use AsIs
from psycopg2.extensions import AsIs
cursor.execute("""
select name
from %s.customer
where firm_id = %s
""",
(AsIs(schema), each['id'])
)
You could try this:
cursor.execute("INSERT INTO table_name (key) VALUES(%s)",(value1,))
You will get an error if you are missing a (,) after value1.
The correct way to pass variables in a SQL command is using the second argument of the execute() method. And i think you should remove single quotes from second parameter, read about it here - http://initd.org/psycopg/docs/usage.html#the-problem-with-the-query-parameters.
Note that you cant pass table name as parameter to execute and it considered as bad practice but there is some workarounds:
Passing table name as a parameter in psycopg2
psycopg2 cursor.execute() with SQL query parameter causes syntax error
To pass table name try this:
cursor.execute("""SELECT name FROM "%s".customer WHERE firm_id=%s""" % (schema, '%s'), (each['id'],))
Every time I have this kind of error, I am passing the wrong amount of values. Try check it