Below I have some code that connects to database and runs a query. This query runs as expected if I punch it into DB Browser for SQLite, but when I run it in python, I receive the error :
sqlite3.OperationalError: no such column: CHARACTER_INNATES.PC_ID
I have read some documentation and multiple StackOverflow comments about parameters, and their purpose to prevent problems like injection attacks. I tried that approach, which littered '?' everywhere, making it unreadable and unmaintainable, and it didn't work. It returned an empty tuple. This leads me to believe there should be an easier way of performing these kinds of SQL queries. So is there? Or am I missing something obvious?
db = sqlite3.connect('TheGame.db')
db.text_factory = str
conn = db.cursor()
query = 'SELECT CHARACTERS.Full_Name, CHARACTER_INNATES.PC_ID, SKILLS_INNATES.Skill, Sum([CHARACTER_INNATES].[Current]*[Weight]) AS [Current Innate], Sum([CHARACTER_INNATES].[Maximum]*[Weight]) AS [Max Innate]\
FROM CHARACTERS INNER JOIN (SKILLS_INNATES INNER JOIN CHARACTER_INNATES ON SKILLS_INNATES.Innate = CHARACTER_INNATES.Innate) ON CHARACTERS.ID = CHARACTER_INNATES.PC_ID\
GROUP BY CHARACTERS.Full_Name, CHARACTER_INNATES.PC_ID, SKILLS_INNATES.Skill\
ORDER BY CHARACTER_INNATES.PC_ID, Sum([CHARACTER_INNATES].[Current]*[Weight]) DESC;'
conn.execute(query)
print conn.fetchall()
My attempt at a fix looked something like
params = ('CHARACTERS.Full_Name', 'CHARACTER_INNATES.PC_ID', ...) #this continued for awhile
query = 'SELECT ?, ?, ?, ...'
conn.execute(query, params)
print conn.fetchall() # prints empty tuple
This query already is unreadable.
Anyway, some SQLite versions have problem with table names hidden inside parentheses. This typically happens when you're using the Access query builder; just write the joins properly instead:
SELECT CHARACTERS.Full_Name,
CHARACTER_INNATES.PC_ID,
SKILLS_INNATES.Skill,
Sum(CHARACTER_INNATES.Current * Weight) AS "Current Innate",
Sum(CHARACTER_INNATES.Maximum * Weight) AS "Max Innate"
FROM CHARACTERS
INNER JOIN CHARACTER_INNATES ON CHARACTERS.ID = CHARACTER_INNATES.PC_ID
INNER JOIN SKILLS_INNATES USING (Innate)
GROUP BY CHARACTERS.Full_Name,
CHARACTER_INNATES.PC_ID,
SKILLS_INNATES.Skill
ORDER BY CHARACTER_INNATES.PC_ID,
Sum(CHARACTER_INNATES.Current * Weight) DESC;
Related
I am using SQL server and need to run the following SQL via Python script
SELECT DISTINCT LEN(Wav)-CHARINDEX('.', Wav) FROM <>;
I have tried to play with the String but couldn’t figure out how to work around the dot character.
sql = 'SELECT DISTINCT LEN(Wav)-CHARINDEX({}, Wav) FROM xxx'.format('.')
print(sql)
cursor = conn.cursor()
cursor.execute(sql)
Any idea how to resolve this
Thank you
'.' is the string ., you want "'.'", the string '.'
>>> print("{}".format('.'))
.
>>> print("{}".format("'.'"))
'.'
As #Justin Ezequiel's answer notes, do beware of SQL injections here!
Specifically, unfiltered user inputs can and will cause an SQL injection where unanticipated commands can be run against the target database by breaking out of the raw string. These can do anything your connection has permission to do, such as retrieving, modifying, or deleting arbitrary data.
A traditional approach is to use prepared statements
In Python, you can also use a regex or other test to explicitly error for statements with control characters (if not re.match(r"^[a-zA-Z\d _+-]+$"), s):raise_) or use (trust) an escaping library to do it for you if you must take arbitrary strings.
Use parameters to avoid SQL-injection attacks.
sql = 'SELECT DISTINCT LEN(Wav)-CHARINDEX(?, Wav) FROM xxx' # note placeholder (?)
print(sql)
params = ('.',) # tuple
cursor = conn.cursor()
cursor.execute(sql, params)
I'll admit it. I've been doing this for literally decades:
sql = 'select * from whatever where key = "%s"' % ('thing')
cursor.execute(sql)
One big reason is that
print(sql)
tells me something useful. Whereas this:
sql = 'select * from whatever where key = :key'
params = {'key': 'thing'}
cursor.execute(sql, params)
has no way to do that. Or does it? Note that this isn't as easy as doing this:
print(replace_with_dict_values(sql, params))
(replace_with_dict_values isn't real, but could easily be made.) Why not? Because in the above example, I would get this printed out:
select * from whatever where key = thing
which is missing the quoting. I want to know what the actual sql would be so I can cut and paste it into a sql editor and run the query myself while debugging. ("actual SQL" basically means "what the equivalent sql would be" and can probably be found by asking the cursor to preform the bindings and return the string. but how?)
I don't know how to make this SQL Injection work in SQLite. I'm using a function in Python that connects to a database and inserts a string.
I have "database.db" that has two tables: "feedback" and "users".
The feedback table has 1 column: message.
The users table has 2 columns: username and password.
def send_feedback(feedback):
conn = sqlite3.connect("database.db")
curs = conn.cursor()
curs.execute("INSERT INTO feedback VALUES ('%s')" % (feedback))
print(curs.fetchall())
conn.close()
I know that the execute function allows me to make a single query to the database, so I can't use ";" to
make multiple queries.
What I have tried, is to make the string look like this:
a') SELECT password FROM users --
feedback = "INSERT INTO feedback VALUES ('a') SELECT password FROM users --')"
But this gives me the following error:
sqlite3.OperationalError: near "SELECT": syntax error
So I've tried to use the UNION command:
a') UNION SELECT password FROM users --
feedback = "INSERT INTO feedback VALUES ('a') UNION SELECT password FROM users --')"
This one works but the fetchall function returns an empty list.
Most SQL injections result in nothing useful to the perpetrator, just a syntax error.
For example, pass the string "I'm not satisfied" to this feedback function and the extra ' character would cause the quotes to be imbalanced, and this would result in an error, causing the INSERT to fail.
sqlite3.OperationalError: near "m": syntax error
That's technically SQL injection. The content interpolated into the query has affected the syntax of the SQL statement. That's all. It doesn't necessarily result in a successful "Mission: Impossible" kind of infiltration.
I can't think of a way to exploit the INSERT statement you show to make it do something clever, besides causing an error.
You can't change an INSERT into a SELECT that produces a result set. Even if you try to inject a semicolon followed by a second SQL query, you just get sqlite3.Warning: You can only execute one statement at a time
Your first try above resulted in a syntax error because you had both a VALUES clause and a SELECT as a source for the data to insert. You can use either one but not both in SQL syntax. See https://www.sqlite.org/lang_insert.html
You probably already know how to make the code safe, so unsafe content cannot even cause a syntax error. But I'll include it for other readers:
curs.execute("INSERT INTO feedback VALUES (?)", (feedback,))
You can do it, for example to get table name
a' || (SELECT tbl_name FROM sqlite_master WHERE type='table' and tbl_name NOT like 'sqlite_%'))-- -
I am trying to use SQL with prepared statements in Python. Python doesn't have its own mechanism for this so I try to use SQL directly:
sql = "PREPARE stmt FROM ' INSERT INTO {} (date, time, tag, power) VALUES (?, ?, ?, ?)'".format(self.db_scan_table)
self.cursor.execute(sql)
Then later, in the loop:
sql = "EXECUTE stmt USING \'{}\', \'{}\', {}, {};".format(d, t, tag, power)
self.cursor.execute(sql)
And in the loop I get:
MySQL Error [1064]: You have an error in your SQL syntax; check the manual that corresponds to your MariaDB server version for the right syntax to use near ''2014-12-25', '12:31:46', 88000000, -6.64' at line 1
What's going on?
Using prepared statements with MySQL in Python is explained e.g at http://zetcode.com/db/mysqlpython/ -- look within that page for Prepared statements.
In your case, that would be, e.g:
sql = ('INSERT INTO {} (date, time, tag, power) VALUES '
'(%s, %s, %s, %s)'.format(self.db_scan_table))
and later, "in the loop" as you put it:
self.cursor.execute(sql, (d, t, tag, power))
with no further string formatting -- the MySQLdb module does the prepare and execute parts on your behalf (and may cache things to avoid repeating work needlessly, etc, etc).
Do consider, depending on the nature of "the loop" you mention, that it's possible that a single call to .execute_many (with a sequence of tuples as the second argument) could take the place of the whole loop (unless you need more processing within that loop beyond just the insertion of data into the DB).
Added: a better alternative nowadays may be to use mysql's own Connector/Python and the explicit prepare=True option in the .cursor() factory -- see http://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursorprepared.html . This lets you have a specific cursor on which statements are prepared (with the "more efficient than using PREPARE and EXECUTE" binary protocol, according to that mysql.com page) and another one for statements that are better not prepared; "explicit is better than implicit" is after all one of the principles in "The Zen of Python" (import this from an interactive prompt to read all those principles). mysqldb doing things implicitly (and it seems the current open-source version doesn't use prepared statements) can't be as good an architecture as Connector/Python's more explicit one.
import mysql.connector
db_con=mysql.connector.connect(host='',
database='',
user='',
password='')
cursor = db_con.cursor(prepared=True,)
#cursor = db_con.cursor(prepared=True)#IT MAY HAVE PROBLEM
sql = """INSERT INTO table (xy,zy) VALUES (%s, %s)"""
input=(1,2)
cursor.execute(sql , input)
db_con.commit()
SELECT STMT
sql = """SELECT * FROM TABLE WHERE XY=%s ORDER BY id DESC LIMIT 1 """
ID=1
input=(ID,)
#input=(ID)# IT MAY HAS PROBLEM
cursor.execute(sql, input)
data = cursor.fetchall()
rowsNumber=cursor.rowcount
Python does support prepared statements:
sql = "INSERT INTO {} (date, time, tag, power) VALUES (%s, %s, %s, %s);"
sql = sql.format(self.db_scan_table)
self.cursor.execute(sql, (d, t, tag, power))
(You should ensure self.db_scan_table is not vulnerable to SQL injection)
This assumes your paramstyle is 'format', which it should be for MySQL.
entrym='entry'
entrym=entrym+ str(idx)
cursor.execute("INSERT INTO im_entry.test ("+entrym+") VALUES ('"+p+"');")
I am using a query like this, where entry1, entry2 etc. are my database tables. The program doesn't show any errors, but the p value does not get inserted in the db. What is wrong here? Please help me.
By default, psycopg2 starts transactions for you automatically, which means that you have to tell it to commit. Note that commit is a method of the connection, not the cursor.
conn = psycopg2.connection('...')
cur = conn.cursor()
cur.execute("...")
conn.commit()
The intent is that you can group multiple statements together in a single transaction, so other queries won't see half-made changes, but also for performance reasons.
Also note that you should always use placeholders, instead of concatenating strings together.
E.g.:
cur.execute("INSERT INTO im_entry.test (colname) VALUES (%s)", [p])
Otherwise you risk making SQL injection attacks possible.