I have an query string in Python as follows:
query = "select name from company where id = 13 order by name;"
I want to be able to change the id dynamically. Thus I want to find id = 13 and replace it with a new id.
I can do it as follows:
query.replace("id = 13", "id = {}".format(some_new_id))
But if in the query is id= 13 or id=13 or id =13, ... it will not work.
How to avoid that?
Gluing variables directly into your query leaves you vulnerable to SQL injection.
If you are passing your query to a function to be executed in your database, that function should accept additional parameters.
For instance,
query = "select name from company where id = %s order by name"
cursor.execute(query, params=(some_other_id,))
It is better to use formatted sql.
Ex:
query = "select name from company where id = %s order by name;".
cursor.execute(query, (id,))
The usual solution when it comes to dynamically building strings is string formatting, ie
tpl = "Hello {name}, how are you"
for name in ("little", "bobby", "table"):
print(tpl.format(name))
BUT (and that's a BIG "but"): you do NOT want to do this for SQL queries (assuming you want to pass this query to your db using your db's python api).
There are two reasons to not use string formatting here: the first one is that correctly handling quoting and escaping is tricky at best, the second and much more important one is that it makes your code vulnerable to SQL injections attacks.
So in this case, the proper solution is to use prepared statements instead:
# assuming MySQL which uses "%" as placeholder,
# consult your db-api module's documentation for
# the proper placeholder
sql = "select name from company where id=%s order by name"
cursor = yourdbconnection.cursor()
cursor.execute(sql, [your_id_here])
Related
im new in programing, starting with python
I found this script in a post (link below)
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
How to use variable for SQLite table name
it worked, but now i have to add some complexity and dont know how...
i had this befour:
def add(order, theme):
parameters = [order, theme]
c.execute("INSERT INTO TABLE_NAME VALUES(NULL, ?,?)", parameters)
so im trying have the same, but with de name of the table free to choose.
trying things like this, but dont really know the syntax for it:
def add(order, theme):
table = input("with what table do you want to work with? ")
parameters = [order, theme]
insert = 'INSERT INTO {} VALUES(NULL, ?,?)'.format(table, parameters)
c.execute(insert)
i suspected it woudnt work Hope you can help! :)
The following line substitutes {} with the contents of the variable table and stores the result in the variable query. format works with any string:
query = 'SELECT * FROM {}'.format(table)
On the other hand the kind of substitution done by c.execute replaces the ? in the string with the values in the list parameters:
c.execute("INSERT INTO TABLE_NAME VALUES(NULL, ?,?)", parameters)
You could combine them both to achieve the effect you want:
table = input("with what table do you want to work with? ")
query = 'INSERT INTO {} VALUES(NULL, ?, ?)'.format(table)
parameters = [order, theme]
c.execute(query, parameters)
You are mixing things.
Format is a method of Python string object used to generate dynamically your string based on variables from your code. In your first example is correct to define the name of the database.
Questions marks as a placeholder for the values of your query is proper of SQL and must be used without format as you did in the second example.
To fix your code, first build the string template of the query using format to define the databasename. Then, execute the resultant string with execute and passing the arguments targeting the question marks.
i wondering if it's possible to make a query with select with the user give the field of table and the value that want. For example:
field=input("Field: ")
value=input("Value: ")
cursorobject.execute('SELECT id FROM users WHERE {}=\'{}\'')
result=cursorobject.fetchall()
for x in result:
print(x)
and if it's not possible , there is any way to do it?
PS: this one not working
Of course you can construct the text of your query as you want using variables. E.g.
query = 'SELECT id FROM users WHERE {}=\'{}\''
print(query.format(field,value))
But, have in mind that you should validate very well the contents of the variables, before executing the query, to avoid SQL injections. For example the contents of the variables should not contain quotes.
E.g. the below code, with the specific values of the variables, will return the full list of users:
field='name'
value='name\' or \'1\'=\'1'
query = 'SELECT id FROM users WHERE {}=\'{}\''
print(query.format(field,value))
The produced query would be:
SELECT id FROM users WHERE name='name' or '1'='1'
Following your edit, you should replace your 3rd line with:
cursorobject.execute('SELECT id FROM users WHERE {}=\'{}\''.format(field,value))
And for making the best to avoid sql injections, you should use the built-in query parameterization features of your framework - pymysql:
cursorobject.execute('SELECT id FROM users WHERE {}=%s'.format(field),(value))
Simply format the query for field and pass value as a parameter in second argument of cursor.execute which must receive an iterable (i.e., tuple/list):
# PREPARED STATEMENT
sql = 'SELECT id FROM users WHERE {} = %s'
# EXECUTE QUERY
cursorobject.execute(sql.format(field), (value,))
result = cursorobject.fetchall()
My objective is to store a JSON object into a MySQL database field of type json, using the mysql.connector library.
import mysql.connector
import json
jsonData = json.dumps(origin_of_jsonData)
cnx = mysql.connector.connect(**config_defined_elsewhere)
cursor = cnx.cursor()
cursor.execute('CREATE DATABASE dataBase')
cnx.database = 'dataBase'
cursor = cnx.cursor()
cursor.execute('CREATE TABLE table (id_field INT NOT NULL, json_data_field JSON NOT NULL, PRIMARY KEY (id_field))')
Now, the code below WORKS just fine, the focus of my question is the use of '%s':
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES (%s, %s)"
values_to_insert = (1, jsonData)
cursor.execute(insert_statement, values_to_insert)
My problem with that: I am very strictly adhering to the use of '...{}'.format(aValue) (or f'...{aValue}') when combining variable aValue(s) into a string, thus avoiding the use of %s (whatever my reasons for that, let's not debate them here - but it is how I would like to keep it wherever possible, hence my question).
In any case, I am simply unable, whichever way I try, to create something that stores the jsonData into the mySql dataBase using something that resembles the above structure and uses '...{}'.format() (in whatever shape or form) instead of %s. For example, I have (among many iterations) tried
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES ({}, {})".format(1, jsonData)
cursor.execute(insert_statement)
but no matter how I turn and twist it, I keep getting the following error:
ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[some_content_from_jsonData})]' at line 1
Now my question(s):
1) Is there a way to avoid the use of %s here that I am missing?
2) If not, why? What is it that makes this impossible? Is it the cursor.execute() function, or is it the fact that it is a JSON object, or is it something completely different? Shouldn't {}.format() be able to do everything that %s could do, and more?
First of all: NEVER DIRECTLY INSERT YOUR DATA INTO YOUR QUERY STRING!
Using %s in a MySQL query string is not the same as using it in a python string.
In python, you just format the string and 'hello %s!' % 'world' becomes 'hello world!'. In SQL, the %s signals parameter insertion. This sends your query and data to the server separately. You are also not bound to this syntax. The python DB-API specification specifies more styles for this: DB-API parameter styles (PEP 249). This has several advantages over inserting your data directly into the query string:
Prevents SQL injection
Say you have a query to authenticate users by password. You would do that with the following query (of course you would normally salt and hash the password, but that is not the topic of this question):
SELECT 1 FROM users WHERE username='foo' AND password='bar'
The naive way to construct this query would be:
"SELECT 1 FROM users WHERE username='{}' AND password='{}'".format(username, password)
However, what would happen if someone inputs ' OR 1=1 as password. The formatted query would then become
SELECT 1 FROM users WHERE username='foo' AND password='' OR 1=1
which will allways return 1. When using parameter insertion:
execute('SELECT 1 FROM users WHERE username=%s AND password=%s', username, password)
this will never happen, as the query will be interpreted by the server separately.
Performance
If you run the same query many times with different data, the performance difference between using a formatted query and parameter insertion can be significant. With parameter insertion, the server only has to compile the query once (as it is the same every time) and execute it with different data, but with string formatting, it will have to compile it over and over again.
In addition to what was said above, I would like to add some details that I did not immediately understand, and that other (newbies like me ;)) may also find helpful:
1) "parameter insertion" is meant for only for values, it will not work for table names, column names, etc. - for those, the Python string substitution works fine in the sql syntax defintion
2) the cursor.execute function requires a tuple to work (as specified here, albeit not immediately clear, at least to me: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html)
EXAMPLE for both in one function:
def checkIfRecordExists(column, table, condition_name, condition_value):
...
sqlSyntax = 'SELECT {} FROM {} WHERE {} = %s'.format(column, table, condition_name)
cursor.execute(sqlSyntax, (condition_value,))
Note both the use of .format in the initial sql syntax definition and the use of (condition_value,) in the execute function.
Question: Is it possible to use a variable as your table name without having to use string constructors to do so?
Info:
I'm working on a project right now that catalogs data from a star simulation of mine. To do so I'm loading all the data into a sqlite database. It's working pretty well, but I've decided to add a lot more flexibility, efficiency, and usability to my db. I plan on later adding planetoids to the simulation, and wanted to have a table for each star. This way I wouldn't have to query a table of 20m some planetoids for the 1-4k in each solar system.
I've been told using string constructors is bad because it leaves me vulnerable to a SQL injection attack. While that isn't a big deal here as I'm the only person with access to these dbs, I would like to follow best practices. And also this way if I do a project with a similar situation where it is open to the public, I know what to do.
Currently I'm doing this:
cursor.execute("CREATE TABLE StarFrame"+self.name+" (etc etc)")
This works, but I would like to do something more like:
cursor.execute("CREATE TABLE StarFrame(?) (etc etc)",self.name)
though I understand that this would probably be impossible. though I would settle for something like
cursor.execute("CREATE TABLE (?) (etc etc)",self.name)
If this is not at all possible, I'll accept that answer, but if anyone knows a way to do this, do tell. :)
I'm coding in python.
Unfortunately, tables can't be the target of parameter substitution (I didn't find any definitive source, but I have seen it on a few web forums).
If you are worried about injection (you probably should be), you can write a function that cleans the string before passing it. Since you are looking for just a table name, you should be safe just accepting alphanumerics, stripping out all punctuation, such as )(][;, and whitespace. Basically, just keep A-Z a-z 0-9.
def scrub(table_name):
return ''.join( chr for chr in table_name if chr.isalnum() )
scrub('); drop tables --') # returns 'droptables'
For people searching for a way to make the table as a variable, I got this from another reply to same question here:
It said the following and it works. It's all quoted from mhawke:
You can't use parameter substitution for the table name. You need to add the table name to the query string yourself. Something like this:
query = 'SELECT * FROM {}'.format(table)
c.execute(query)
One thing to be mindful of is the source of the value for the table name. If that comes from an untrusted source, e.g. a user, then you need to validate the table name to avoid potential SQL injection attacks. One way might be to construct a parameterised query that looks up the table name from the DB catalogue:
import sqlite3
def exists_table(db, name):
query = "SELECT 1 FROM sqlite_master WHERE type='table' and name = ?"
return db.execute(query, (name,)).fetchone() is not None
I wouldn't separate the data into more than one table. If you create an index on the star column, you won't have any problem efficiently accessing the data.
Try with string formatting:
sql_cmd = '''CREATE TABLE {}(id, column1, column2, column2)'''.format(
'table_name')
db.execute(sql_cmd)
Replace 'table_name' with your desire.
To avoid hard-coding table names, I've used:
table = "sometable"
c = conn.cursor()
c.execute('''CREATE TABLE IF NOT EXISTS {} (
importantdate DATE,
somename VARCHAR,
)'''.format(table))
c.execute('''INSERT INTO {} VALUES (?, ?)'''.format(table),
(datetime.strftime(datetime.today(), "%Y-%m-%d"),
myname))
As has been said in the other answers, "tables can't be the target of parameter substitution" but if you find yourself in a bind where you have no option, here is a method of testing if the table name supplied is valid.
Note: I have made the table name a real pig in an attempt to cover all of the bases.
import sys
import sqlite3
def delim(s):
delims="\"'`"
use_delim = []
for d in delims:
if d not in s:
use_delim.append(d)
return use_delim
db_name = "some.db"
db = sqlite3.connect(db_name)
mycursor = db.cursor()
table = 'so""m ][ `etable'
delimiters = delim(table)
if len(delimiters) < 1:
print "The name of the database will not allow this!"
sys.exit()
use_delimiter = delimiters[0]
print "Using delimiter ", use_delimiter
mycursor.execute('SELECT name FROM sqlite_master where (name = ?)', [table])
row = mycursor.fetchall()
valid_table = False
if row:
print (table,"table name verified")
valid_table = True
else:
print (table,"Table name not in database", db_name)
if valid_table:
try:
mycursor.execute('insert into ' +use_delimiter+ table +use_delimiter+ ' (my_data,my_column_name) values (?,?) ',(1,"Name"));
db.commit()
except Exception as e:
print "Error:", str(e)
try:
mycursor.execute('UPDATE ' +use_delimiter+ table +use_delimiter+ ' set my_column_name = ? where my_data = ?', ["ReNamed",1])
db.commit()
except Exception as e:
print "Error:", str(e)
db.close()
you can use something like this
conn = sqlite3.connect()
createTable = '''CREATE TABLE %s (# );''' %dateNow)
conn.execute(createTable)
basically, if we want to separate the data into several tables according to the date right now, for example, you want to monitor a system based on the date.
createTable = '''CREATE TABLE %s (# );''' %dateNow) means that you create a table with variable dateNow which according to your coding language, you can define dateNow as a variable to retrieve the current date from your coding language.
You can save your query in a .sql or txt file and use the open().replace() method to use variables in any part of your query. Long time reader but first time poster so I apologize if anything is off here.
```SQL in yoursql.sql```
Sel *
From yourdbschema.tablenm
```SQL to run```
tablenm = 'yourtablename'
cur = connect.cursor()
query = cur.execute(open(file = yoursql.sql).read().replace('tablenm',tablenm))
You can pass a string as the SQL command:
import sqlite3
conn = sqlite3.connect('db.db')
c = conn.cursor()
tablename, field_data = 'some_table','some_data'
query = 'SELECT * FROM '+tablename+' WHERE column1=\"'+field_data+"\""
c.execute(query)
I've been using this syntax with great success in python mysql.
search = "O'%" # find names like O'Brien or O'Connell...
cursor.execute ("""
select userid
from usertab
where name like %s
""" , (search))
But sometimes I need to build my sql string before I execute it like the following, but the substitution technique is different than above and doesn't work in all cases.
search = "O'%" # find names like O'Brien or O'Connell...
sql = """
select userid
from usertab
where name like '%s'
""" % (search)
cursor.execute(sql)
How can I achieve the same kind of string substitution that works well in the first example, without executing the cursor?
MySQLdb uses the connection's literal() method to escape the arguments, so you could use:
sql = """
select userid
from usertab
where name like %s
""" % cursor.connection.literal(search)