I am trying to accomplish something like the following:
cursor = db.cursor()
cursor.execute('INSERT INTO media_files (%s, %s, %s, %s ... ) VALUES (%s, %s, %s, %s, ...)', (fieldlist, valuelist))
cursor.commit()
I have 2 lists, fieldlist and valuelist which each contain the same number of items. What is the best way to generate a dynamic MySQL query statement where the collumns are stored in fieldlist and the values are stored in valuelist?
cursor.execute('INSERT INTO media_files (%s) VALUES (%%s, %%s, %%s, %%s, ...)' % ','.join(fieldlist), valuelist)
To make it clearer:
sql = 'INSERT INTO media_files (%s) VALUES (%%s, %%s, %%s, %%s, ...)' % ','.join(fieldlist)
cursor.execute(sql, valuelist)
The cursor expects parameters to be passed as a single sequence, so you need to combine - in order - the field and value lists.
itertools.chain() does exactly that however it returns a generator and I'm not sure if cursor.execute() will accept that as it's param sequence. Try it. If it fails, wrap it with list()
import itertools
sql = 'INSERT INTO media_files (%s, %s, %s, %s ... ) VALUES (%s, %s, %s, %s, ...)'
cursor.execute(sql, itertools.chain(fieldlist, valuelist))
EDIT:
This solution will not work. This would cause the field names to be escaped and wrapped with quotes which would cause an sql syntax error.
I'll leave this answer as it might serve as a useful example but look to #Trent's answer for the solution.
Related
sql = """INSERT INTO (Product_details) VALUES (%s, %s, %s, %s, %s, %s)"""
val = [name, Spec ,Ratings ,Delivery ,Discount ,Price ] # list of data into list
for values in val: #loping the variables in the list and adding it to database
engine.execute(sql, values)
It looks like you're using Python - Try ? instead of %s - sometimes the parameter marker is not what you would expect it to be so do check which one you need to use for the language you're embedding the SQL in
I have the following values pulled from a csv file:
vars = (
'001U0000016lf5jIAA',
'Bam Margera Presents: Where the #$&% is Santa? (Unrated)',
'a0KU000000JMpgzMAD'
)
How would I insert these? This is what I'm currently doing but it's giving a SQL error probably because of the unescaped % in the second var --
cursor.execute('''INSERT INTO mytable VALUES (%s, %s, %s)''', vars)
TypeError: not all arguments converted during string formatting
Note: I'd rather keep the vars as-is (without trying to escape anything within it and just modify the SQL/python statement.
Try this
query = '''INSERT INTO mytable VALUES (%s, %s, %s)''', vars
cursor.execute(db.escape_string(query))
or
escaped = tuple(db.escape_string(x) for x in vars)
cursor.execute('''INSERT INTO mytable VALUES (%s, %s, %s)''', escaped)
I'm working on a DB and I'm having trouble when using pymysql to INSERT some values
cur.execute("""INSERT INTO orders (name, size, type, is_done) VALUES (%s, %s, %s, %s)"""
% (name, size, type, is_done))
Where name, size and type are strings and is_done is a bool
It gives me the typical error You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near, so I suppose the problem is a ', but how can I solve it?
Edit
I should also add that the name value is retrieved from a MySQL DB
The current accepted solution has a SQL injection vulnerability. You are not supposed to format the string with the % operator - just pass the tuple of arguments as a second argument, and the library will deal with the rest.
cur.execute("INSERT INTO orders (name, size, type, is_done) VALUES (%s, %s, %s, %s)",
(name, size, type, is_done))
Also see this answer and pymysql documentation.
I have found the problem, which was that instead of
cur.execute("""INSERT INTO orders (name, size, type, is_done)
VALUES (%s, %s, %s, %s)"""
% (name, size, type, is_done))
I should have done
cur.execute("""INSERT INTO orders (name, size, type, is_done)
VALUES ("%s", "%s", "%s", "%s")"""
% (name, size, type, is_done))
if you don't input value for id. You have an error. Try this query.
cur.execute("insert into orders values(%s, %s, %s, %s, %s)", (None, name, size, type, is_done))
"%s" and "None" for id column. This query running my code.
Note: Don't forget commit()
I have the following query in python-mysql:
self.cursor.execute('SELECT apple_id FROM main_catalog WHERE apple_id=%s', apple_id)
if not self.cursor.fetchone():
cursor.execute('''INSERT INTO main_catalog
(apple_id, provider_id, show_title, artwork_apple_url, title,
itunes_url, network_name, length, episode_production_number, synopsis)
VALUES (%s, %s, %s, %s, %s, %s, %s, %s, %s, %s)''',
apple_id, provider_obj.pk, show_title, artwork_apple_url, episode_title,
itunes_url, network_name, length, episode_production_number, synopsis)
This seems incredible verbose. Is there a way to write this more compactly and easier to read/understand?
I suggest you use pytho-sql or sqlpuzzle to be avoid of strings containing log SQL statements
catalog = Table('main_catalog')
catalog.insert(columns=[
catalog.apple_id, user.provide_id],
values=[['Foo', 'foo'], ['Bar', 'bar']])
)
Using this library you can build the query from a dict:
columns = [Column(catalog, col) for col in data.keys()]
sql = catalog.insert(
columns=columns,
values=[data.values()]
)
Well, so far there's no logical operations you can minimize into this peace of code. It's basically a query which inserts values into a table which has many fields. I doubt there's something you can do about it.
This might be a rather silly question but what am I doing wrong here? It creates the table but the INSERT INTO doesn't work, I guess I'm doing something wrong with the placeholders?
conn = psycopg2.connect("dbname=postgres user=postgres")
cur = conn.cursor()
escaped_name = "TOUR_2"
cur.execute('CREATE TABLE %s(id serial PRIMARY KEY, day date, elapsed_time varchar, net_time varchar, length float, average_speed float, geometry GEOMETRY);' % escaped_name)
cur.execute('INSERT INTO %s (day,elapsed_time, net_time, length, average_speed, geometry) VALUES (%s, %s, %s, %s, %s, %s)', (escaped_name, day ,time_length, time_length_net, length_km, avg_speed, myLine_ppy))
conn.commit()
cur.close()
conn.close()
The INSERT INTO call doesn't work, it gives me
cur.execute('INSERT INTO %s (day,elapsed_time, net_time, length, average_speed,
geometry) VALUES (%s, %s, %s, %s, %s, %s)'% (escaped_name, day ,time_length,
time_length_net, length_km, avg_speed, myLine_ppy))
psycopg2.ProgrammingError: syntax error at or near ":"
LINE 1: ...h, average_speed, geometry) VALUES (2013/09/01 , 2:56:59, 02...
Can someone help me on this one? Thanks a bunch!
You are using Python string formatting and this is a Very Bad Idea (TM). Think SQL-injection. The right way to do it is to use bound variables:
cur.execute('INSERT INTO %s (day, elapsed_time, net_time, length, average_speed, geometry) VALUES (%s, %s, %s, %s, %s, %s)', (escaped_name, day, time_length, time_length_net, length_km, avg_speed, myLine_ppy))
where the tuple of parameters is given as second argument to execute(). Also you don't need to escape any value, psycopg2 will do the escaping for you. In this particular case is also suggested to not pass the table name in a variable (escaped_name) but to embed it in the query string: psycopg2 doesn't know how to quote table and column names, only values.
See psycopg2 documentation:
https://www.psycopg.org/docs/usage.html#passing-parameters-to-sql-queries
If you want to programmatically generate the SQL statement, the customary way is to use Python formatting for the statement and variable binding for the arguments. For example, if you have the table name in escaped_name you can do:
query = "INSERT INTO %s (col1, ...) VALUES (%%s, ...)" % escaped_name
curs.execute(query, args_tuple)
Obviously, to use placeholders in your query you need to quote any % that introduce a bound argument in the first format.
Note that this is safe if and only if escaped_name is generated by your code ignoring any external input (for example a table base name and a counter) but it is at risk of SQL injection if you use data provided by the user.
To expand on #Matt's answer, placeholders do not work for identifiers like table names because the name will be quoted as a string value and result in invalid syntax.
If you want to generate such a query dynamically, you can use the referred to pyscopg2.sql module:
from psycopg2.sql import Identifier, SQL
cur.execute(SQL("INSERT INTO {} VALUES (%s)").format(Identifier('my_table')), (10,))
As of psycopg2 v2.7 there is a supported way to do this: see the psycopg2.sql docs.