I'm hoping to create a build a query dynamically that I can populate through paramerters. E.g. Something like:
INSERT INTO Table (Col1, Col2, Col3, ...) VALUES (%s, %s, %s, ...)
Then I want to populate it like shown in
How to put parameterized sql query into variable and then execute in Python? :
sql = "INSERT INTO table VALUES (%s, %s, %s)"
args= var1, var2, var3
cursor.execute(sql, args)
Building the query string above should be pretty easy, however I'm not sure how to build the list of args. Once I can do that, the rest should be easy.
Looks like my question was easier than I thought, it looks like arrays behave like lists in python. The following gave me what I needed:
results = []
results.append("a")
results.append("b")
results.append("c")
results.append("d")
print results
I can use the above to make my arg list.
I typically use something like this to achieve what you describe:
>>> def prepquery(qrystr, args):
... # Assumes Qry Str has a %s in it instead of column placeholders.
... return qrystr % ', '.join('?' * len(args)) # ['%s'] * len(args), depending on DB.
...
>>>
>>> prepquery('Insert Into Table Values (%s)', range(5))
'Insert Into Table Values (?, ?, ?, ?, ?)'
Related
sql = """INSERT INTO (Product_details) VALUES (%s, %s, %s, %s, %s, %s)"""
val = [name, Spec ,Ratings ,Delivery ,Discount ,Price ] # list of data into list
for values in val: #loping the variables in the list and adding it to database
engine.execute(sql, values)
It looks like you're using Python - Try ? instead of %s - sometimes the parameter marker is not what you would expect it to be so do check which one you need to use for the language you're embedding the SQL in
I have used this code to use two variables in a single SQL code in Python :
cursor.execute("select * from customers WHERE username=%s and password=%s", (a, b))
but I've got this error :
MySQLInterfaceError: Python type tuple cannot be converted
though I've converted my strings into a tuple like this:
a = tuple(map(str, emaile.split(",")))
b = tuple(map(str, passe.split(",")))
how can I use these two variables in my cursor.execute code?
query = """select * from customers WHERE username=%s and password=%s"""
tuple1 = ("mini", 9000)
cursor.execute(query, tuple1)
cursor.execute("INSERT INTO table VALUES (%s, %s, %s)", (var1, var2, var3))
I'm trying to insert rows on a MySQL table using pymysql (Python 3), the relevant code is the following.
def saveLogs(DbConnection, tableName, results):
for row in results:
formatStrings = ",".join(["?"]*len(row))
sql = "INSERT INTO %s VALUES (%s);"%(tableName,formatStrings)
DbConnection.cursor().execute(sql, tuple(row))
DbConnection.commit()
I'm using "?" for the types, but I get the error not all arguments converted during string formatting. row is a list composed of strings, ints and datetime.datetime. I guess the issue is the "?" but I have checked the PEP 249 and it's still not clear to me how should I do it. Any suggestions?
Use string formatting for the table name only (though make sure you trust the source or have a proper validation in place). For everything else, use query parameters:
def saveLogs(DbConnection, tableName, results):
cursor = DbConnection.cursor()
sql = "INSERT INTO {0} VALUES (%s, %s, %s)".format(tableName)
for row in results:
cursor.execute(sql, row)
DbConnection.commit()
There is also that executemany() method:
def saveLogs(DbConnection, tableName, results):
cursor = DbConnection.cursor()
cursor.executemany("INSERT INTO {0} VALUES (%s, %s, %s)".format(tableName), results)
DbConnection.commit()
This might be a rather silly question but what am I doing wrong here? It creates the table but the INSERT INTO doesn't work, I guess I'm doing something wrong with the placeholders?
conn = psycopg2.connect("dbname=postgres user=postgres")
cur = conn.cursor()
escaped_name = "TOUR_2"
cur.execute('CREATE TABLE %s(id serial PRIMARY KEY, day date, elapsed_time varchar, net_time varchar, length float, average_speed float, geometry GEOMETRY);' % escaped_name)
cur.execute('INSERT INTO %s (day,elapsed_time, net_time, length, average_speed, geometry) VALUES (%s, %s, %s, %s, %s, %s)', (escaped_name, day ,time_length, time_length_net, length_km, avg_speed, myLine_ppy))
conn.commit()
cur.close()
conn.close()
The INSERT INTO call doesn't work, it gives me
cur.execute('INSERT INTO %s (day,elapsed_time, net_time, length, average_speed,
geometry) VALUES (%s, %s, %s, %s, %s, %s)'% (escaped_name, day ,time_length,
time_length_net, length_km, avg_speed, myLine_ppy))
psycopg2.ProgrammingError: syntax error at or near ":"
LINE 1: ...h, average_speed, geometry) VALUES (2013/09/01 , 2:56:59, 02...
Can someone help me on this one? Thanks a bunch!
You are using Python string formatting and this is a Very Bad Idea (TM). Think SQL-injection. The right way to do it is to use bound variables:
cur.execute('INSERT INTO %s (day, elapsed_time, net_time, length, average_speed, geometry) VALUES (%s, %s, %s, %s, %s, %s)', (escaped_name, day, time_length, time_length_net, length_km, avg_speed, myLine_ppy))
where the tuple of parameters is given as second argument to execute(). Also you don't need to escape any value, psycopg2 will do the escaping for you. In this particular case is also suggested to not pass the table name in a variable (escaped_name) but to embed it in the query string: psycopg2 doesn't know how to quote table and column names, only values.
See psycopg2 documentation:
https://www.psycopg.org/docs/usage.html#passing-parameters-to-sql-queries
If you want to programmatically generate the SQL statement, the customary way is to use Python formatting for the statement and variable binding for the arguments. For example, if you have the table name in escaped_name you can do:
query = "INSERT INTO %s (col1, ...) VALUES (%%s, ...)" % escaped_name
curs.execute(query, args_tuple)
Obviously, to use placeholders in your query you need to quote any % that introduce a bound argument in the first format.
Note that this is safe if and only if escaped_name is generated by your code ignoring any external input (for example a table base name and a counter) but it is at risk of SQL injection if you use data provided by the user.
To expand on #Matt's answer, placeholders do not work for identifiers like table names because the name will be quoted as a string value and result in invalid syntax.
If you want to generate such a query dynamically, you can use the referred to pyscopg2.sql module:
from psycopg2.sql import Identifier, SQL
cur.execute(SQL("INSERT INTO {} VALUES (%s)").format(Identifier('my_table')), (10,))
As of psycopg2 v2.7 there is a supported way to do this: see the psycopg2.sql docs.
I am trying to accomplish something like the following:
cursor = db.cursor()
cursor.execute('INSERT INTO media_files (%s, %s, %s, %s ... ) VALUES (%s, %s, %s, %s, ...)', (fieldlist, valuelist))
cursor.commit()
I have 2 lists, fieldlist and valuelist which each contain the same number of items. What is the best way to generate a dynamic MySQL query statement where the collumns are stored in fieldlist and the values are stored in valuelist?
cursor.execute('INSERT INTO media_files (%s) VALUES (%%s, %%s, %%s, %%s, ...)' % ','.join(fieldlist), valuelist)
To make it clearer:
sql = 'INSERT INTO media_files (%s) VALUES (%%s, %%s, %%s, %%s, ...)' % ','.join(fieldlist)
cursor.execute(sql, valuelist)
The cursor expects parameters to be passed as a single sequence, so you need to combine - in order - the field and value lists.
itertools.chain() does exactly that however it returns a generator and I'm not sure if cursor.execute() will accept that as it's param sequence. Try it. If it fails, wrap it with list()
import itertools
sql = 'INSERT INTO media_files (%s, %s, %s, %s ... ) VALUES (%s, %s, %s, %s, ...)'
cursor.execute(sql, itertools.chain(fieldlist, valuelist))
EDIT:
This solution will not work. This would cause the field names to be escaped and wrapped with quotes which would cause an sql syntax error.
I'll leave this answer as it might serve as a useful example but look to #Trent's answer for the solution.