I used MySQL Connector/Python API, NOT MySQLdb.
I need to dynamically insert values into a sparse table so I wrote the Python code like this:
cur.executemany("UPDATE myTABLE SET %s=%s WHERE id=%s" % data)
where
data=[('Depth', '17.5cm', Decimal('3003')), ('Input_Voltage', '110 V AC', Decimal('3004'))]
But it resulted an error:
TypeError: not enough arguments for format string
Is there any solution for this problem? Is it possible to use executemany when there is a
substitution of a field in query?
Thanks.
Let's start with the original method:
As the error message suggests you have a problem with your SQL syntax (not Python). If you insert your values you are effectively trying to execute
UPDATE myTABLE SET 'Depth'='17.5cm' WHERE id='3003'
You should notice that you are trying to assign a value to a string 'Depth', not a database field. The reason for this is that the %s substitution of the mysql module is only possible for values, not for tables/fields or other object identifiers.
In the second try you are not using the substitution anymore. Instead you use generic python string interpolation, which however looks similar. This does not work for you because you have a , and a pair of brackets too much in your code. It should read:
cur.execute("UPDATE myTABLE SET %s=%s WHERE id=%s" % data)
I also replaced executemany with execute because this method will work only for a single row. However your example only has one row, so there is no need to use executemany anyway.
The second method has some drawbacks however. The substitution is not guaranteed to be quoted or formatted in a correct manner for the SQL query, which might cause unexpected behaviour for certain inputs and may be a security concern.
I would rather ask, why it is necessary to provide the field name dynamically in the first place. This should not be necessary and might cause some trouble.
Related
Pretty new to sqlite3, so bear with me here..
I'd like to have a function to which I can pass the table name, and the values to update.
I initially started with something like this:
def add_to_table(table_name, string):
cursor.execute('INSERT INTO {table} VALUES ({var})'
.format(
table=table_name,
var=string)
)
Which works A-OK, but further reading about sqlite3 suggested that this was a terribly insecure way to go about things. However, using their ? syntax, I'm unable to pass in a name to specify the variable.
I tried adding in a ? in place of the table, but that throws a syntax error.
cursor.execute('INSERT INTO ? VALUES (?)', ('mytable','"Jello, world!"'))
>> >sqlite3.OperationalError: near "?": syntax error
Can the table in an sql statement be passed in safely and dynamically?
Its not the dynamic string substitution per-se thats the problem. Its dynamic string substitution with an user-supplied string thats the big problem because that opens you to SQL-injection attacks. If you are absolutely 100% sure that the tablename is a safe string that you control then splicing it into the SQL query will be safe.
if some_condition():
table_name = 'TABLE_A'
else:
table_name = 'TABLE_B'
cursor.execute('INSERT INTO '+ table_name + 'VALUES (?)', values)
That said, using dynamic SQL like that is certainly a code smell so you should double check to see if you can find a simpler alternative without the dynamically generated SQL strings. Additionally, if you really want dynamic SQL then something like SQLAlchemy might be useful to guarantee that the SQL you generate is well formed.
Composing SQL statements using string manipulation is odd not only because of security implications, but also because strings are "dumb" objects. Using sqlalchemy core (you don't even need the ORM part) is almost like using strings, but each fragment will be a lot smarter and allow for easier composition. Take a look at the sqlalchemy wiki to get a notion of what I'm talking about.
For example, using sqlsoup your code would look like this:
db = SQLSoup('sqlite://yourdatabase')
table = getattr(db, tablename)
table.insert(fieldname='value', otherfield=123)
db.commit()
Another advantage: code is database independent - want to move to oracle? Change the connection string and you are done.
I'm having this function that communicates via pymysql to an SQL database stored to my localhost. I know there are similar posts about formatting an SQL section especially this one but could anyone suggest a solution?
Always getting TypeError: can't concat tuple to bytes. I suppose it's sth with the WHERE clause.
def likeMovement(pID):
print("Give a rating for the movement with #id:%s" %pID)
rate=input("Give from 0-5: ")
userID=str(1)
print(rate,type(rate))
print(pID,type(pID))
print(userID,type(userID))
cursor=con.cursor()
sqlquery='''UDPATE likesartmovement SET likesartmovement.rating=%s WHERE
likesartmovement.artisticID=? AND likesartmovement.userID=?''' % (rate,),
(pID,userID)
cursor.execute(sqlquery)
TypeError: not all arguments converted during string formatting
Thanks in advance!
The problem is that you're storing (pID,userID) as part of a tuple stored in sqlquery, instead of passing them as the arguments to execute:
sqlquery='''UDPATE likesartmovement SET likesartmovement.rating=%s WHERE
likesartmovement.artisticID=? AND likesartmovement.userID=?''' % (rate,)
cursor.execute(sqlquery, (pID,userID))
It may be clearer to see why these are different if you take a simpler example:
s = 'abc'
spam(s, 2)
s = 'abc', 2
spam(s)
Obviously those two don't do the same thing.
While we're at it:
You have to spell UPDATE right.
You usually want to use query parameters for SET clauses for exactly the same reasons you want to for WHERE clauses.
You don't need to include the table name in single-table operations, and you're not allowed to include the table name in SET clauses in single-table updates.
So:
sqlquery='''UDPATE likesartmovement SET rating=? WHERE
artisticID=? AND userID=?'''
cursor.execute(sqlquery, (rating, pID, userID))
I'm trying to write an SQL query in PyQt5 that updates some data in a table, but cannot get the query to work. I've read countless forums but as far as I can tell my code is correct. I also have read the documentation back to front so maybe I'm missing something?
I am using PyQt5, python3.5 and SQLITE. The following code (lastError/lastQuery not shown):
self.sqlWrite('ct','MarkerSize',123)
def sqlWrite(self,tbl,var,val):
query = QtSql.QSqlQuery(self.db) # First create query instance.
# Prepare query with placeholders, then bind values.
query.prepare('UPDATE :tbl SET value=:val WHERE property=:var')
query.bindValue(0,tbl)
query.bindValue(1,val)
query.bindValue(2,var)
# Finally execute query.
query.exec_()
...produces the error:
near "?": syntax error Unable to execute statement
near "?": syntax error Unable to execute statement
UPDATE :tbl SET value=:val WHERE property=:var
Parameter count mismatch
Have I lost the plot? What am I missing?
Thanks in advance.
A table name is not a parameter, so you cannot bind a value to it. Placeholders are intended for use with literal values, not arbitrary strings. For the latter, you should just use normal string interpolation:
query.prepare('UPDATE "%s" SET value=:val WHERE property=:var' % tbl)
query.bindValue(':val', val)
query.bindValue(':var', var)
For a more generic way to escape identifiers, use the query's driver:
tbl = query.driver().escapeIdentifier(tbl, QSqlDriver.TableName)
query.prepare('UPDATE %s SET value=:val WHERE property=:var' % tbl)
Pretty new to sqlite3, so bear with me here..
I'd like to have a function to which I can pass the table name, and the values to update.
I initially started with something like this:
def add_to_table(table_name, string):
cursor.execute('INSERT INTO {table} VALUES ({var})'
.format(
table=table_name,
var=string)
)
Which works A-OK, but further reading about sqlite3 suggested that this was a terribly insecure way to go about things. However, using their ? syntax, I'm unable to pass in a name to specify the variable.
I tried adding in a ? in place of the table, but that throws a syntax error.
cursor.execute('INSERT INTO ? VALUES (?)', ('mytable','"Jello, world!"'))
>> >sqlite3.OperationalError: near "?": syntax error
Can the table in an sql statement be passed in safely and dynamically?
Its not the dynamic string substitution per-se thats the problem. Its dynamic string substitution with an user-supplied string thats the big problem because that opens you to SQL-injection attacks. If you are absolutely 100% sure that the tablename is a safe string that you control then splicing it into the SQL query will be safe.
if some_condition():
table_name = 'TABLE_A'
else:
table_name = 'TABLE_B'
cursor.execute('INSERT INTO '+ table_name + 'VALUES (?)', values)
That said, using dynamic SQL like that is certainly a code smell so you should double check to see if you can find a simpler alternative without the dynamically generated SQL strings. Additionally, if you really want dynamic SQL then something like SQLAlchemy might be useful to guarantee that the SQL you generate is well formed.
Composing SQL statements using string manipulation is odd not only because of security implications, but also because strings are "dumb" objects. Using sqlalchemy core (you don't even need the ORM part) is almost like using strings, but each fragment will be a lot smarter and allow for easier composition. Take a look at the sqlalchemy wiki to get a notion of what I'm talking about.
For example, using sqlsoup your code would look like this:
db = SQLSoup('sqlite://yourdatabase')
table = getattr(db, tablename)
table.insert(fieldname='value', otherfield=123)
db.commit()
Another advantage: code is database independent - want to move to oracle? Change the connection string and you are done.
I am currently trying to use place holders in my PostgreSQL query within Python's psycopg's module. Here is a sample of the code I am using.
table.execute('SELECT * FROM table WHERE col2 = %s ORDER BY pID ASC LIMIT %s OFFSET %s;',(val1,val2,val3))
I read somewhere that it is not possible to use placeholders like this for LIMIT and OFFSET however I should use this placeholder format for WHERE =.
safely specifying 'order by' clause from user input in python / postgresql / psycopg2
Does anyone know the proper placeholder syntax for this sql query? Thanks!
Limit and offset can both be used with placeholders without any issue.
Generally speaking you can use placeholders wherever a 'value' would be allowed in an expression.
cur.execute("select * from node where node_name = %s limit %s offset %s", ('test', 5, 5))
Works just fine.
As already noted in the referenced article you cannot use placeholders to refer to tables, columns, schemas, or aliases for any of them. In those cases you generally need to do your own variable substitution before calling execute.
In very old versions of PostgreSQL, it was indeed not possible to use placeholders in LIMIT and OFFSET clauses. This functionality was added in version 7.4, so it is safe to assume that it exists in current installations.
But that only applies to server-side prepared statements. Psycopg does not use server-side prepared statements. It does its own string substitution and sends the resulting string to the backend as a constant. So in principle, you can use its parameter substitution feature anywhere there resulting literal would be syntactically valid.
So what you are proposing to do is fine either way.