Develop Reference

How to do I use %s in a mysql query without errors

I am having some trouble selecting from my database using python to execute a MySql query. I have tried two methods to achieve this, but both methods have returned the error shown below:
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '%s' at line 1
What Id like to do is return the row count (which is always zero or one) when a username parameter is passed. I have looked at other examples where people have had this issue but I cant find a good fix.
The first method I tried was this:
def check_data(username):
sql = """SELECT count(*) FROM tbl_user WHERE username = %s"""
mycursor.execute(sql, username)
#do something with the data
I then tried using SELECT (CASE WHEN (uname = %s) THEN TRUE ELSE FALSE END) AS IsEmtpy FROM tbl_user limit 1;
This works database side, but still throws the same error when run in the application. I tried wrapping the %s like '%s' but it didn't help.
Any suggestions?

You're missing enclosing the string between quotes (singles or doubles).
You can check the query you're executing by printing it before the mycursor.execute statement, but basically you're sending MySQL something like SELECT count(*) FROM tbl_user WHERE username = foobar.
Try fixing it with SELECT count(*) FROM tbl_user WHERE username = '%s'.
On a side note, your approach is vulnerable to SQL Injection. You should check the documentation of the tool you're using to connect to the DBMS for "prepared statements".

SQL Server variables not working

I have a script with the following:
UPDATE table
SET column to update = ?
WHERE condition = ?", "text to insert", "text to test condition"
For some reason SQL is not executing or even reading this line. When I misspell any of the reserved words or column names I do not get an error.
HOWEVER, when I have
UPDATE table
SET column to update = "text to insert"
WHERE Name = "text to test condition"
SQL behaves as expected.
The problem is the second method,which works, is not adequate for my needs. Am I missing something?
Thanks

Since this is tagged with pyodbc, I'm assuming you're trying to do run a query with parameters. Your code should probably read something like this:
pyodbc.execute(
"""
UPDATE table
SET column_to_update = ?
WHERE other_column = ?
""",
"text to put in column_to_update",
"text to test condition in other_column",
)
Please note that parameters marked with a ? must be tied to a data typed object such as a column, so they can be bound. See:
https://github.com/mkleehammer/pyodbc/wiki/Getting-started#parameters
Good luck!

I'm going to assume that you are trying to run a SQL query from some client code by passing in variables with the query, though I'm not sure of this or what language you might be using - please clarify and add tags so we can help.
SQL Server does not use ? for parameter placeholders like in other DBMS queries (say, SQLite which uses ? in the way you are trying to use them). Instead, you need to either let an ORM tool declare variables for you, and use those, or explicitly declare your own parameters and pass them in. You haven't told us the development environment you're using or what method of SQL connection and ORM (if any) you're using, but here's a quick example using the very excellent Dapper ORM in C# from here, given an open connection conn:
string val = "my value";
conn.Execute("insert MyTable (val) values(#val)", new {val});
// or: conn.Execute("insert MyTable (val) values(#val)", new {val = val});"
conn.Execute("update MyTable set val = #val where Id = #id", new {val, id = 1});
In the background, Dapper handles the mapping and creation of variables, such that the actual SQL script received by SQL Server is something closer to this:
-- first query:
declare #val nvarchar(max) = 'my value';
insert MyTable (val) values(#val);
-- second query:
declare #val nvarchar(max) = 'my value', #id int = 1;
update MyTable set val = #val where Id = #id
On the other hand, if you are just going to execute a raw query directly with a SqlConnection, try something like this (equivalent to the first query above):
// Assumes an open connection conn
string val = "my value";
using (SqlCommand cmd = conn.CreateCommand())
{
cmd.CommandText = "insert MyTable (val) values(#val)";
cmd.Parameters.AddWithValue("#val", val); // This creates the #val declaration for you when the query is executed
cmd.ExecuteNonQuery();
}
Whatever you do, parameterize your parameters, and beware of SQL injection!
Hope that helps. If you'd like a clearer example, please give us some code to show how you're passing the query to the SQL Connection for execution.

Use of '.format()' vs. '%s' in cursor.execute() for mysql JSON field, with Python mysql.connector,

My objective is to store a JSON object into a MySQL database field of type json, using the mysql.connector library.
import mysql.connector
import json
jsonData = json.dumps(origin_of_jsonData)
cnx = mysql.connector.connect(**config_defined_elsewhere)
cursor = cnx.cursor()
cursor.execute('CREATE DATABASE dataBase')
cnx.database = 'dataBase'
cursor = cnx.cursor()
cursor.execute('CREATE TABLE table (id_field INT NOT NULL, json_data_field JSON NOT NULL, PRIMARY KEY (id_field))')
Now, the code below WORKS just fine, the focus of my question is the use of '%s':
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES (%s, %s)"
values_to_insert = (1, jsonData)
cursor.execute(insert_statement, values_to_insert)
My problem with that: I am very strictly adhering to the use of '...{}'.format(aValue) (or f'...{aValue}') when combining variable aValue(s) into a string, thus avoiding the use of %s (whatever my reasons for that, let's not debate them here - but it is how I would like to keep it wherever possible, hence my question).
In any case, I am simply unable, whichever way I try, to create something that stores the jsonData into the mySql dataBase using something that resembles the above structure and uses '...{}'.format() (in whatever shape or form) instead of %s. For example, I have (among many iterations) tried
insert_statement = "INSERT INTO table (id_field, json_data_field) VALUES ({}, {})".format(1, jsonData)
cursor.execute(insert_statement)
but no matter how I turn and twist it, I keep getting the following error:
ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near '[some_content_from_jsonData})]' at line 1
Now my question(s):
1) Is there a way to avoid the use of %s here that I am missing?
2) If not, why? What is it that makes this impossible? Is it the cursor.execute() function, or is it the fact that it is a JSON object, or is it something completely different? Shouldn't {}.format() be able to do everything that %s could do, and more?

First of all: NEVER DIRECTLY INSERT YOUR DATA INTO YOUR QUERY STRING!
Using %s in a MySQL query string is not the same as using it in a python string.
In python, you just format the string and 'hello %s!' % 'world' becomes 'hello world!'. In SQL, the %s signals parameter insertion. This sends your query and data to the server separately. You are also not bound to this syntax. The python DB-API specification specifies more styles for this: DB-API parameter styles (PEP 249). This has several advantages over inserting your data directly into the query string:
Prevents SQL injection
Say you have a query to authenticate users by password. You would do that with the following query (of course you would normally salt and hash the password, but that is not the topic of this question):
SELECT 1 FROM users WHERE username='foo' AND password='bar'
The naive way to construct this query would be:
"SELECT 1 FROM users WHERE username='{}' AND password='{}'".format(username, password)
However, what would happen if someone inputs ' OR 1=1 as password. The formatted query would then become
SELECT 1 FROM users WHERE username='foo' AND password='' OR 1=1
which will allways return 1. When using parameter insertion:
execute('SELECT 1 FROM users WHERE username=%s AND password=%s', username, password)
this will never happen, as the query will be interpreted by the server separately.
Performance
If you run the same query many times with different data, the performance difference between using a formatted query and parameter insertion can be significant. With parameter insertion, the server only has to compile the query once (as it is the same every time) and execute it with different data, but with string formatting, it will have to compile it over and over again.

In addition to what was said above, I would like to add some details that I did not immediately understand, and that other (newbies like me ;)) may also find helpful:
1) "parameter insertion" is meant for only for values, it will not work for table names, column names, etc. - for those, the Python string substitution works fine in the sql syntax defintion
2) the cursor.execute function requires a tuple to work (as specified here, albeit not immediately clear, at least to me: https://dev.mysql.com/doc/connector-python/en/connector-python-api-mysqlcursor-execute.html)
EXAMPLE for both in one function:
def checkIfRecordExists(column, table, condition_name, condition_value):
...
sqlSyntax = 'SELECT {} FROM {} WHERE {} = %s'.format(column, table, condition_name)
cursor.execute(sqlSyntax, (condition_value,))
Note both the use of .format in the initial sql syntax definition and the use of (condition_value,) in the execute function.

Print if MySQL returns no results

This is my code so far. I'm attempting to print No results found if no results are returned by MySQL however I can't figure it out. Perhaps I'm using incorrect arguments. Could anyone provide me with an example? Much appreciated!
def movie_function(film):
connection = mysql connection info
cursor = connection.cursor()
sql = "SELECT * FROM film_database WHERE film_name = '"+film+"' ORDER BY actor"
cursor.execute(sql)
rows = cursor.fetchall()
for row in rows:
print row[1]

When you execute a select statement, cursor.rowcount is set to the number of results retrieved. Also, there is no real need to call cursor.fetchall(); looping over the cursor directly is easier:
def movie_function(film):
connection = mysql connection info
cursor = connection.cursor()
sql = "SELECT * FROM film_database WHERE film_name = %s ORDER BY actor"
cursor.execute(sql, (film,))
if not cursor.rowcount:
print "No results found"
else:
for row in cursor:
print row[1]
Note that I also switched your code to use SQL parameters; there is no need to use string interpolation here, leave that to the database adapter. The %s placeholder is replaced for you by a correctly quoted value taken from the second argument to cursor.execute(), a sequence of values (here a tuple of one element).
Using SQL parameters also lets a good database reuse the query plan for the select statement, and leaving the quoting up to the database adapter prevents SQL injection attacks.

You could use cursor.rowcount after your code to see how many rows were actually returned. See here for more.

I guess, this should work.
def movie_function(film):
connection = mysql connection info
cursor = connection.cursor()
sql = "SELECT * FROM film_database WHERE film_name = %s ORDER BY actor"
cursor.execute(sql, [film])
rows = cursor.fetchall()
if not rows:
print 'No resulrs found'
return
for row in rows:
print row[1]
Note, that I changed the way the film parameter is passed to query. I don't know, how exactly it should be (this depends on what MySQL driver for python you use), but important thing to know, is that you should not pass your parameters directly to the query string, because of security reasons.

You can also use :
rows_affected=cursor.execute("SELECT ... ") -> you have directly the number of returned rows

Python, MySQLdb, unable to update row

I know SO is not a "debug my code service" but after checking and retrying for hours either I'm missing something very stupid or maybe there's a bug (or bad compilation) in my MySQLdb module...
Also I've some related question I've put along with the posted code...
def NextDocumentIdToCache():
if not OpenConnection():
return 0
# ...setting string values... #
cur = connection.cursor(mysql.cursors.DictCursor)
cur.execute('SELECT * FROM documents WHERE resolutions <> %s AND pdf > 0 AND status = %s AND expire > %s AND locked = %s LIMIT 0,1', (resolutions, status, expireLimit, ''))
rowsCount = cur.rowcount
if rowsCount==0:
return 0
row = cur.fetchone()
id = row['document_id']
Everything ok since now. Connection opens, I get one row and retrieve the correct id that is returned at the end of the function.
Then I need to perform an update operation on the row fetched...
cur.close()
cur = connection.cursor(mysql.cursors.Cursor)
I've closed the cursor and opened a new one. Do I actually need to do this? Or can I reuse the same cursor?
cur.execute("""UPDATE documents SET locked = %s WHERE document_id = %s""", ("worker: rendering pages", id))
This actually doesn't update the row. No exceptions happens, just don't work.
Finally the function ends...
cur.close()
return id
Also a couple of questions.
What's the difference between
cur.execute("""UPDATE documents....
and
cur.execute("UPDATE documents....
I've seen around both versions. What's the functional difference between triple double-quote, single double-quote, single single-quote?
And finally
If I write the query
cur.execute("""UPDATE documents SET locked = %s WHERE document_id = %d""", ("worker: rendering pages", id))
(note the %d instead of %s) I get an error. But id is a long integer, I've checked. So what's wrong?
Thank you

The change doesn't take effect:
Are you doing a transaction? You'll need to commit() if that's the case. PEP 249, which defines the Python Database API, states that a connection will use transactions even if you have not explicitly enabled them: "Note that if the database supports an auto-commit feature, this must be initially off."
Quoting
Triple quoted strings are multiline, single quoted strings aren't.
Interpolation
Because this is not actually string interpolation. Those placeholders are interpreted by MySQLdb, which will handle quoting for you.