In PgAdmin, I can do the following query successfully:
select * from "Faces" where "Face_Name" = 'Alex'
However, when I try to do the exact same query in python, I get endless syntax errors.
I am trying to write the line like this:
cursor.execute('SELECT * from "Faces" where ("Face_Name" = 'Alex')
I understand the table and column names need to be in double quotes, and the whole query needs to be in single quotes. Also seems the string (in this case 'Alex') that I am searching for needs to be in single quotes.
How do I put all this together into a single line?
Assuming you did need to escape the table and column names, you could use double quotes. In that case, just escape the double quotes inside the Python SQL string:
sql = "SELECT * FROM \"Faces\" WHERE \"Face_Name\" = 'Alex'"
cursor.execute(sql)
There are two issues here:
As others already wrote, you need to be careful not to mix up the Python and SQL quotes; depending on the field name you may need to have both in the query, and either escape one of them or use """ for the Python string.
If the name "Alex" comes from a variable in Python, rather than being a constant, you should use a placeholder and pass it separately. This will help avoid security problems (SQL Injection) and is a good habit to get into whether or not it's required in this particular case.
Putting these together, the query should be:
cursor.execute('SELECT * from "Faces" where "Face_Name" = %s', ('Alex',))
Related
I am trying to execute mysql query from python. I want the output
query = "UPDATE 'college_general' SET 'fees' = '180000' WHERE ('college_id' = '2')"
Below is the snippet of the code
def update(table, column, value):
return f"UPDATE '{table}' SET '{column}' = '{value}' WHERE ('college_id' = '{id}')"
query = update("college_general", "fees", fee)
cursor.execute(query)
Instead Python is storing it like
query = 'UPDATE \'college_general\' SET \'fees\' = \'180000\' WHERE (\'college_id\' = \'2\')'
which is causing the script to fail. How can I achieve the desired output?
Thanks in advance!
You can replace the identifiers single quotes with backticks. For more detailed answers visit this question.
There are two types of quotes in MySQL:
' for enclosing string literals
` for enclosing identifiers such as table and column names
There are multiple issues here:
First, I suspect that the string handling bit of your program is actually working, but you are being confused by the external representation of strings. For example, if you do
x = "O'Reilly"
Python will, in some circumstances, display the string as
'O\'Reilly'
Second, I think you are using the wrong kind of quotes. Single quotes in SQL are for strings; MySQL uses backticks for names when necessary, while other SQL implementations usually use double quotes for this.
Third, AND THIS IS IMPORTANT! Do not use string manipulation for building SQL queries. The database library almost certainly has a feature for parametrized queries and you should be using that. Your query should look something like this:
query = 'UPDATE college_general SET fees = ? WHERE college_ID = ?'
cursor.execute(query, [180000, '2'])
but the details will depend on the DB library you are using. For example, some use %s instead of ?. This saves you from all kinds of headaches with quoting strings.
raw string is the simplest solution to your problem.
I believe the code below will achieve what you wanted.
def update(table, column, value):
return fr"UPDATE '{table}' SET '{column}' = '{value}' WHERE ('college_id' = '{id}')"
query = update("college_general", "fees", fee)
cursor.execute(query)
I am using hand crafted SQL to fetch data from a PG database, using SqlAlchemy. I am trying a query which contains the SQL like operator '%' and that seems to throw SqlAlcjhemy through a loop:
sql = """
SELECT DISTINCT u.name from user u
INNER JOIN city c ON u.city_id = c.id
WHERE c.designation=upper('fantasy')
AND c.id IN (select id from ref_geog where short_name LIKE '%opt')
"""
# The last line in the above statement throws the error mentioned in the title.
# However if the last line is change to:
# AND c.id IN (select id from ref_geog where short_name = 'helloopt')
# the script runs correctly.
#
# I also tried double escaping the '%' i.e. using '%%' instead - that generated the same error as previously.
connectDb()
res = executeSql(sql)
print res
closeDbConnection()
Any one knows what is causing this misleading error message and how I may fix it?
[[Edit]]
Before any one asks, there is nothing special or fancy about the functions included above. For example the function executeSql() simply invokes conn.execute(sql) and returns the results. The variable conn is simply the previously established connection to the database.
You have to give %% to use it as % because % in python is use as string formatting so when you write single % its assume that you are going to replace some value with this.
So when you want to place single % in string with query allways place double %.
SQLAlchemy has a text() function for wrapping text which appears to correctly escape the SQL for you.
I.e.
res = executeSql(sqlalchemy.text(sql))
should work for you and save you from having to do the manual escaping.
I cannot find the "executeSql" in sqlalchemy version 1.2 docs , but the below line worked for me
engine.execute(sqlalchemy.text(sql_query))
I found one more case when this error shows up:
c.execute("SELECT * FROM t WHERE a = %s")
In other words, if you provide parameter (%s) in query, but you forget to add query params. In this case error message is very misleading.
It seems like your problem may be related to this bug.
In which case, you should triple-escape as a workaround.
One more note- you must escape (or delete) % characters in comments as well. Unfortunately, sqlalchemy.text(query_string) does not escape the percent signs in the comments.
Another way of solving your problem, if you don't want to escape % characters or use sqlalchemy.text(), is to use a regular expression.
Instead of:
select id from ref_geog where short_name LIKE '%opt'
Try (for case-sensitive match):
select id from ref_geog where short_name ~ 'opt$'
or (for case-insensitive):
select id from ref_geog where short_name ~* 'opt$'
Both LIKE and regex are covered in the documentation on pattern matching.
Note that:
Unlike LIKE patterns, a regular expression is allowed to match anywhere within a string, unless the regular expression is explicitly anchored to the beginning or end of the string.
For an anchor, you can use the assertion $ for end of string (or ^ for beginning).
This could also result from the case - in case parameters to be passed onto the SQL are declared in DICT formate and are being manipulated in the SQL in the form of LIST or TUPPLE.
I have a function, where I get a string as parameter. I want to save this string to a database. So I have a command like:
sql_command = """INSERT INTO some_table(some_text_row) VALUE (
'{0}');""".format(some_text)
But the parameter can contain characters like '. So I need to replace this sort of characters. I do this with this function:
some_text = given_parameter.replace("'", r"\'")
But now comes the strange behavior: Sometimes, I get a result of \\' and sometimes I get a result of \'. I want to have the second one.
To give you more information: The given_parameter is the HTML code of a webpage. I get the HTML code from the library called requests
Does anyone have some tipps?
Don't construct the query using string formatting - this is unsafe, you are making it vulnerable to SQL injections.
Instead, parameterize the query and let the mysql driver worry about quotes:
sql_command = """
INSERT INTO
some_table(some_text_row)
VALUES
(%s)"""
cursor.execute(sql_command, (some_text, ))
I am using hand crafted SQL to fetch data from a PG database, using SqlAlchemy. I am trying a query which contains the SQL like operator '%' and that seems to throw SqlAlcjhemy through a loop:
sql = """
SELECT DISTINCT u.name from user u
INNER JOIN city c ON u.city_id = c.id
WHERE c.designation=upper('fantasy')
AND c.id IN (select id from ref_geog where short_name LIKE '%opt')
"""
# The last line in the above statement throws the error mentioned in the title.
# However if the last line is change to:
# AND c.id IN (select id from ref_geog where short_name = 'helloopt')
# the script runs correctly.
#
# I also tried double escaping the '%' i.e. using '%%' instead - that generated the same error as previously.
connectDb()
res = executeSql(sql)
print res
closeDbConnection()
Any one knows what is causing this misleading error message and how I may fix it?
[[Edit]]
Before any one asks, there is nothing special or fancy about the functions included above. For example the function executeSql() simply invokes conn.execute(sql) and returns the results. The variable conn is simply the previously established connection to the database.
You have to give %% to use it as % because % in python is use as string formatting so when you write single % its assume that you are going to replace some value with this.
So when you want to place single % in string with query allways place double %.
SQLAlchemy has a text() function for wrapping text which appears to correctly escape the SQL for you.
I.e.
res = executeSql(sqlalchemy.text(sql))
should work for you and save you from having to do the manual escaping.
I cannot find the "executeSql" in sqlalchemy version 1.2 docs , but the below line worked for me
engine.execute(sqlalchemy.text(sql_query))
I found one more case when this error shows up:
c.execute("SELECT * FROM t WHERE a = %s")
In other words, if you provide parameter (%s) in query, but you forget to add query params. In this case error message is very misleading.
It seems like your problem may be related to this bug.
In which case, you should triple-escape as a workaround.
One more note- you must escape (or delete) % characters in comments as well. Unfortunately, sqlalchemy.text(query_string) does not escape the percent signs in the comments.
Another way of solving your problem, if you don't want to escape % characters or use sqlalchemy.text(), is to use a regular expression.
Instead of:
select id from ref_geog where short_name LIKE '%opt'
Try (for case-sensitive match):
select id from ref_geog where short_name ~ 'opt$'
or (for case-insensitive):
select id from ref_geog where short_name ~* 'opt$'
Both LIKE and regex are covered in the documentation on pattern matching.
Note that:
Unlike LIKE patterns, a regular expression is allowed to match anywhere within a string, unless the regular expression is explicitly anchored to the beginning or end of the string.
For an anchor, you can use the assertion $ for end of string (or ^ for beginning).
This could also result from the case - in case parameters to be passed onto the SQL are declared in DICT formate and are being manipulated in the SQL in the form of LIST or TUPPLE.
I could write a SP inside Mysql and excute with a call statement. But looking to write it in python instead. I got stuck with using sql script on multiple lines.
conn = pyodbc.connect('DSN=MySQL;PWD=xxxx')
csr = conn.cursor()
Sql= 'SELECT something, something
FROM table
WHERE foo=bar
ORDER BY foo '
csr.execute(Sql)
sqld = csr.fetchall()
Heh, I don't mind to make it a proper answer.
String literals in triple quotes can include linebreaks and won't cause syntax errors. Otherwise (with "string" or 'string') you will need to include a backslash before every linebreak to make it work. And from experience, that's easy to screw up. :)
As a minor note, in Python variables are usually started with a lowercase letter, names starting with capital letters usually being given to classes.
So:
Sql = """SELECT something, something
FROM table
WHERE foo=bar
ORDER BY foo"""
If you don't mind the overhead, take a look at sqlalchemy:
SQLAlchemy is the Python SQL toolkit and Object Relational Mapper that gives application developers the full power and flexibility of SQL.