I've been using this syntax with great success in python mysql.
search = "O'%" # find names like O'Brien or O'Connell...
cursor.execute ("""
select userid
from usertab
where name like %s
""" , (search))
But sometimes I need to build my sql string before I execute it like the following, but the substitution technique is different than above and doesn't work in all cases.
search = "O'%" # find names like O'Brien or O'Connell...
sql = """
select userid
from usertab
where name like '%s'
""" % (search)
cursor.execute(sql)
How can I achieve the same kind of string substitution that works well in the first example, without executing the cursor?
MySQLdb uses the connection's literal() method to escape the arguments, so you could use:
sql = """
select userid
from usertab
where name like %s
""" % cursor.connection.literal(search)
Related
This question already has an answer here:
Dynamic SQL Queries with Python and mySQL
(1 answer)
Closed 3 months ago.
I am unable to execute the following statement I keep getting SQL syntax errors.
According to all the examples I can find this should work
Any help will be greatly appreciated.
d2 = df.iloc[-1,:]
q = symbol+'_ivol'
query = """SELECT close FROM %s WHERE date = %s"""
VALUES= (q, d2[1])
cursor.execute(query, VALUES)
ivol = cursor.fetchall()
conn.close()
Query parameters in SQL are not just string substitution. You can't use a query parameter for a table identifier. Parameters can only be used where you would normally use a quoted string literal or numeric literal.
Stated another way, all the identifiers must be fixed in the query string before you prepare it, because identifiers must be validated during the prepare phase, to make sure the table is a valid identifier, and that the table exists. You can't pass the name of a table identifier after the query has been prepared.
The Python driver unfortunately makes this more confusing because it uses %s instead of MySQL's own ? symbol for the parameter placeholder. This makes developers naturally think that %s is simply string substitution, like it is for Python string formatting.
So there's %s and there's %s, and they are handled differently. I know, it's confusing.
So you can do a plain string-formatting substitution to put your table into the query string:
query = """SELECT close FROM %s WHERE date = %%s""".format(q)
But it's more idiomatic for modern Python to use f-string formatting:
query = f"""SELECT close FROM `{q}` WHERE date = %s"""
I put back-ticks around the table name, just in case it's a SQL reserved keyword or something.
Then the other %s is an actual query parameter, because it works as a scalar value in the SQL expression. In this query, there is just one query parameter.
VALUES= [ d2[1] ]
cursor.execute(query, VALUES)
How can one get executemany to format a prepared SQL to write unicode data?
I have a prepared query that is similar to this:
insert into foobar select(select baz.somestring from baz where baz.whatever = %s ), %s
Changing the %s directly into something like N'%s' in the prepared query won't work, how do i get pymssql to encase my inputs with N'mystring' instead of a simple 'mystring'?
The current behaviour results in questionmarks in the table when something like a \u5000 comes around, for example, while it should save the \u5000 or any other unicode character.
how do i get pymssql to encase my inputs with N'mystring' instead of a simple 'mystring'?
pymssql will do that if you create a proper parameterized query:
print(sys.version) # 3.7.2 ...
print(pymssql.__full_version__) # 2.1.4
#
sql = "SELECT COUNT(*) AS n FROM #tmp WHERE whatever = %s"
params = ('Ώπα',)
crsr.execute(sql, params)
print(crsr.fetchone())
SQL Profiler shows that the query is sent as
SELECT COUNT(*) AS n FROM #tmp WHERE whatever = N'Ώπα'
I have an query string in Python as follows:
query = "select name from company where id = 13 order by name;"
I want to be able to change the id dynamically. Thus I want to find id = 13 and replace it with a new id.
I can do it as follows:
query.replace("id = 13", "id = {}".format(some_new_id))
But if in the query is id= 13 or id=13 or id =13, ... it will not work.
How to avoid that?
Gluing variables directly into your query leaves you vulnerable to SQL injection.
If you are passing your query to a function to be executed in your database, that function should accept additional parameters.
For instance,
query = "select name from company where id = %s order by name"
cursor.execute(query, params=(some_other_id,))
It is better to use formatted sql.
Ex:
query = "select name from company where id = %s order by name;".
cursor.execute(query, (id,))
The usual solution when it comes to dynamically building strings is string formatting, ie
tpl = "Hello {name}, how are you"
for name in ("little", "bobby", "table"):
print(tpl.format(name))
BUT (and that's a BIG "but"): you do NOT want to do this for SQL queries (assuming you want to pass this query to your db using your db's python api).
There are two reasons to not use string formatting here: the first one is that correctly handling quoting and escaping is tricky at best, the second and much more important one is that it makes your code vulnerable to SQL injections attacks.
So in this case, the proper solution is to use prepared statements instead:
# assuming MySQL which uses "%" as placeholder,
# consult your db-api module's documentation for
# the proper placeholder
sql = "select name from company where id=%s order by name"
cursor = yourdbconnection.cursor()
cursor.execute(sql, [your_id_here])
I hope i can be clear about my problem. thank you :)
I'm using impala connection (library: from impala.dbapi import connect).
In order to run a query i'm using the execute command:
cursor.execute(query.value, (year_var, month_var,day_var))
Generally - it works just fine, also with variables. the problem begins when i use a SQL LIKE statement (e.g. like '%seo' - which contain %s in it).
The 1st argument (query.value) is a string:
create table bi_db.search_terms as
select search_query,search_contain,count(*) searches
from (
select search_query,
case when lower(search_query) like '%logo%' then 'logo'
when lower(search_query) like '%google%' then 'google'
when lower(search_query) like '%facebook%' then 'facebook'
when lower(search_query) like '%instagram%' then 'instagram'
when lower(search_query) like '%etsy%' then 'etsy'
when lower(search_query) like '%seo%' then 'seo'
when lower(search_query) like '%social media%' then 'social media'
else 'else' end as search_contain
from traffic_db.traffic_parq a
where year = %s AND month = %s AND day = %s AND controller = 'search' and action in ('gigs','users')
and search_query is not null and search_query<>'' ) t
group by search_query,search_contain
the second argument of the cursor.execute (e.g. (year_var, month_var,day_var)) refer to %s i'm putting on the query i run in order to use dynamic variables.
** The problem is that the python thinks it has 5 arguments instead of only 3. that caused because i have %seo and %social in the LIKE statments **
Anyone encountered this kind of problem? know how to solve it?
Many thanks!
You could escape literal percent signs in the query (e.g. %%seo%%), although it would be cleaner to pass the patterns as parameters to execute() as well:
sql = """
create table bi_db.search_terms as
select search_query,search_contain,count(*) searches
from (
select search_query,
case when lower(search_query) like %s then 'logo'
...
"""
cursor.execute(sql, ('%logo%', ...))
I tried to see if this question had been asked, which it probably has, but I couldn't find an answer. I am querying MS SQL Server from a python script using pyodbc.
I am defining a function to query SQL. The query string has both a '%R_%' which is meant as a wildcard for SQL to interpret, but also a '%s' which is meant to place a variable in python between single quotes. It looks like this:
def numcust(matter):
QryString = """
select count(distinct user_id)
from dbo.tbl_case_details
where request like '%r_%'
and project_id in
(
select distinct project_id
from dbo.tbl_projects
where matter_number = '%s'
);
""" % matter
cursor.execute(QryString)
row = cursor.fetchone()
return row[0]
How can I escape the wildcards for r_ so that I can successfully pass it through to SQL?
Thank you kindly.
Double the % where you want to escape them (i.e. '%%r_%%').
Also, interpolate the string in the cursor.execute function like so:
cursor.execute(QryString, matter)
"%% %s"%("Hello",) two % signs should become one after applying the %s stuff ...
but really you should be using the built in query stuff stuff
QryString = """
select count(distinct user_id)
from dbo.tbl_case_details
where request like '%r_%'
and project_id in
(
select distinct project_id
from dbo.tbl_projects
where matter_number = '%s'
);
"""
cursor.execute(QryString,matter)
although you still may need to do %% for a literal %