I hope i can be clear about my problem. thank you :)
I'm using impala connection (library: from impala.dbapi import connect).
In order to run a query i'm using the execute command:
cursor.execute(query.value, (year_var, month_var,day_var))
Generally - it works just fine, also with variables. the problem begins when i use a SQL LIKE statement (e.g. like '%seo' - which contain %s in it).
The 1st argument (query.value) is a string:
create table bi_db.search_terms as
select search_query,search_contain,count(*) searches
from (
select search_query,
case when lower(search_query) like '%logo%' then 'logo'
when lower(search_query) like '%google%' then 'google'
when lower(search_query) like '%facebook%' then 'facebook'
when lower(search_query) like '%instagram%' then 'instagram'
when lower(search_query) like '%etsy%' then 'etsy'
when lower(search_query) like '%seo%' then 'seo'
when lower(search_query) like '%social media%' then 'social media'
else 'else' end as search_contain
from traffic_db.traffic_parq a
where year = %s AND month = %s AND day = %s AND controller = 'search' and action in ('gigs','users')
and search_query is not null and search_query<>'' ) t
group by search_query,search_contain
the second argument of the cursor.execute (e.g. (year_var, month_var,day_var)) refer to %s i'm putting on the query i run in order to use dynamic variables.
** The problem is that the python thinks it has 5 arguments instead of only 3. that caused because i have %seo and %social in the LIKE statments **
Anyone encountered this kind of problem? know how to solve it?
Many thanks!
You could escape literal percent signs in the query (e.g. %%seo%%), although it would be cleaner to pass the patterns as parameters to execute() as well:
sql = """
create table bi_db.search_terms as
select search_query,search_contain,count(*) searches
from (
select search_query,
case when lower(search_query) like %s then 'logo'
...
"""
cursor.execute(sql, ('%logo%', ...))
Related
I'm trying to get my db.execute to work, managed to resolve the syntax error when using the LIKE operator along with a variable passed in from HTML, but it still doesn't give me results.
(got closed by admin, so re-posting)
Used flask console to print and find out if any values passed at all, and it didn't.
found variable not returning anything from dbExecute function
Why are my results not getting passed from db.execute?
My code extracted below:
#app.route("/search", methods=["POST"])
def search():
"""Search for books"""
# best example, but no data passed:
found = db.execute("SELECT * FROM books_table WHERE (isbn LIKE :lookingFor) OR (title LIKE :lookingFor) OR (title LIKE :lookingFor) OR (year::text LIKE :lookingFor)", {"lookingFor": f"%{searchBookVariableOnApplication_py}%"}).fetchall();
return render_template("search.html", found=found)
You're using an f-string in an attempt to use the variable searchBookVariableOnApplication_py, but not interpolating it within the f-string.
This:
{"lookingFor": f"\"%searchBookVariableOnApplication_py%\""}
Should be this:
{"lookingFor": f"\"%{searchBookVariableOnApplication_py}%\""}
The problem here is that you are not binding your wildcard string literals correctly to your query string. You should be using a prepared statement here. Assuming you are using psycopg2, you may try:
import psycopg2
searchBook = request.form['searchBook']
sql = """SELECT *
FROM books_table
WHERE isbn LIKE %s OR title LIKE %s OR year::text LIKE %s"""
param = "%" + searchBook + "%"
found = db.execute(sql, (searchBook, searchBook, searchBook,))
I am trying to extract the names from the db that have A in the second position.
In sql it's simple but python sees the '_A%' as end of query.
Has anyone faced this problem before and came out with a solution?
I saw a similar question and the accept result was to use '% %' instead of ' %', but this didn't worked.
This is my query:
def queryDelivery(start_date):
query_basictable = """
SELECT Code,Quantity, Datetime
FROM Mytable
WHERE Datetime>= '%s 12:00:00' AND Name LIKE '_A%'
""" %(start_date)
delivery_data= pd.read_sql(sql=query_basictable, con=engine)
return delivery_data
I was thinking about passing the symbol '_A%' to a variable and the do something like a substitute but when try to assign the symbol hits syntax error
variable = ''_A%' '
Name LIKE variable
How can I do this in a clean way?
Don't do it this way. As soon as you do this, if someone inserts a start_date like "'; drop table students; --" you have a problem.
I tested placeholders in Python 2.7 and it looks like you don't run into the problem until you use the % operator.
A much better way is to write your SQL statements in a way that every value passed in can be used in a placeholder. Then use placeholder syntax and the syntax becomes LIKE ? || '%'
What I want is execute the sql
select * from articles where author like "%steven%".
For the sake of safety, i used like this way :
cursor.execute('select * from articles where %s like %s', ('author', '%steven%')
Then the result is just empty, not get a syntax error, but just empty set.
But I am pretty sure there is some thing inside, I can get result use the first sql. Is there anything run with my code ?
You can't set a column name like a parameter where you're doing where %s like %s. To dynamically set the column name you need to do actual string manipulation like:
sql = 'select * from articles where '+ sql_identifier('author') +' like %s'
cursor.execute(sql, ('%steven%',))
Where sql_identifier is your lib's function for making an identifier safe for SQL injection. Something like:
# don't actually use this!
def sql_identifier(s):
return '"%s"' % s.replace('"','')
But with actual testing and knowledge of the DB engine you're using.
The problem here is fact a minor mistake. Thanks to #Asad Saeeduddin, when I try to use print cursor._last_executed to check what has happened. I found that what is in fact executed is
SELECT * FROM articles WHERE 'title' LIKE '%steven%', look the quotation mark around the title, that's the reason why I got empty set.
So always remember the string after formatting will have a quotation around
I tried to see if this question had been asked, which it probably has, but I couldn't find an answer. I am querying MS SQL Server from a python script using pyodbc.
I am defining a function to query SQL. The query string has both a '%R_%' which is meant as a wildcard for SQL to interpret, but also a '%s' which is meant to place a variable in python between single quotes. It looks like this:
def numcust(matter):
QryString = """
select count(distinct user_id)
from dbo.tbl_case_details
where request like '%r_%'
and project_id in
(
select distinct project_id
from dbo.tbl_projects
where matter_number = '%s'
);
""" % matter
cursor.execute(QryString)
row = cursor.fetchone()
return row[0]
How can I escape the wildcards for r_ so that I can successfully pass it through to SQL?
Thank you kindly.
Double the % where you want to escape them (i.e. '%%r_%%').
Also, interpolate the string in the cursor.execute function like so:
cursor.execute(QryString, matter)
"%% %s"%("Hello",) two % signs should become one after applying the %s stuff ...
but really you should be using the built in query stuff stuff
QryString = """
select count(distinct user_id)
from dbo.tbl_case_details
where request like '%r_%'
and project_id in
(
select distinct project_id
from dbo.tbl_projects
where matter_number = '%s'
);
"""
cursor.execute(QryString,matter)
although you still may need to do %% for a literal %
I've been using this syntax with great success in python mysql.
search = "O'%" # find names like O'Brien or O'Connell...
cursor.execute ("""
select userid
from usertab
where name like %s
""" , (search))
But sometimes I need to build my sql string before I execute it like the following, but the substitution technique is different than above and doesn't work in all cases.
search = "O'%" # find names like O'Brien or O'Connell...
sql = """
select userid
from usertab
where name like '%s'
""" % (search)
cursor.execute(sql)
How can I achieve the same kind of string substitution that works well in the first example, without executing the cursor?
MySQLdb uses the connection's literal() method to escape the arguments, so you could use:
sql = """
select userid
from usertab
where name like %s
""" % cursor.connection.literal(search)