i was trying to update a table row by row in postgresql using python.
the code used was
cursor.execute("UPDATE im_entry.pr_table
SET selected_entry = im_entry.usr_table.",entryn,"
FROM im_entry.usr_table
WHERE im_entry.pr_table.image_1d = ",idn,"")
...where entryn and idn are two string variables ( entry1,entry2.. id1,id2..etc)
I am getting an error
TypeError: function takes at most 3 arguments (5 given)
My table is
image_1d | entry1 | entry2 | entry3 | entry4 | entry5
----------+--------+--------+--------+--------+--------
How can I solve this?
Try:
cursor.execute(
'''UPDATE im_entry.pr_table
SET selected_entry = im_entry.usr_table.{0}
FROM im_entry.usr_table
WHERE im_entry.pr_table.image_1d = ?'''.format(entryn),[idn])
Normally, you would want to call cursor.execute(sql,args), where sql is a parametrized sql query, and args is a list or tuple of values to be substituted for the parameter placeholders.
For Postgresql, the usual db driver, pyscopg, uses question marks for the parameter placeholders.
Thus, usually, you'd want to use something like
sql='''UPDATE im_entry.pr_table
SET selected_entry = ?
FROM im_entry.usr_table
WHERE im_entry.pr_table.image_1d = ?'''
but in your case, you aren't trying to set selected_entry to a specific value, but rather to a column name. Is that correct? In that case, you unfortunately can't use a parameter placeholder. Instead, you have to use string formatting, which is what I suggested above.
You can't bind table or column names, only the values associated with them.
Related
I´m trying to write a sql query in python, where I want the user to pass a date, saved in a variable. Then I want this to be processed in the query. While it´s pretty easy to do so in R, I´m curious if there´s an easy approach in python too.
The Date should be passed where 2022-12-12 is.
date = input("this format YYYY-MM-DD")
query_new_Limits = """
SELECT #######
, a.RULE_NAME
, a.RESULT_VALUE
, a.SUMMARY_DETAIL
, a.REF_DATE
FROM ######## a ################### b
ON b.T_PORTFOLIO_ID = a.PORTFOLIO_ID
WHERE a.REF_DATE = TO_DATE('2022-12-12','YYYY-MM-DD')
AND REGEXP_LIKE(a.PORTFOLIO_CODE,'^[[:digit:]]+$')
AND NOT b.POR_INVESTMENT_TYPE IN #######
AND b.REF_TO_DATE > TO_DATE('2022-12-12','YYYY-MM-DD')
AND a.RESULT_STATE_ID > 12
"""
Just a quick advice:
avoid "date" as variable name as it's a reserved word
maybe the underlying driver is not using parameters "bind by name" - try passing an array of parameters as ( '2022-12-31', '2022-12-31') or such, just to rule out the parameter name issue
From the docs
SQL and PL/SQL statements that pass data to and from Oracle Database should use placeholders in SQL and PL/SQL statements that mark where data is supplied or returned. These placeholders are referred to as bind variables or bind parameters. A bind variable is a colon-prefixed identifier or numeral. For example, there are two bind variables (dept_id and dept_name) in this SQL statement:
sql = """insert into departments (department_id, department_name)
values (:dept_id, :dept_name)"""
cursor.execute(sql, [280, "Facility"])
[or named binding]
cursor.execute("""
insert into departments (department_id, department_name)
values (:dept_id, :dept_name)""", dept_id=280,
dept_name="Facility")
Which applied to your example gives:
date = input("this format YYYY-MM-DD")
query_new_Limits = """
SELECT #######
, a.RULE_NAME
, a.RESULT_VALUE
, a.SUMMARY_DETAIL
, a.REF_DATE
FROM ######## a ################### b
ON b.T_PORTFOLIO_ID = a.PORTFOLIO_ID
WHERE a.REF_DATE = TO_DATE(:date,'YYYY-MM-DD')
AND REGEXP_LIKE(a.PORTFOLIO_CODE,'^[[:digit:]]+$')
AND NOT b.POR_INVESTMENT_TYPE IN #######
AND b.REF_TO_DATE > TO_DATE(:date,'YYYY-MM-DD')
AND a.RESULT_STATE_ID > 12
"""
cursor.execute(query_new_Limits, date=date) # using kwarg binding
I have a python script with a basic GUI that logs into a DB and executes a query.
The Python script also asks for 1 parameter called "collection Name" which is taken from the tkinter .get function and is added as a %s inside the Query text. The result is that each time I can execute a query with a different "Collection name". This works and it is fine
Now, I want to add a larger string of Collection Names into my .get function so I can do cursor.execute a query with multiple collection names to get more complex data. But I am having issues with inputing multiple "collection names" into my app.
Below is a piece of my Query1, which has the %s variable that it then gets from the input to tkinter.
From #Session1
Join vGSMRxLevRxQual On(#Session1.SessionId = vGSMRxLevRxQual.SessionId)
Where vGSMRxLevRxQual.RxLevSub<0 and vGSMRxLevRxQual.RxLevSub>-190
and #Session1.CollectionName in (%s)
Group by
#Session1.Operator
Order by #Session1.Operator ASC
IF OBJECT_ID('tempdb..#SelectedSession1') IS NOT NULL DROP TABLE #SelectedSession1
IF OBJECT_ID('tempdb..#Session1') IS NOT NULL DROP TABLE #Session1
Here, is where I try to execute the query
if Query == "GSMUERxLevelSub" :
result = cursor.execute(GSMUERxLevelSub, (CollectionName,))
output = cursor.fetchmany
df = DataFrame(cursor.fetchall())
filename = "2021_H1 WEEK CDF GRAPHS().xlsx"
df1 = DataFrame.transpose(df, copy=False)
Lastly, here is where I get the value for the Collection name:
CollectionName = f_CollectionName.get()
enter image description here
enter code here
Your issues are due to a list/collection being a invalid parameter.
You'll need to transform collectionName
collection_name: list[str] = ['collection1', 'collection2']
new_collection_name = ','.join(f'"{c}"' for c in collection_name)
cursor.execute(sql, (new_collection_name,))
Not sure if this approach will be susceptible to SQL injection if that's a concern.
Edit:
Forgot the DBAPI would put another set of quotes around the parameters. If you can do something like:
CollectionName = ["foo", "bar"]
sql = f"""
From #Session1
Join vGSMRxLevRxQual On(#Session1.SessionId = vGSMRxLevRxQual.SessionId)
Where vGSMRxLevRxQual.RxLevSub<0 and vGSMRxLevRxQual.RxLevSub>-190
and #Session1.CollectionName in ({",".join(["%s"] * len(CollectionName))})
"""
sql += """
Group by
#Session1.Operator
Order by #Session1.Operator ASC
"""
cursor.execute(sql, (CollectionName,))
EDIT: Update to F-string
I'm trying to create a DataFrame through a sql query with pandas read_sql_query method. The query has a where clause that includes a like operation but it also includes a = operation that depends on a variable. The issue is that python is interpreting the % in the like operation as a place holder, just like in the = variable operation which is something I DO want.
Here's an example of it:
sql_string = """ SELECT a,b from table WHERE a = %(variable)s
AND b like '%fixed_chars%' """
params = {'variable':'AA'}
df = pandas.read_sql_query(sql_string, params=params, con=connection)
The error that I get is TypeError: not enough arguments for format string since it interprets the % you usually use as wildcard in mysql as the place holder in python.
In this case, you'll have to use two % for those not being formatting placeholders:
sql_string = "SELECT a,b from table WHERE a = %(variable)s AND \
b like '%%fixed_chars%%'"
Hope this helps!
I have an sqlalchemy core bulk update query that I need to programmatically pass the name of the column that is to be updated.
The function looks as below with comments on each variable:
def update_columns(table_name, pids, column_to_update):
'''
1. table_name: a string denoting the name of the table to be updated
2. pid: a list of primary ids
3. column_to_update: a string representing the name of the column that will be flagged. Sometimes the name can be is_processed or is_active and several more other columns. I thus need to pass the name as a parameter.
'''
for pid in pids:
COL_DICT_UPDATE = {}
COL_DICT_UPDATE['b_id'] = pid
COL_DICT_UPDATE['b_column_to_update'] = True
COL_LIST_UPDATE.append(COL_DICT_UPDATE)
tbl = Table(table_name, meta, autoload=True, autoload_with=Engine)
trans = CONN.begin()
stmt = tbl.update().where(tbl.c.id == bindparam('b_id')).values(tbl.c.column_to_update==bindparam('b_column_to_update'))
trans.commit()
The table parameter gets accepted and works fine.
The column_to_update doesn't work when passed as a parameter. It fails with the error raise AttributeError(key) AttributeError: column_to_mark. If I however hard code the column name, the query runs.
How can I pass the name of the column_to_update for SQLAlchemy to recognize it?
EDIT: Final Script
Thanks to #Paulo, the final script looks like this:
def update_columns(table_name, pids, column_to_update):
for pid in pids:
COL_DICT_UPDATE = {}
COL_DICT_UPDATE['b_id'] = pid
COL_DICT_UPDATE['b_column_to_update'] = True
COL_LIST_UPDATE.append(COL_DICT_UPDATE)
tbl = Table(table_name, meta, autoload=True, autoload_with=Engine)
trans = CONN.begin()
stmt = tbl.update().where(
tbl.c.id == bindparam('b_id')
).values(**{column_to_update: bindparam('b_column_to_update')})
CONN.execute(stmt, COL_LIST_UPDATE)
trans.commit()
I'm not sure if I understood what you want, and your code looks very different from what I consider idiomatic sqlalchemy (I'm not criticizing, just commenting we probably use orthogonal code styles).
If you want to pass a literal column as a parameter use:
from sqlalchemy.sql import literal_column
...
tbl.update().where(
tbl.c.id == bindparam('b_id')
).values({
tbl.c.column_to_update: literal_column('b_column_to_update')
})
If you want to set the right side of the expression dynamically, use:
tbl.update().where(
tbl.c.id == bindparam('b_id')
).values({
getattr(tbl.c, 'column_to_update'): bindparam('b_column_to_update')
})
If none of this is not what you want, comment on the answer or improve your question and I will try to help.
[update]
The values method uses named arguments like .values(column_to_update=value) where column_to_update is the actual column name, not a variable holding the column name. Example:
stmt = users.update().\
where(users.c.id==5).\
values(id=-5)
Note that where uses the comparison operator == while values uses the attribution operator = instead - the former uses the column object in a Boolean expression and the latter uses the column name as a keyword argument binding.
If you need it to be dynamic, use the **kwargs notation: .values(**{'column_to_update': value})
But probably you want to use the values argument instead of the values method.
There is also another simple way:
tbl.c[column_name_here]
I have a table with three columns, cell, trx and type.
This is the query I'm trying to run:
db.execute("SELECT cell,trx FROM tchdrop").fetchall()
It gives the correct output.
However when I try a = ("cell", "trx") and then
db.execute("SELECT ?,? FROM tchdrop", t).fetchall()
the output is [(u'cell', u'trx'), (u'cell', u'trx')] (which is wrong)
I'm doing this to figure out how to extract columns dynamically which is a part of a bigger problem.
The place holder (?) of python DB-API (like sqlite3) don't support columns names to be passed, so you have to use python string formatting like this:
a = ("cell", "trx")
query = "SELECT {0},{1} FROM tchdrop".format(*a)
db.execute(query)
EDIT:
if you don't know the length of the columns that you want to pass , you can do something like this:
a = ("cell", "trx", "foo", "bar")
a = ", ".join(a)
query = "SELECT {0} FROM tchdrop".format(a)
# OUTPUT : 'SELECT cell, trx, foo, bar FROM tchdrop'
db.execute(query)
The library replaces the specified values ("cell", "trx") with their quoted SQL equivalent, so what you get is SELECT "cell", "trx" FROM tchdrop. The result is correct.
What you are trying to achieve is not possible with the ? syntax. Instead, do string replacement yourself. You can check column names with regular expressions (like ^[a-zA-Z_]$) for more security.
For example:
columns = ",".join(("cell", "trx"))
db.execute("SELECT %s FROM tchdrop" % columns).fetchall()