How could i use psycopg2 query parameters with on conflict?
for example, i have this data:
#This is the data for the insert query
q_values = {'customer_app_id': '35', 'email': 'my#email.com', 'access_counter': 1, 'company_name': 'twitter', 'status': 'start'}
#This is the data for the on conflict query part
conflict_values = {'access_counter': +1, 'status': 'check', 'status_info': 'already exist'}
This is the query:
insert into access (customer_app_id,email,access_counter,company_name,status)
values (%s,%s,%s,%s,%s) ON CONFLICT (customer_app_id,email) DO UPDATE SET
%s,%s,%s RETURNING *
Then i run this line:
q_values = q_values.update(conflict_values)
cursor.execute(query, q_values)
First, how at all to run on conflict with query parameters ?
Second, the update i am doing with the dict is not good since if it will be duplicate keys it will merge them, and then number of parameters wont be equal to number of values.
and access_counter +1 - on conflict i'm trying to increase the access _counter by 1
please can you help ?
Thank you!
To run on conflict with query parameter:
The %s has to be replaced by %(key value that you want to access)s. This has to be done since you are using a dictionary. So in this case the query would look as follows-
insert into access (customer_app_id,email,access_counter,company_name,status)
values (%(customer_app_id)s,%(email)s,%(access_counter)s,%(company_name)s,%(status)s) ON CONFLICT
To update the access counter:
A separate dictionary need not be used. Rather ON CONFLICT you can directly make changes in the database. So in this case the final query would look as follows-
insert into access (customer_app_id,email,access_counter,company_name,status)
values (%(customer_app_id)s,%(email)s,%(access_counter)s,%(company_name)s,%(status)s) ON CONFLICT (customer_app_id,email) DO UPDATE SET access_counter = access.access_counter + 1
In case you want to update one of your column values to a new value sent in the INSERT query then you will have to use EXCLUDED keyword.
Example(this example is not related to the question):
INSERT INTO user_details (user_id, username,email,last_login)
VALUES (%s, %s, %s,%s)
ON CONFLICT (user_id) DO UPDATE
SET last_login = EXCLUDED.last_login
As you are passing a dictionary of values (instead of a list or a tuple), I think you need to use %(name)s placeholders, not %s.
This means you can give the placeholders alternative names, so they don't conflict in the q_values dictionary.
See the documentation for details.
Related
I have a table name globalData, in my sqlite database, with column 'index', 'rank_d30'. While executing this query with python, i receive syntax error near ON ...
cur.execute('INSERT INTO globalData (`index`, rank_d30) VALUES(0, 9) ON CONFLICT(`index`) DO UPDATE SET rank_d30 = VALUES(rank_d30)')
How it can be corrected?
SQLite does not recognize the VALUES() syntax in the DO UPDATE clause of the query, as MySQL does in the INSERT ... ON DUPLICATE KEY syntax. To refer to the value that was initially given for insert, you must use pseudo-table excluded instead.
Consider:
INSERT INTO globalData (`index`, rank_d30)
VALUES(0, 9)
ON CONFLICT(`index`) DO UPDATE SET rank_d30 = EXCLUDED.rank_d30
Note that, for this to work, you need a unique or primary key constraint on column index.
How can I dynamically build an update query where the number and field names I'm updating are different without having to set up a different update query for every single update statement.
I'm new to python, but in other languages I've seen it be done with a key value pair where you would dynamically do something like:
UPDATE my_table SET key1 = value1 WHERE key2 = value2
Then you'd pass an array of key value pairs to the function and off you go.
Is the best way to do this in python to just create an update string and pass in the fields? Something like:
"UPDATE my_table SET {} = ? WHERE {} = ?".format(key1, key2)
Then I guess you'd have to separately pass in the parameters to pyodbc.executemany?
But then I'm not sure how you'd handle a variable number of fields to update. I'm sure there's a way to do this easily, so hopefully someone can clue me in.
Yes, with vanilla Python and pyodbc the basic process you've described is correct. If you have a list of column names to update
>>> col_names_to_set = ['LastName', 'FirstName']
you can build the required items for the SET clause,
>>> set_tokens = ','.join([f'{x}=?' for x in col_names_to_set])
>>> set_tokens
'LastName=?,FirstName=?'
include that in your SQL command text,
>>> sql = f"UPDATE TableName SET {set_tokens} WHERE ..."
>>> sql
'UPDATE TableName SET LastName=?,FirstName=? WHERE ...'
and then pass that statement to executemany along with your list of tuples containing the values to be updated.
Note that for safety you probably should enclose the column names in the delimiters for your SQL dialect, e.g., for SQL Server (T-SQL):
>>> set_tokens = ','.join([f'[{x}]=?' for x in col_names_to_set])
>>> set_tokens
'[LastName]=?,[FirstName]=?'
I am trying to load the data inside the table trial and it says Invalid Column name - Name.
I am passing values inside Name and Area dynamically.
cursor.execute("insert into trial (NameofTheProperty, AreaofTheProperty)
values (Name, Area)")
cnxn.commit()
You need to have quotes around the column values so that they are not gonna be interpreted as column names instead:
insert into
trial (NameofTheProperty, AreaofTheProperty)
values
("Name", "Area")
Now, since you mentioned that you dynamically insert these values into the query, you can just let your database driver handle the quotes and other things like type conversions:
property_name = "Name"
property_area = "Area"
cursor.execute("""
insert into
trial (NameofTheProperty, AreaofTheProperty)
values
(?, ?)""", (property_name, property_area))
cnxn.commit()
This is called query parameterization and is considered the safest and the most robust way to insert values into the SQL queries. These ? values are called "placeholders".
Note that the database driver is gonna put quotes around the string values automatically - no need to do it manually.
I have a working MySQL query in python using (?) place holders, my problem is when I add a WHERE clause in the query, with an extra placeholder (?) SQL won't allow it, I think because it groups placeholders so won't allow the 6th placeholder in the where clause.
Here's my code snippet:
self.query.prepare("UPDATE tbl_dev (user_id, dev_ref,dev_mac,dev_ka_der_file_path,dev_ds_der_file_path,dev_type) VALUES (?,?,?,?,?,?) WHERE id=?;")
self.query.bindValue(0, userID)
self.query.bindValue(1, dev_id)
self.query.bindValue(2, dev_mac)
self.query.bindValue(3, dev_ka_cert)
self.query.bindValue(4, dev_ds_cert)
self.query.bindValue(5, dev_type)
self.query.bindValue(6, devID)
self.query.exec_()
Basically it won't except the final devID placeholder as I think it's grouping the 6th (devID) with the first group of placeholders. Is this even possible in Python/MySQL?
There's an incorrect syntax as pointed out by #vkp, the working solution was with a select clause, with an update clause there are no grouped values. Solution is below.
self.query.prepare("UPDATE tbl_dev SET user_id=?,dev_ref=?,dev_mac=?,dev_ka_der_file_path=?,dev_ds_der_file_path=?,dev_type=? WHERE id=?;")
self.query.bindValue(0, userID)
self.query.bindValue(1, dev_id)
self.query.bindValue(2, dev_mac)
self.query.bindValue(3, dev_ka_cert)
self.query.bindValue(4, dev_ds_cert)
self.query.bindValue(5, dev_type)
self.query.bindValue(6, devID)
self.query.exec_()
What would be the best way to get the PK of the following:
self.cursor.execute('INSERT IGNORE INTO table (url, country) VALUES (%s, %s)', (line['url'], line['country']))
In other words, if it's already there, I would need to get that PK, but if it's not there, it would be INSERTing and then getting the LAST_INSERT_ID. Is there a way to do this without doing three queries? What would be the best way to do this pattern?
To get the LAST_INSERT_ID while inserting data, don't use INSERT IGNORE. Instead, use the ON DUPLICATE KEY UPDATE clause to get the id:
INSERT INTO table (url, country)
VALUES (%s, %s)
ON DUPLICATE KEY
UPDATE
id = LAST_INSERT_ID(id);
where id represents the unique column of your table.
You'd still need another query to fetch the updated LAST_INSERT_ID now.
I think the most straightforward way to do this without altering previous data would be to do an INSERT IGNORE followed by a SELECT to retrieve the id.
cursor.execute('INSERT IGNORE INTO...')
cursor,execute('SELECT id FROM table...')
id = cursor.fetchone()[0]