Develop Reference

Python MySQL connector SELECT statement issue

When I'm trying to format an SQL statement this way:
cursor.execute('SELECT (%s) FROM table WHERE id = 12345', (column,))
it doesn't work properly. Instead of returning values from the specified column, it just returns the name of the column. Any way to fix that?
It only works with formatted string, but according to reviews it is not the safest approach

You can't bind column names like that -you're binding a string literal with that name.
If you want to dynamically set the column names, you'll have to resort to string manipulation of some sort, e.g.:
cursor.execute(f'SELECT {column} FROM table WHERE id = 12345')
Of course if column is created by user-input, you'll have to sanitize it somehow.

Python string formatting for an SQL insert statement with unknown variables

Although there are various similar questions around this topic, I can't find one that answers my problem.
I am using pscopg to build and insert data into a postgresql database, where the insert statement exists inside a for loop, and therefore will be different each iteration.
insert_string = sql.SQL(
"INSERT INTO {region}(id, price, house_type) VALUES ({id}, {price}, {house_type})").format(
region=sql.Literal(region),
id=sql.Literal(str(id)),
price=sql.Literal(price),
house_type=sql.Literal(house_type))
cur.execute(insert_string)
The variables region, id, price, house_type are all defined somewhere else inside said for loop.
The error I'm getting is as follows:
psycopg2.errors.SyntaxError: syntax error at or near "'Gorton'"
LINE 1: INSERT INTO 'Gorton'(id, price, house_typ...
^
where 'Gorton' is the value of the variable 'region' at that particular iteration.
From what I can see psycopg seems to be struggling with the apostrophe around Gorton, calling it a syntax error.
I have read the docs and can't figure out if sql.Literal is the correct choice here, or if I should use sql.Identifier instead?
Thanks in advance

How to loop through dataframe and execute cursor to update database row by row

Hi apologies if this is a dumb question, super new to python.
Context: I have a dataframe with values that need to be updated into a table in a database. The idea is to loop through each row, pass a required value from each row to the SQL string, execute sql string.
What i have tried so far:
sales_output = merge_view[['SALES_FORECAST_ID','SALES_DATE','WEEK_NO','ADJUSTED_BASE']]
cursor = sql_connection.cursor()
for row in sales_output:
sales = row['ADJUSTED_BASE']
fid = row['SALES_FORECAST_ID']
date = row['SALES_DATE']
print('''UPDATE a SET a.WEEKLY_SALES =? FROM MA.SALES_FORECAST_VW a WHERE a.SALES_FORECAST_ID =? AND
SALES_DATE =?''',(sales,fid,date))
I am using a print statement to test if this is producing the correct SQL string to update.
This doesn't work as i get a TypeError: string indices must be, i think i understand why. you cannot select columns like that in a for loop but I'm not sure what the alternative is. I've done some googling but I think I may have confused myself.
Any help is much appreciated.
Cheers

Substituting column names in Python sqlite3 query [duplicate]

This question already has answers here:
How do you escape strings for SQLite table/column names in Python?
(8 answers)
Closed 7 years ago.
I have a wide table in a sqlite3 database, and I wish to dynamically query certain columns in a Python script. I know that it's bad to inject parameters by string concatenation, so I tried to use parameter substitution instead.
I find that, when I use parameter substitution to supply a column name, I get unexpected results. A minimal example:
import sqlite3 as lite
db = lite.connect("mre.sqlite")
c = db.cursor()
# Insert some dummy rows
c.execute("CREATE TABLE trouble (value real)")
c.execute("INSERT INTO trouble (value) VALUES (2)")
c.execute("INSERT INTO trouble (value) VALUES (4)")
db.commit()
for row in c.execute("SELECT AVG(value) FROM trouble"):
print row # Returns 3
for row in c.execute("SELECT AVG(:name) FROM trouble", {"name" : "value"}):
print row # Returns 0
db.close()
Is there a better way to accomplish this than simply injecting a column name into a string and running it?

As Rob just indicated in his comment, there was a related SO post that contains my answer. These substitution constructions are called "placeholders," which is why I did not find the answer on SO initially. There is no placeholder pattern for column names, because dynamically specifying columns is not a code safety issue:
It comes down to what "safe" means. The conventional wisdom is that
using normal python string manipulation to put values into your
queries is not "safe". This is because there are all sorts of things
that can go wrong if you do that, and such data very often comes from
the user and is not in your control. You need a 100% reliable way of
escaping these values properly so that a user cannot inject SQL in a
data value and have the database execute it. So the library writers do
this job; you never should.
If, however, you're writing generic helper code to operate on things
in databases, then these considerations don't apply as much. You are
implicitly giving anyone who can call such code access to everything
in the database; that's the point of the helper code. So now the
safety concern is making sure that user-generated data can never be
used in such code. This is a general security issue in coding, and is
just the same problem as blindly execing a user-input string. It's a
distinct issue from inserting values into your queries, because there
you want to be able to safely handle user-input data.
So, the solution is that there is no problem in the first place: inject the values using string formatting, be happy, and move on with your life.

Why not use string formatting?
for row in c.execute("SELECT AVG({name}) FROM trouble".format(**{"name" : "value"})):
print row # => (3.0,)

Python: using pyodbc and replacing row field values

I'm trying to figure out if it's possible to replace record values in a Microsoft Access (either .accdb or .mdb) database using pyodbc. I've poured over the documentation and noted where it says that "Row Values Can Be Replaced" but I have not been able to make it work.
More specifically, I'm attempting to replace a row value from a python variable. I've tried:
setting the connection autocommit to "True"
made sure that it's not a data type issue
Here is a snippet of the code where I'm executing a SQL query, using fetchone() to grab just one record (I know with this script the query is only returning one record), then I am grabbing the existing value for a field (the field position integer is stored in the z variable), and then am getting the new value I want to write to the field by accessing it from an existing python dictionary created in the script.
pSQL = "SELECT * FROM %s WHERE %s = '%s'" % (reviewTBL, newID, basinID)
cursor.execute(pSQL)
record = cursor.fetchone()
if record:
oldVal = record[z]
val = codeCrosswalk[oldVal]
record[z] = val
I've tried everything I can think bit cannot get it to work. Am I just misunderstanding the help documentation?
The script runs successfully but the newly assigned value never seems to commit. I even tried putting "print str(record[z])this after the record[z] = val line to see if the field in the table has the new value and the new value would print like it worked...but then if I check in the table after the script has finished the old values are still in the table field.
Much appreciate any insight into this...I was hoping this would work like how using VBA in MS Access databases you can use an ADO Recordset to loop through records in a table and assign values to a field from a variable.
thanks,
Tom

The "Row values can be replaced" from the pyodbc documentation refers to the fact that you can modify the values on the returned row objects, for example to perform some cleanup or conversion before you start using them. It does not mean that these changes will automatically be persisted in the database. You will have to use sql UPDATE statements for that.