Develop Reference

in MySQL and python how to access fields by using field name not field index

python & mysql
I am making a query on MySQL database in python module, as follows :
qry = "select qtext,a1,a2,a3,a4,rightanswer from question where qno = 1 ")
mycursor.execute(qry)
myresult = mycursor.fetchone()
qtext.insert('1', myresult[0])
I access the fields by their index number (i.e myresult[0])
my question is how can I access fields by their field-name instead of their index in the query ?

I have to add the following line before executing the query
mycursor = mydb.cursor(dictionary=True)
this line converts the query result to a dictionary that enabled me to access fields by their names names instead of index as follows
qtext.insert('1', myresult["qtext"])
qanswer1.insert('1',myresult["a1"]) # working
qanswer2.insert('1',myresult["a2"]) # working
qanswer3.insert('1',myresult["a3"]) # working
qanswer4.insert('1',myresult["a4"]) # working
r = int(myresult["rightanswer"])

Here is your answer: How to retrieve SQL result column value using column name in Python?
cursor.execute("SELECT name, category FROM animal")
result_set = cursor.fetchall()
for row in result_set:
print "%s, %s" % (row["name"], row["category"])```

Why MySql connection not working on Flask Python API

I was using sqlite however moved to MySqL
Sqlite code that worked:
#app.route('/api/v1/users/user', methods=['GET'])
def api_filter():
query_parameters = request.args
userid = query_parameters.get('userid')
username = query_parameters.get('username')
query = "SELECT * FROM tblUser WHERE"
to_filter = []
if userid:
query += ' user_id=? AND'
to_filter.append(userid)
if username:
query += ' username=? AND'
to_filter.append(username)
if not (userid or username):
return page_not_found(404)
query = query[:-4] + ';'
conn = sqlite3.connect('dbApp.db')
conn.row_factory = dict_factory
cur = conn.cursor()
results = cur.execute(query, to_filter).fetchall()
return jsonify(results)
Now when I change it to MySQL it does not work:
#app.route('/api/v1/users/user', methods=['GET'])
def api_filter():
query_parameters = request.args
userid = query_parameters.get('userid')
username = query_parameters.get('username')
query = "SELECT * FROM tblUser WHERE"
to_filter = []
if userid:
query += ' user_id=? AND'
to_filter.append(userid)
if username:
query += ' username=? AND'
to_filter.append(username)
if not (userid or username):
return page_not_found(404)
query = query[:-4] + ';'
conn = mysql.connector.connect(host="localhost", user="root", password="green", database="dbApp")
conn.row_factory = dict_factory
cur = conn.cursor()
results = cur.execute(query, to_filter).fetchall()
return jsonify(results)
The error returned:
results = cur.execute(query, to_filter).fetchall()
File "/Library/Frameworks/Python.framework/Versions/3.8/lib/python3.8/site-packages/mysql/connector/cursor_cext.py", line 260, in execute
raise errors.ProgrammingError(
mysql.connector.errors.ProgrammingError: Not all parameters were used in the SQL statement
127.0.0.1 - - [28/Dec/2020 17:47:11] "
This is after running:
http://127.0.0.1:5000/api/v1/users/user?userid=3&username=Jo
http://127.0.0.1:5000/api/v1/users/user?userid=3
http://127.0.0.1:5000/api/v1/users/user?username=Jo
Also added this:
from flask import request, jsonify
import mysql.connector

Here is what the documentation says:
cursor.execute(operation, params=None, multi=False)
This method executes the given database operation (query or command).
The parameters found in the tuple or dictionary params are bound to
the variables in the operation. Specify variables using %s or %(name)s
parameter style (that is, using format or pyformat style). execute()
returns an iterator if multi is True.
I think there are two problems with your code: you are using ? for bind variables instead of %s, and you are passing in a list instead of a tuple. Alternatively use named parameters and pass in the values with a dict. Try both and see what you like most.
user_id = %s" and to_filter = (userid, ). You can use the list you have now and just convert it with tuple().
" user_id = %{user_id}" and set to_filter = {'user_id': userid }.
It is cleaner to build a list of conditions, then ' AND '.join(conditions) instead of removing the extra AND. You can simplify the code a little bit by just leaving out the ';'.

use row as variable with python and sql

I am trying to update some values into a database. The user can give the row that should be changed. The input from the user, however is a string. When I try to parse this into the MySQL connector with python it gives an error because of the apostrophes. The code I have so far is:
import mysql.connector
conn = mysql.connector
conn = connector.connect(user=dbUser, password=dbPasswd, host=dbHost, database=dbName)
cursor = conn.cursor()
cursor.execute("""UPDATE Search SET %s = %s WHERE searchID = %s""", ('maxPrice', 300, 10,))
I get this error
mysql.connector.errors.ProgrammingError: 1064 (42000): You have an error in your SQL syntax; check the manual that corresponds to your MySQL server version for the right syntax to use near ''maxPrice' = 300 WHERE searchID = 10' at line 1
How do I get rid of the apostrophes? Because I think they are causing problems.

As noted, you can't prepare it using a field.
Perhaps the safest way is to allow only those fields that are expected, e.g.
#!/usr/bin/python
import os
import mysql.connector
conn = mysql.connector.connect(user=os.environ.get('USER'),
host='localhost',
database='sandbox',
unix_socket='/var/run/mysqld/mysqld.sock')
cur = conn.cursor(dictionary=True)
query = """SELECT column_name
FROM information_schema.columns
WHERE table_schema = DATABASE()
AND table_name = 'Search'
"""
cur.execute(query)
fields = [x['column_name'] for x in cur.fetchall()]
user_input = ['maxPrice', 300, 10]
if user_input[0] in fields:
cur.execute("""UPDATE Search SET {0} = {1} WHERE id = {1}""".format(user_input[0], '%s'),
tuple(user_input[1:]))
print cur.statement
Prints:
UPDATE Search SET maxPrice = 300 WHERE id = 10
Where:
mysql> show create table Search\G
*************************** 1. row ***************************
Search
CREATE TABLE `Search` (
`id` int(11) DEFAULT NULL,
`maxPrice` float DEFAULT NULL
) ENGINE=InnoDB DEFAULT CHARSET=latin1

A column name is not a parameter. Put the column name maxPrice directly into your SQL.
cursor.execute("""UPDATE Search SET maxPrice = %s WHERE searchID = %s""", (300, 10))
If you want to use the same code with different column names, you would have to modify the string itself.
sql = "UPDATE Search SET {} = %s WHERE searchID = %s".format('maxPrice')
cursor.execute(sql, (300,10))
But bear in mind that this is not safe from injection the way parameters are, so make sure your column name is not a user-input string or anything like that.

You cannot do it like that. You need to place the column name in the string before you call cursor.execute. Column names cannot be used when transforming variables in cursor.execute.
Something like this would work:
sql = "UPDATE Search SET {} = %s WHERE searchID = %s".format('maxPrice')
cursor.execute(sql, (300, 10,))

You cannot dynamically bind object (e.g., column) names, only values. If that's the logic you're trying to achieve, you'd have to resort to string manipulation/formatting (with all the risks of SQL-injection attacks that come with it). E.g.:
sql = """UPDATE Search SET {} = %s WHERE searchID = %s""".format('maxPrice')
cursor.execute(sql, (300, 10,))

Passing Array Parameter to SQL for BigQuery in Python

I have a set of IDs (~200k) and I need to get all the rows in a BigQuery Table with those IDs. I tried to construct a list in python and pass it as a parameter to the SQL query using # but I get TypeError: 'ArrayQueryParameter' object is not iterable error. Here is the code I tried (very similar to https://cloud.google.com/bigquery/querying-data#running_parameterized_queries):
id_list = ['id1', 'id2']
query = """
SELECT id
FROM `my-db`
WHERE id in UNNEST(#ids)
"""
query_job = client.run_async_query(
str(uuid.uuid4()),
query,
query_parameters=(
bigquery.ArrayQueryParameter('ids', 'ARRAY<STRING>', id_list)
)
)

Probably the issue here is that you are not passing a tuple to the function.
Try adding a comma before closing the parenthesis, like so:
id_list = ['id1', 'id2']
query = """
SELECT id
FROM `my-db`
WHERE id in UNNEST(#ids)
"""
query_job = client.run_async_query(
str(uuid.uuid4()),
query,
query_parameters=(
bigquery.ArrayQueryParameter('ids', 'STRING', id_list),
)
)
In Python if you do:
t = (1)
and then run:
type(t)
You will find the result to be int. But if you do:
t = (1,)
Then it results in a tuple.

You need to use 'STRING' rather than 'ARRAY<STRING>' for the array element type, e.g.:
query_parameters=(
bigquery.ArrayQueryParameter('ids', 'STRING', id_list)
The example from the querying data topic is:
def query_array_params(gender, states):
client = bigquery.Client()
query = """
SELECT name, sum(number) as count
FROM `bigquery-public-data.usa_names.usa_1910_2013`
WHERE gender = #gender
AND state IN UNNEST(#states)
GROUP BY name
ORDER BY count DESC
LIMIT 10;
"""
query_job = client.run_async_query(
str(uuid.uuid4()),
query,
query_parameters=(
bigquery.ScalarQueryParameter('gender', 'STRING', gender),
bigquery.ArrayQueryParameter('states', 'STRING', states)))
query_job.use_legacy_sql = False
# Start the query and wait for the job to complete.
query_job.begin()
wait_for_job(query_job)
print_results(query_job.results())

Above answers are a better solution but you may find a use for this too whe quickly drafting something in notebooks:
turn a list into a string of date values, comma-separated and in quotes. Then pass the string into the query like so:
id_list = ['id1', 'id2']
# format into a query valid string
id_string = '"'+'","'.join(id_list)+'"'
client = bigquery.Client()
query = f"""
SELECT id
FROM `my-db`
WHERE id in {id_string}
"""
query_job=client.query(query)
results = query_job.result()

If you want to use the simple query like client.query, not client.run_async_query as shown in the answers above. You can to pass an additional parameter QueryJobConfig. Simply add your arrays to query_parameters using bigquery.ArrayQueryParameter.
The following code worked for me:
query = f"""
SELECT distinct pipeline_commit_id, pipeline_id, name
FROM `{self.project_id}.{self.dataset_id}.pipelines_{self.table_suffix}`,
UNNEST(labels) AS label
where label.value IN UNNEST(#labels)
"""
job_config = bigquery.QueryJobConfig(
query_parameters=[
bigquery.ArrayQueryParameter('labels', 'STRING', labels)
]
)
query_job = self.client.query(query, job_config=job_config)
Based on those examples:
https://cloud.google.com/bigquery/docs/parameterized-queries

In python, changing MySQL query based on function variables

I'd like to be able to add a restriction to the query if user_id != None ... for example:
"AND user_id = 5"
but I am not sure how to add this into the below function?
Thank you.
def get(id, user_id=None):
query = """SELECT *
FROM USERS
WHERE text LIKE %s AND
id = %s
"""
values = (search_text, id)
results = DB.get(query, values)
This way I can call:
get(5)
get(5,103524234) (contains user_id restriction)

def get(id, user_id=None):
query = """SELECT *
FROM USERS
WHERE text LIKE %s AND
id = %s
"""
values = [search_text, id]
if user_id is not None:
query += ' AND user_id = %s'
values.append(user_id)
results = DB.get(query, values)
As you see, the main difference wrt your original code is the small if block in the middle, which enriches query string and values if needed. I also made values a list, rather than a tuple, so it can be enriched with the more natural append rather than with
values += (user_id,)
which is arguably less readable - however, you can use it if you want to keep values a tuple for some other reasons.
edit: the OP now clarifies in a comment (!) that his original query has an ending LIMIT clause. In this case I would suggest a different approach, such as:
query_pieces = ["""SELECT *
FROM USERS
WHERE text LIKE %s AND
id = %s
""", "LIMIT 5"]
values = [search_text, id]
if user_id is not None:
query_pieces.insert(1, ' AND user_id = %s')
values.append(user_id)
query = ' '.join(query_pieces)
results = DB.get(query, values)
You could do it in other ways, but keeping a list of query pieces in the proper order, enriching it as you go (e.g. by insert), and joining it with some whitespace at the end, is a pretty general and usable approach.

What's wrong with something like:
def get(id, user_id=None):
query = "SELECT * FROM USERS WHERE text LIKE %s"
if user_id != None:
query = query + " AND id = %s"%(user_id)
:
:
That syntax may not be perfect, I haven't done Python for a while - I'm just trying to get the basic idea across. This defaults to the None case and only adds the extra restriction if you give a real user ID.

You could build the SQL query using a list of conditions:
def get(id, user_id=None):
query = """SELECT *
FROM USERS
WHERE
"""
values = [search_text, id]
conditions=[
'text LIKE %s',
'id = %s']
if user_id is not None:
conditions.append('user_id = %s')
values.append(user_id)
query+=' AND '.join(conditions)+' LIMIT 1'
results = DB.get(query, values)