My application uses SQLAlchemy/SQL to query a database. I want to print out the result of the query, but I am getting a <sqlalchemy.engine.result.ResultProxy object in response.
I tried out the suggestions in How to access the results of queries? but I am getting an "Uncaught exception"
See code below:
query = f"SELECT COUNT(DISTINCT id)"\
f"FROM group"
result = db.session.execute(query)
id_count = result.first()[0]
print(id_count)
Try this one:
query = f"SELECT COUNT(DISTINCT id)"\
f"FROM group"
result = db.session.execute(query)
id_count = result.first()
for i in id_count:
print(i[0]) # Access by positional index
print(i['my_column']) # Access by column name as a string
r_dict = dict(i.items()) # convert to dict keyed by column names
Related
I would want to extract specific value from a column in a table but I cannot do that because the way that I do it in python keeps the specific value inside "" instead of ''. So I was wondering if there is any way to do it. In the following you can see the snippet of the query which I have tried.
query = sql.SQL("SELECT {fields} FROM {table} WHEN {col_name} = {var}").format(fields=sql.SQL(',')
.join([sql.Identifier(cols[0]),sql.Identifier(cols[1]),]),table=sql.Identifier(table),col_name=sql
.Identifier(cols[1]),var=sql.Identifier(var))
and the following is the printed query using
print(query.as_string(conn))
SELECT "column1","column2" FROM "table" WHEN "column2" = "string_value"
I was wondering if I could have something like "column2" = 'string_value', which would solve my problem.
Thanks in advance.
var should be a sql.Placeholder, which is going to be inserted by psycopg2 while executing the query:
from psycopg2 import sql
cols = ('c1', 'c2')
table = "foo"
var = "value"
query = sql.SQL("SELECT {fields} FROM {table} WHEN {col_name} = {var}").format(
fields=sql.SQL(',').join(
[
sql.Identifier(cols[0]),
sql.Identifier(cols[1]),
]
),
table=sql.Identifier(table),
col_name=sql.Identifier(cols[1]),
var=sql.Placeholder()
)
print(query.as_string(con))
print(cur.mogrify(query, (var, )).decode('utf-8'))
# cur.execute(query, (var, ))
Out:
SELECT "c1","c2" FROM "foo" WHEN "c2" = %s
SELECT "c1","c2" FROM "foo" WHEN "c2" = 'value'
python & mysql
I am making a query on MySQL database in python module, as follows :
qry = "select qtext,a1,a2,a3,a4,rightanswer from question where qno = 1 ")
mycursor.execute(qry)
myresult = mycursor.fetchone()
qtext.insert('1', myresult[0])
I access the fields by their index number (i.e myresult[0])
my question is how can I access fields by their field-name instead of their index in the query ?
I have to add the following line before executing the query
mycursor = mydb.cursor(dictionary=True)
this line converts the query result to a dictionary that enabled me to access fields by their names names instead of index as follows
qtext.insert('1', myresult["qtext"])
qanswer1.insert('1',myresult["a1"]) # working
qanswer2.insert('1',myresult["a2"]) # working
qanswer3.insert('1',myresult["a3"]) # working
qanswer4.insert('1',myresult["a4"]) # working
r = int(myresult["rightanswer"])
Here is your answer: How to retrieve SQL result column value using column name in Python?
cursor.execute("SELECT name, category FROM animal")
result_set = cursor.fetchall()
for row in result_set:
print "%s, %s" % (row["name"], row["category"])```
I am using bigquery hook in my airflow code.
Query example : select count(*) from 'table-name';
so it will return only 1 integer as a result.
How can I save it in an Integer python variable instead of entire pandas dataframe ?
Below is my code example,
hook = BigQueryHook(bigquery_conn_id=BQ_CON, use_legacy_sql=False)
bq_client = bigquery.Client(project = hook._get_field("project"), credentials = hook._get_credentials())
query = "select count(*) from dataset1.table1;"
df = bq_client.query(query).to_dataframe()
If it is just a single row, you could name the column col1 and access it by this key name
query = "select count(*) as col1 from dataset1.table1;"
query_result = client.query(query)
result = query_result[0]['col1']
or if you have already called to_dataframe()
result = int(df.values[0])
I have a set of IDs (~200k) and I need to get all the rows in a BigQuery Table with those IDs. I tried to construct a list in python and pass it as a parameter to the SQL query using # but I get TypeError: 'ArrayQueryParameter' object is not iterable error. Here is the code I tried (very similar to https://cloud.google.com/bigquery/querying-data#running_parameterized_queries):
id_list = ['id1', 'id2']
query = """
SELECT id
FROM `my-db`
WHERE id in UNNEST(#ids)
"""
query_job = client.run_async_query(
str(uuid.uuid4()),
query,
query_parameters=(
bigquery.ArrayQueryParameter('ids', 'ARRAY<STRING>', id_list)
)
)
Probably the issue here is that you are not passing a tuple to the function.
Try adding a comma before closing the parenthesis, like so:
id_list = ['id1', 'id2']
query = """
SELECT id
FROM `my-db`
WHERE id in UNNEST(#ids)
"""
query_job = client.run_async_query(
str(uuid.uuid4()),
query,
query_parameters=(
bigquery.ArrayQueryParameter('ids', 'STRING', id_list),
)
)
In Python if you do:
t = (1)
and then run:
type(t)
You will find the result to be int. But if you do:
t = (1,)
Then it results in a tuple.
You need to use 'STRING' rather than 'ARRAY<STRING>' for the array element type, e.g.:
query_parameters=(
bigquery.ArrayQueryParameter('ids', 'STRING', id_list)
The example from the querying data topic is:
def query_array_params(gender, states):
client = bigquery.Client()
query = """
SELECT name, sum(number) as count
FROM `bigquery-public-data.usa_names.usa_1910_2013`
WHERE gender = #gender
AND state IN UNNEST(#states)
GROUP BY name
ORDER BY count DESC
LIMIT 10;
"""
query_job = client.run_async_query(
str(uuid.uuid4()),
query,
query_parameters=(
bigquery.ScalarQueryParameter('gender', 'STRING', gender),
bigquery.ArrayQueryParameter('states', 'STRING', states)))
query_job.use_legacy_sql = False
# Start the query and wait for the job to complete.
query_job.begin()
wait_for_job(query_job)
print_results(query_job.results())
Above answers are a better solution but you may find a use for this too whe quickly drafting something in notebooks:
turn a list into a string of date values, comma-separated and in quotes. Then pass the string into the query like so:
id_list = ['id1', 'id2']
# format into a query valid string
id_string = '"'+'","'.join(id_list)+'"'
client = bigquery.Client()
query = f"""
SELECT id
FROM `my-db`
WHERE id in {id_string}
"""
query_job=client.query(query)
results = query_job.result()
If you want to use the simple query like client.query, not client.run_async_query as shown in the answers above. You can to pass an additional parameter QueryJobConfig. Simply add your arrays to query_parameters using bigquery.ArrayQueryParameter.
The following code worked for me:
query = f"""
SELECT distinct pipeline_commit_id, pipeline_id, name
FROM `{self.project_id}.{self.dataset_id}.pipelines_{self.table_suffix}`,
UNNEST(labels) AS label
where label.value IN UNNEST(#labels)
"""
job_config = bigquery.QueryJobConfig(
query_parameters=[
bigquery.ArrayQueryParameter('labels', 'STRING', labels)
]
)
query_job = self.client.query(query, job_config=job_config)
Based on those examples:
https://cloud.google.com/bigquery/docs/parameterized-queries
How to avoid inserting duplicate data? I only want to insert data that does not already exist. I have written following queries but its not working properly. I'm using PostgreSQL.
title_exits = cursor.execute ("SELECT title,pageid FROM movie_movie WHERE title = %s AND pageid = %s;",(title,pageid))
if title_exits == 0:
cursor.execute("INSERT INTO movie_movie (title,pageid,slug,language) values (%s,%s,%s,%s);",(title,pageid,slug,id))
db.commit()
Update: I tried result = cursor.fetchone ("SELECT count(*) FROM movie_movie WHERE title = %s AND pageid = %s;",(title,pageid)). But I'm getting error message. TypeError: fetchone() takes not arugments (2 given).
Answer related to your update:
You should use "%" symbol instead comma:
result = cursor.fetchone ("SELECT count(*) FROM movie_movie WHERE title = %s AND pageid = %s;" % (title,pageid))
update
as #no_freedom said in comments, think better approach would be
result = cursor.fetchone ("SELECT count(*) FROM movie_movie WHERE title = :1 AND pageid = :2", [title,pageid])
But i'm not sure, just try it.
Try to define title field as unique(must define as varchar(constant_length)). Then try insert title into database if title exists, db return error else will insert
As I suspected (and #tony points out) cursor.execute does not return the number of rows. It always return None.