Develop Reference

How to select all values in case of default in drop-down menu in sqlite3 query where clause

I want to display a data in QTableWidget according to QComboBoxes. In case of select all gender or select all ages, I want apply select all in the column in sqlite3 query
I want gender to be all
gender = "select all both male and female"
connection.execute("SELECT * FROM child where region=? and hospital=? and ageInMonths=? and gender=?", (region,hospital,ageInMonths,gender))

Welcome to Stackoverflow.
While it's a little tedious, the most sensible way to attack this problem is to build a list of the conditions you want to apply, and another of the data values that need to be inserted. Something like the following (untested) code, in which I assume that the variables are set to None if they aren't required in the search.
conditions = []
values = []
if region is not None:
conditions.append('region=?')
values.append(region)
# And similar logic for each other value ...
if gender is not None:
conditions.append('gender=?')
values.append(gender)
query = 'SELECT * FROM child'
if conditions:
query = query + ' WHERE ' + ' AND '.join(conditions)
connection.execute(query, values)
This way, if you want to include all values of a column you simply exclude if from the conditions by setting it to None.

You can build your where clause and your parameter list conditionally.
Below I am assuming that the ageInMonths and gender variables actually contain the value 'all' when this is selected on your form. You can change this to whichever value is actually passed to your code, if it is something different.
When it comes to your actual query, the best way to get all values for a field is to simply exclude it from the where clause of your query entirely.
So something like:
query_parameters = []
query_string = "SELECT * FROM child where region=? and hospital=?"
query_parameters.append(region)
query_parameters.append(hospital)
if ageInMonths != 'all':
query_string += " and ageInMonths=?"
query_parameters.append(ageInMonths)
if gender != 'all':
query_string += " and gender=?"
query_parameters.append(gender)
connection.execute(query_string, query_parameters)
Basically, at the same time we are testing and building the dynamic parts of the SQL statement (in query_string), we are also dynamically defining the list of variables to pass to the query in query_parameters, which is a list object.

Filter query by linked object key in SQLAlchemy

Judging by the title this would be the exact same question, but I can't see how any of the answers are applicable to my use case:
I have two classes and a relationship between them:
treatment_association = Table('tr_association', Base.metadata,
Column('chronic_treatments_id', Integer, ForeignKey('chronic_treatments.code')),
Column('animals_id', Integer, ForeignKey('animals.id'))
)
class ChronicTreatment(Base):
__tablename__ = "chronic_treatments"
code = Column(String, primary_key=True)
class Animal(Base):
__tablename__ = "animals"
treatment = relationship("ChronicTreatment", secondary=treatment_association, backref="animals")
I would like to be able to select only the animals which have undergon a treatment which has the code "X". I tried quite a few approaches.
This one fails with an AttributeError:
sql_query = session.query(Animal.treatment).filter(Animal.treatment.code == "chrFlu")
for item in sql_query:
pass
mystring = str(session.query(Animal))
And this one happily returns a list of unfiltered animals:
sql_query = session.query(Animal.treatment).filter(ChronicTreatment.code == "chrFlu")
for item in sql_query:
pass
mystring = str(session.query(Animal))
The closest thing to the example from the aforementioned thread I could put together:
subq = session.query(Animal.id).subquery()
sql_query = session.query(ChronicTreatment).join((subq, subq.c.treatment_id=="chrFlu"))
for item in sql_query:
pass
mystring = str(session.query(Animal))
mydf = pd.read_sql_query(mystring,engine)
Also fails with an AttributeError.
Can you hel me sort this list?

First, there are two issues with table definitions:
1) In the treatment_association you have Integer column pointing to chronic_treatments.code while the code is String column.
I think it's just better to have an integer id in the chronic_treatments, so you don't duplicate the string code in another table and also have a chance to add more fields to chronic_treatments later.
Update: not exactly correct, you still can add more fields, but it will be more complex to change your 'code' if you decide to rename it.
2) In the Animal model you have a relation named treatment. This is confusing because you have many-to-many relation, it should be plural - treatments.
After fixing the above two, it should be clearer why your queries did not work.
This one (I replaced treatment with treatments:
sql_query = session.query(Animal.treatments).filter(
Animal.treatments.code == "chrFlu")
The Animal.treatments represents a many-to-many relation, it is not an SQL Alchemy mode, so you can't pass it to the query nor use in a filter.
Next one can't work for the same reason (you pass Animal.treatments into the query.
The last one is closer, you actually need join to get your results.
I think it is easier to understand the query as SQL (and you anyway need to know SQL to be able to use sqlalchemy):
animals = session.query(Animal).from_statement(text(
"""
select distinct animals.* from animals
left join tr_association assoc on assoc.animals_id = animals.id
left join chronic_treatments on chronic_treatments.id = assoc.chronic_treatments_id
where chronic_treatments.code = :code
""")
).params(code='chrFlu')
It will select animals and join chronic_treatments through the tr_association and filter the result by code.
Having this it is easy to rewrite it using SQL-less syntax:
sql_query = session.query(Animal).join(Animal.treatments).filter(
ChronicTreatment.code == "chrFlu")
That will return what you want - a list of animals who have related chronic treatment with given code.

Iterate List of tables for specific column

I have a list of tables which I would like to iterate and find a specific row based on a foreign key column, then delete it.
This is what my list of tables look like:
subrep_tables = [ TCableResistance.__table__,
TCapacitorBankTest.__table__,
TCapAndDf.__table__,
TMeasuredData.__table__,
TMultiDeviceData.__table__,
TStepVoltage.__table__,
TTemperatureData.__table__,
TTransformerResistance.__table__,
TTransformerTurnsRatio.__table__,
TTripUnit.__table__,
TVectorDiagram.__table__,
TWithstandTest.__table__,
]
I called the list subrep_tables because all of those tables contains a foreign key called ixSubReport.
What I'm trying to do is iterate the list and find all the rows that have a certain sub report and delete those rows instead of going to each table and running the query to delete them(very tedious)
This is what I've come up with thus far.
for report in DBSession.query(TReport).filter(TReport.ixDevice == device_id).all():
for sub_report in DBSession.query(TSubReport).filter(TSubReport.ixReport == report.ixReport).all():
for table in subrep_tables:
for item in DBSession.query(table).filter(table.ixSubReport == sub_report.ixSubReport).all():
print "item: " + str(item)
#DBSession.delete(item)
I'm having some difficulty accessing the table's ixSubReport column for my WHERE clause. The code I have right now gives me an error saying: 'Table' Object has no attribute 'ixSubReport'.
How can I access my iterated table's ixSubReport column to use in my WHERE clause to find the specific row so I can delete it?

If you really want to query the tables, the columns are under the c attribute, use table.c.ixSubReport.
There's no reason to create a list of the __table__ attributes though, just query the models directly. Also, you can avoid a ton of overhead by not performing the first two queries; you can do all this in a single query per model. (This example assumes there are relationships set up between te models).
from sqlalchemy.orm import contains_eager
has_subrep_models = [TCableResistance, TCapacitorBankTest, ...]
# assuming each has_subrep model has a relationship "subrep"
# assuming TSubReport has a relationship "report"
for has_subrep_model in has_subrep_models:
for item in DBSession.query(has_subrep_model).join(has_subrep_model.subrep, TSubReport.report).filter(TReport.ixDevice == device_id).options(contains_eager(has_subrep_model.subrep), contains_eager(TSubReport.report)):
DBSession.delete(item)
This simply joins the related sub report and report when querying each model that has a sub report, and does the filtering on the report's device there. So you end up doing one query per model, rather than 1 + <num reports> + (<num reports> * <num models with sub reports>) = a lot.

Thanks to Denis for the input, I ended up with this :
for report in DBSession.query(TReport).filter(TReport.ixDevice == device_id).all():
for sub_report in DBSession.query(TSubReport).filter(TSubReport.ixReport == report.ixReport).all():
for table in subrep_tables:
for item in DBSession.query(table).filter(table.c.ixSubreport == sub_report.ixSubReport).all():
DBSession.delete(item)

SQLAlchemy execute() return ResultProxy as Tuple, not dict

I have the following code:
query = """
SELECT Coalesce((SELECT sp.param_value
FROM sites_params sp
WHERE sp.param_name = 'ci'
AND sp.site_id = s.id
ORDER BY sp.id DESC
LIMIT 1), -1) AS ci
FROM sites s
WHERE s.deleted = 0
AND s.id = 10
"""
site = db_session.execute(query)
# print site
# <sqlalchemy.engine.result.ResultProxy object at 0x033E63D0>
site = db_session.execute(query).fetchone()
print site # (u'375')
print list(site) # [u'375']
Why does SQLAlchemy return tuples, not dicts, for this query? I want to use the following style to access the results of the query:
print site.ci
# u'375'

This is an old question, but still relevant today. Getting SQL Alchemy to return a dictionary is very useful, especially when working with RESTful based APIs that return JSON.
Here is how I did it using the db_session in Python 3:
resultproxy = db_session.execute(query)
d, a = {}, []
for rowproxy in resultproxy:
# rowproxy.items() returns an array like [(key0, value0), (key1, value1)]
for column, value in rowproxy.items():
# build up the dictionary
d = {**d, **{column: value}}
a.append(d)
The end result is that the array a now contains your query results in dictionary format.
As for how this works in SQL Alchemy:
Thedb_session.execute(query) returns a ResultProxy object
The ResultProxy object is made up of RowProxy objects
The RowProxy object has an .items() method that returns key, value tuples of all the items in the row, which can be unpacked as key, value in a for operation.
And here a one-liner alternative:
[{column: value for column, value in rowproxy.items()} for rowproxy in resultproxy]
From the docs:
class sqlalchemy.engine.RowProxy(parent, row, processors, keymap)
Proxy values from a single cursor row.
Mostly follows “ordered dictionary” behavior, mapping result values to the string-based column name, the integer position of the result in the row, as well as Column instances which can be mapped to the original Columns that produced this result set (for results that correspond to constructed SQL expressions).
has_key(key)
Return True if this RowProxy contains the given key.
items()
Return a list of tuples, each tuple containing a key/value pair.
keys()
Return the list of keys as strings represented by this RowProxy.
Link: http://docs.sqlalchemy.org/en/latest/core/connections.html#sqlalchemy.engine.RowProxy.items

Did you take a look at the ResultProxy docs?
It describes exactly what #Gryphius and #Syed Habib M suggest, namely to use site['ci'].
The ResultProxy does not "return a tuple" as you claim - it is (not surprisingly) a proxy that behaves (e.g. prints) like a tuple but also supports dictionary-like access:
From the docs:
Individual columns may be accessed by their integer position,
case-insensitive column name, or by schema.Column object. e.g.:
row = fetchone()
col1 = row[0] # access via integer position
col2 = row['col2'] # access via name
col3 = row[mytable.c.mycol] # access via Column object.

I've built a simple class to work like a database interface in our processes. Here it goes:
from sqlalchemy import create_engine
class DBConnection:
def __init__(self, db_instance):
self.db_engine = create_engine('your_database_uri_string')
self.db_engine.connect()
def read(self, statement):
"""Executes a read query and returns a list of dicts, whose keys are column names."""
data = self.db_engine.execute(statement).fetchall()
results = []
if len(data)==0:
return results
# results from sqlalchemy are returned as a list of tuples; this procedure converts it into a list of dicts
for row_number, row in enumerate(data):
results.append({})
for column_number, value in enumerate(row):
results[row_number][row.keys()[column_number]] = value
return results

You can easily convert each result row to a dictionary by using dict(site).
Then site['ci'] would be available if ci column is exists.
In order to have site.ci (according to https://stackoverflow.com/a/22084672/487460):
from collections import namedtuple
Site = namedtuple('Site', site.keys())
record = Site(*site)

This may help solve the OPs question. I think the problem he was having is that the row object only contained column values, but not the column names themselves, as is the case with ORM queries where the results have a dict attribute with both keys and values.
python sqlalchemy get column names dynamically?

The easiest way that I found is using list comprehension with calling dict() func on every RowProxy:
site = db_session.execute(query)
result = [dict(row) for row in site]

Based on Essential SQLAlchemy book:
A ResultProxy is a wrapper around a DBAPI cursor object, and its main
goal is to make it easier to use and manipulate the results of a
statement
Simple select example:
from sqlalchemy.sql import select
stmnt = select([cookies])
result_proxy = connection.execute(stmnt)
results = result_proxy.fetchall()
Results going to be like this:
# ID, cookie_name, quantity, amount
[
(1, u'chocolate chip', 12, Decimal('0.50')),
(2, u'dark chocolate chip', 1, Decimal('0.75')),
(3, u'peanut butter', 24, Decimal('0.25')),
(4, u'oatmeal raisin', 100, Decimal('1.00'))
]
It makes handling query results easier by allowing access using an index, name, or Column object.
Accessing cookie_name in different ways:
first_row = results[0]
first_row[1]
first_row.cookie_name
first_row[cookies.c.cookie_name]
These all result in u'chocolate chip' and they each reference the exact same data element in the first record of our results variable. This flexibility in access is only part of the power of the ResultProxy.
We can also leverage the ResultProxy as an iterable:
result_proxy = connection.execute(stmnt)
for record in result_proxy:
print(record.cookie_name)

This method uses list comprehensions, it receives a sql alchemy rowset object and returns the same items as a list of dictionaries:
class ResultHelper():
#classmethod
def resultproxy_to_list(cls, sql_alchemy_rowset):
return [{tuple[0]: tuple[1] for tuple in rowproxy.items()}
for rowproxy in sql_alchemy_rowset]

As you call db.execute(sql).fetchall(), you can easily use the following function to parse the return data to a dict:
def query_to_dict(ret):
if ret is not None:
return [{key: value for key, value in row.items()} for row in ret if row is not None]
else:
return [{}]

How to pass the name of a column as a parameter in SQLAlchemy Core?

I have an sqlalchemy core bulk update query that I need to programmatically pass the name of the column that is to be updated.
The function looks as below with comments on each variable:
def update_columns(table_name, pids, column_to_update):
'''
1. table_name: a string denoting the name of the table to be updated
2. pid: a list of primary ids
3. column_to_update: a string representing the name of the column that will be flagged. Sometimes the name can be is_processed or is_active and several more other columns. I thus need to pass the name as a parameter.
'''
for pid in pids:
COL_DICT_UPDATE = {}
COL_DICT_UPDATE['b_id'] = pid
COL_DICT_UPDATE['b_column_to_update'] = True
COL_LIST_UPDATE.append(COL_DICT_UPDATE)
tbl = Table(table_name, meta, autoload=True, autoload_with=Engine)
trans = CONN.begin()
stmt = tbl.update().where(tbl.c.id == bindparam('b_id')).values(tbl.c.column_to_update==bindparam('b_column_to_update'))
trans.commit()
The table parameter gets accepted and works fine.
The column_to_update doesn't work when passed as a parameter. It fails with the error raise AttributeError(key) AttributeError: column_to_mark. If I however hard code the column name, the query runs.
How can I pass the name of the column_to_update for SQLAlchemy to recognize it?
EDIT: Final Script
Thanks to #Paulo, the final script looks like this:
def update_columns(table_name, pids, column_to_update):
for pid in pids:
COL_DICT_UPDATE = {}
COL_DICT_UPDATE['b_id'] = pid
COL_DICT_UPDATE['b_column_to_update'] = True
COL_LIST_UPDATE.append(COL_DICT_UPDATE)
tbl = Table(table_name, meta, autoload=True, autoload_with=Engine)
trans = CONN.begin()
stmt = tbl.update().where(
tbl.c.id == bindparam('b_id')
).values(**{column_to_update: bindparam('b_column_to_update')})
CONN.execute(stmt, COL_LIST_UPDATE)
trans.commit()

I'm not sure if I understood what you want, and your code looks very different from what I consider idiomatic sqlalchemy (I'm not criticizing, just commenting we probably use orthogonal code styles).
If you want to pass a literal column as a parameter use:
from sqlalchemy.sql import literal_column
...
tbl.update().where(
tbl.c.id == bindparam('b_id')
).values({
tbl.c.column_to_update: literal_column('b_column_to_update')
})
If you want to set the right side of the expression dynamically, use:
tbl.update().where(
tbl.c.id == bindparam('b_id')
).values({
getattr(tbl.c, 'column_to_update'): bindparam('b_column_to_update')
})
If none of this is not what you want, comment on the answer or improve your question and I will try to help.
[update]
The values method uses named arguments like .values(column_to_update=value) where column_to_update is the actual column name, not a variable holding the column name. Example:
stmt = users.update().\
where(users.c.id==5).\
values(id=-5)
Note that where uses the comparison operator == while values uses the attribution operator = instead - the former uses the column object in a Boolean expression and the latter uses the column name as a keyword argument binding.
If you need it to be dynamic, use the **kwargs notation: .values(**{'column_to_update': value})
But probably you want to use the values argument instead of the values method.

There is also another simple way:
tbl.c[column_name_here]