I am trying to use setFilter method in a custom class extended from QSqlTableModel class. I used the same method before for other classes and it worked well but this time whatever filter I apply, I get 0 results. Even when I do:
self.setFilter("")
I get 0 results. However, for other classes I used to use this line to reset existing filters meaning I'm supposed to return all the objects in the table as a result.
Note: If i dont use any filters, I can retrieve all the objects properly.
Here is my code:
I retrieve the data from a csv file:
file = open(fileName, 'r', encoding="utf-8")
csvreader = csv.reader(file)
rows = []
for row in csvreader:
rows.append(row)
file.close()
return rows
Creation of the table:
createTableQ= QSqlQuery()
createTableQ.exec_(
"""
CREATE TABLE cities (
id INTEGER PRIMARY KEY UNIQUE NOT NULL,
name VARCHAR(100) NOT NULL,
countryId INTEGER,
countryCode VARCHAR(100) NOT NULL,
countryName VARCHAR(100) NOT NULL
)
"""
)
createTableQ.finish()
Then apply a few preprocessing on the data then insert them in the sqlite file:
cities=convertToValue(cities)
for city in cities:
insertDataQ=QSqlQuery()
if not insertDataQ.exec_(city):
print(insertDataQ.lastError().text())
insertDataQ.finish()
convertToValue method takes the data and formats it in a SQL insert query format. That function is not the problem I use it in another class where I can use filter.
The above methods work only once to create the sqlite file anyway.
My class (NOTE: setCityData method is for the sqlite file creation i explained above, runs only once if the file does not exist):
class CityListModel(QSqlTableModel):
def __init__(self, parent=None):
super(CityListModel, self).__init__(parent)
setcityData()
self.setTable(table_name)
self.setSort(1, Qt.AscendingOrder)
self.setEditStrategy(QSqlTableModel.OnFieldChange)
self.recipient = ""
self.select()
while self.canFetchMore():
self.fetchMore()
def data(self, index, role):
if role < Qt.UserRole:
return QSqlTableModel.data(self, index, role)
sql_record = QSqlRecord()
sql_record = self.record(index.row())
return sql_record.value(role - Qt.UserRole)
def roleNames(self):
"""Converts dict to hash because that's the result expected
by QSqlTableModel"""
names = {}
id = "id".encode()
name = "name".encode()
countryId = "countryId".encode()
countryCode = "countryCode".encode()
countryName = "countryName".encode()
names[hash(Qt.UserRole + 0)] = id
names[hash(Qt.UserRole + 1)] = name
names[hash(Qt.UserRole + 2)] = countryId
names[hash(Qt.UserRole + 3)] = countryCode
names[hash(Qt.UserRole + 4)] = countryName
return names
My filter method which returns 0 results with any filter text:
#Slot(str)
def applyFilter(self, filterCountryName):
#self.setFilter("countryName = 'Turkey'")
self.setFilter("")
Edit: I call the function from qml side the same way I do for my other custom classes. Where I call it is irrelevant but basically this will be the end product:
CustomSelector{
id:countryList
Layout.fillHeight: true
Layout.fillWidth: true
isLocked: false
model:countryListModel
shownItemCount:4
selectedItem.onTextChanged: {
cityListModel.applyFilter(countryList.selectedItem.text)
}
}
While trying to give a reproducable example I realized the connection to my sqlite database gets broken somewhere before trying to set the filter. Therefore, setFilter can never succeed.
I am trying to build an accounting database using flask as the front end. The main page is the ledger, with nine columns "date" "description" "debit" "credit" "amount" "account" "reference" "journal" and "year", I need to be able to query each and some times two at once, there are over 8000 entries, and growing. My code so far displays all the rows, 200 at a time with pagination, I have read "pep 8" which talks about readable code, I have read this multiple parameters and this multiple parameters and like the idea of using
request.args.get
But I need to display all the rows until I query, I have also looked at this nested ifs and I thought perhaps I could use a function for each query and "If" out side of the view function and then call each in the view function, but I am not sure how to. Or I could have a view function for each query. But I am not sure how that would work, here is my code so far,
#bp.route('/books', methods=['GET', 'POST'])
#bp.route('/books/<int:page_num>', methods=['GET', 'POST'])
#bp.route('/books/<int:page_num>/<int:id>', methods=['GET', 'POST'])
#bp.route('/books/<int:page_num>/<int:id>/<ref>', methods=['GET', 'POST'])
#login_required
def books(page_num, id=None, ref=None):
if ref is not None:
books = Book.query.order_by(Book.date.desc()).filter(Book.REF==ref).paginate(per_page=100, page=page_num, error_out=True)
else:
books = Book.query.order_by(Book.date.desc()).paginate(per_page=100, page=page_num, error_out=True)
if id is not None:
obj = Book.query.get(id) or Book()
form = AddBookForm(request.form, obj=obj)
if form.validate_on_submit():
form.populate_obj(obj)
db.session.add(obj)
db.session.commit()
return redirect(url_for('books.books'))
else:
form = AddBookForm()
if form.validate_on_submit():
obj = Book(id=form.id.data, date=form.date.data, description=form.description.data, debit=form.debit.data,\
credit=form.credit.data, montant=form.montant.data, AUX=form.AUX.data, TP=form.TP.data,\
REF=form.REF.data, JN=form.JN.data, PID=form.PID.data, CT=form.CT.data)
db.session.add(obj)
db.session.commit()
return redirect(url_for('books.books', page_num=1))
return render_template('books/books.html', title='Books', books=books, form=form)
With this code there are no error messages, this is a question asking for advice on how to keep my code as readable and as simple as possible and be able to query nine columns of the database whilst displaying all the rows queried and all the rows when no query is activated
All help is greatly appreciated. Paul
I am running this on debian 10 with python 3.7
Edit: I am used to working with Libre Office Base
My question is How do I search one or two columns at a time in My database where I have nine columns out of twelve that I want to be able to search, I want to be able to search one or more at a time, example: column "reference" labels a document reference like "A32", and "account" by a the name of the supplier "FILCUI", possibly both at the same time. I have carried out more research and found that most people advocate a "fulltext" search engine such as "Elastic or Whoosh", But in my case I feel if I search "A32" ( a document number) I will get anything in the model of 12 columns with A 1 2. I have looked at Flask Tutorial 101 search Whoosh all very good tutorials, by excellent people, I thought about trying to use SQLAlchemy as a way, but in the first "Flask Tutorial" he says
but given the fact that SQLAlchemy does not support this functionality,
I thought that this SQLAlchemy-Intergrations will not work either.
So therefor is there a way to "search" "query" "filter" multiple different columns of a model with possibly a form for each search without ending up with a "sack of knots" like code impossible to read or test? I would like to stick to SQLAlchemy if possible
I need just a little pointer in the right direction or a simple personal opinion that I can test.
Warm regards.
EDIT:
I have not answered my question but I have advanced, I can query one row at a time and display all the results on the one page, with out a single "if" statement, i think my code is clear and readable (?) I divided each query into its own view function returning to the same main page, each function has its own submitt button. This has enabled me to render the same page. here is my routes code.
#bp.route('/search_aux', methods=['GET', 'POST'])
#login_required
def search_aux():
page_num = request.args.get('page_num', default = 1, type = int)
books = Book.query.order_by(Book.date.desc()).paginate(per_page=100, page=page_num, error_out=True)
add_form = AddBookForm()
aux_form = SearchAuxForm()
date_form = SearchDateForm()
debit_form = SearchDebitForm()
credit_form = SearchCreditForm()
montant_form = SearchMontantForm()
jn_form = SearchJNForm()
pid_form = SearchPIDForm()
ref_form = SearchREForm()
tp_form = SearchTPForm()
ct_form = SearchCTForm()
des_form = SearchDescriptionForm()
if request.method == 'POST':
aux = aux_form.selectaux.data
books = Book.query.order_by(Book.date.desc()).filter(Book.AUX == str(aux)).paginate(per_page=100, page=page_num, error_out=True)
return render_template('books/books.html', books=books, add_form=add_form, aux_form=aux_form, date_form=date_form, debit_form=debit_form,
credit_form=credit_form, montant_form=montant_form, jn_form=jn_form, pid_form=pid_form, ref_form=ref_form,
tp_form=tp_form, ct_form=ct_form, des_form=des_form)
There is a simple form for each query, it works a treat for each single query. Here is the form and html code:
class SearchAuxForm(FlaskForm):
selectaux = QuerySelectField('Aux', query_factory=AUX, get_label='id')
submitaux = SubmitField('submit')
def AUX():
return Auxilliere.query
html:
<div class="AUX">
<form action="{{ url_for('books.search_aux') }}" method="post">
{{ aux_form.selectaux(class="input") }}{{ aux_form.submitaux(class="submit") }}
</form>
</div>
I tried to do this as a single function with one submit button, but it ended in disaster. I have not submitted this as an answer, Because it does not do all I asked but it is a start.
FINAL EDIT:
I would like to thank the person(s) who reopened this question, allowing mr Lucas Scott to provide a fascinating and informative answer to help me and others.
There are many ways to achieve your desired result of being able to query/filter multiple columns in a table. I will give you an example of how I would approach creating an endpoint that will allow you to filter on one column, or multiple columns.
Here is our basic Books model and the /books endpoint as a stub
import flask
from flask_sqlalchemy import SQLAlchemy
app = flask.Flask(__name__)
db = SQLAlchemy(app) # uses in memory sqlite3 db by default
class Books(db.Model):
__tablename__ = "book"
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
title = db.Column(db.String(255), nullable=False)
author = db.Column(db.String(255), nullable=False)
supplier = db.Column(db.String(255))
published = db.Column(db.Date, nullable=False)
db.create_all()
#app.routes("/books", methods=["GET"])
def all_books():
pass
The first step is to decide on a method of querying a collection by using url parameters. I will use fact that multiple instances of the same key in a query parameter are given as lists to allow us to filter on multiple columns.
For example /books?filter=id&filter=author will turn into {"filter": ["id", "author"]}.
For our querying syntax we will use comma separated values for the filter value.
example:
books?filter=author,eq,jim&suplier,eq,self published
Which turns into {"filter": ["author,eq,jim", "supplier,eq,self published"]}. Notice the space in self published. flask will handle the url-encoding for us and give back a string with a space instead of %20.
Let's clean this up a bit by adding a Filter class to represent our filter query parameter.
class QueryValidationError(Exception):
""" We can handle specific exceptions and
return a http response with flask """
pass
class Filter:
supported_operators = ("eq", "ne", "lt", "gt", "le", "ge")
def __init__(self, column, operator, value):
self.column = column
self.operator = operator
self.value = value
self.validate()
def validate(self):
if operator not in self.supported_operators:
# We will deal with catching this later
raise QueryValidationError(
f"operator `{operator}` is not one of supported "
f"operators `{self.supported_operators}`"
)
Now we will create a function for processing our list of filters into a list of Filter objects.
def create_filters(filters):
filters_processed = []
if filters is None:
# No filters given
return filters_processed
elif isinstance(filters, str):
# if only one filter given
filter_split = filters.split(",")
filters_processed.append(
Filter(*filter_split)
)
elif isinstance(filters, list):
# if more than one filter given
try:
filters_processed = [Filter(*_filter.split(",")) for _filter in filters]
except Exception:
raise QueryValidationError("Filter query invalid")
else:
# Programer error
raise TypeError(
f"filters expected to be `str` or list "
f"but was of type `{type(filters)}`"
)
return filters_processed
and now we can add our helper functions to our endpoint.
#app.route("/books", methods=["GET"])
def all_books():
args = flask.request.args
filters = create_filters(args.get("filter"))
SQLAlchemy allows us to do filtering by using operator overloading. That is using filter(Book.author == "some value"). The == here does not trigger the default == behaviour. Instead the creator of SQLAlchemy has overloaded this operator and instead it creates the SQL query that checks for equality and adds it to the
query. We can leverage this behaviour by using the Pythons operator module. For example:
import operator
from models import Book
authors = Book.query.filter(operator.eq(Book.author, "some author")).all()
This does not seem helpful by it's self, but gets us a step closer to creating a generic and dynamic filtering mechanism. The next important step to making this more dynamic is with the built-in getattr which allows us to look up attributes on a given object using strings. Example:
class Anything:
def say_hi(self):
print("hello")
# use getattr to say hello
getattr(Anything, "say_hi") # returns the function `say_hi`
getattr(Anything, "say_hi")() # calls the function `say_hi`
We can now tie this all together by creating a generic filtering function:
def filter_query(filters, query, model):
for _filter in filters:
# get our operator
op = getattr(operator, _filter.operator)
# get the column to filter on
column = getattr(model, _filter.column)
# value to filter for
value = _filter.value
# build up a query by adding multiple filters
query = query.filter(op(column, value))
return query
We can filter any model with our implementation, and not just by one column.
#app.route("/books", methods=["GET"])
def all_books():
args = flask.request.args
filters = create_filters(args.get("filter"))
query = Books.query
query = filter_query(filters, query, Books)
result = []
for book in query.all():
result.append(dict(
id=book.id,
title=book.title,
author=book.author,
supplier=book.supplier,
published=str(book.published)
))
return flask.jsonify(result), 200
Here is everything all together, and including the error handling of validation errors
import flask
import json
import operator
from flask_sqlalchemy import SQLAlchemy
app = flask.Flask(__name__)
db = SQLAlchemy(app) # uses in memory sqlite3 db by default
class Books(db.Model):
__tablename__ = "book"
id = db.Column(db.Integer, primary_key=True, autoincrement=True)
title = db.Column(db.String(255), nullable=False)
author = db.Column(db.String(255), nullable=False)
supplier = db.Column(db.String(255))
published = db.Column(db.Date, nullable=False)
db.create_all()
class QueryValidationError(Exception):
pass
class Filter:
supported_operators = ("eq", "ne", "lt", "gt", "le", "ge")
def __init__(self, column, operator, value):
self.column = column
self.operator = operator
self.value = value
self.validate()
def validate(self):
if self.operator not in self.supported_operators:
raise QueryValidationError(
f"operator `{self.operator}` is not one of supported "
f"operators `{self.supported_operators}`"
)
def create_filters(filters):
filters_processed = []
if filters is None:
# No filters given
return filters_processed
elif isinstance(filters, str):
# if only one filter given
filter_split = filters.split(",")
filters_processed.append(
Filter(*filter_split)
)
elif isinstance(filters, list):
# if more than one filter given
try:
filters_processed = [Filter(*_filter.split(",")) for _filter in filters]
except Exception:
raise QueryValidationError("Filter query invalid")
else:
# Programer error
raise TypeError(
f"filters expected to be `str` or list "
f"but was of type `{type(filters)}`"
)
return filters_processed
def filter_query(filters, query, model):
for _filter in filters:
# get our operator
op = getattr(operator, _filter.operator)
# get the column to filter on
column = getattr(model, _filter.column)
# value to filter for
value = _filter.value
# build up a query by adding multiple filters
query = query.filter(op(column, value))
return query
#app.errorhandler(QueryValidationError)
def handle_query_validation_error(err):
return flask.jsonify(dict(
errors=[dict(
title="Invalid filer",
details=err.msg,
status="400")
]
)), 400
#app.route("/books", methods=["GET"])
def all_books():
args = flask.request.args
filters = create_filters(args.get("filter"))
query = Books.query
query = filter_query(filters, query, Books)
result = []
for book in query.all():
result.append(dict(
id=book.id,
title=book.title,
author=book.author,
supplier=book.supplier,
published=str(book.published)
))
return flask.jsonify(result), 200
I hope this answers your question, or gives you some ideas on how to tackle your problem.
I would also recommend looking at serialising and marshalling tools like marshmallow-sqlalchemy which will help you simplify turning models into json and back again. It is also helpful for nested object serialisation which can be a pain if you are returning relationships.
How can I use getattr without "Class" per se ?
So I have this situation: I have 'columns' that are asking mysql for specific data in a specific order. data is printed via flask/apache so that user has ability to manipulate this data. Now, From flask, POST methdd, I'm receiving changed(?) values and I am storing them in python attributes.I need to check if values within those attributes are same as in original data. Sure, I could hardcore it but I would like have possibility of change columns dynamically.
columns = ["username", "email", "admin"]
data = ("john", "john#snow.com", "True")
username = "john"
email = "different#email.com"
admin = False
Not sure how can I approach it ?
for i in data:
if i == getattr(???, 'username'):
print("it's the same")
or something like this?:
for i in data:
if i == getattr(data, '?????'):
print("it's the same")
Everything is within flask, I cannot embed it into the Class per se. So I don't have 'self' etc.
If I could create class I would probably make something like
class Myclass:
def __init__(self):
self.columns = ["username", "email", "admin"]
self.data = ("john", "john#snow.com", "True")
self.result = []
self.username = "john"
self.email = "different#email.com"
self.admin = False
def test(self):
for i in self.data:
if i == getattr(self, self.columns[self.data.index(i)]):
self.result.append("same")
else:
self.result.append("different")
return self.result
Myclass().test()
['same', 'different', 'different']
It turned out that I was looking for simple eval(). getattr() is designed for different purposes.
so simple:
for i in data:
if i == eval(cols[data.index(i)]):
print("it's the same")
did the trick
Flask is just Python code. You can create a class and use that if that fits your use-case. Or, if you used Flask-SQLAlchemy to manage database-backed data you'd have classes and instances anyway (and get easier data updates to boot).
And classes and instances are not the only objects with attributes; modules and functions have attributes too (although you wouldn't store your data as attributes on either of those), and when you look up methods on anything, you are looking up attributes too.
Pick a storage, then either wrap that storage with an instance of a class, and use getattr(), or pick a different data structure and use the methods for that data structure to get at the different fields. A dictionary, for instance, would make it trivial to get the current value for a given name.
If you do stick to instances, then note that in your loop you'd want to zip your columns and data values together:
for name, value in zip(columns, data):
if getattr(self, name) == value:
self.result.append("same")
else:
self.result.append("different")
Note that you do not have to add "self." in front, the whole point of getattr() is do the same work the . syntax does.
You probably want to put your columns and data lists together as a dictionary:
self.data = {'username': 'john', 'email': 'john#snow.com', 'admin': 'True'}
because that's how you'd process POST data from a form anyway; that way you can iterate over the dict.items() pairs, or use just the columns list to access values:
for name, value in self.data.items():
# ...
or use dict.get() to retrieve values, allowing for missing entries:
for name in self.columns:
if getattr(self, name) == self.data.get(name):
# ...
I don't know how set name for this question.. sorry.
I have function:
myFunction(request, {'Username': 'MyNewUsername', 'Sex': 'Woman', 'SexWant': 'Man'})
def myFunction(self, data):
dataquery = UserData.objects.get(Username = "Patrycja")
for name, key in data.items():
dataquery.name = key
dataquery.save()
Generally speaking this line: dataquery.name
name is 'Username', if I set dataquery.Username = good. But I have to do it as above
From what I understand, your query should be
dataquery = UserData.objects.get(username="Patrycja")
then be aware that the line
dataquery.name = key
sets the attribute name of the object.
In order to set the attributes whose name is specified in data you need to use setattr
for name, value in data.items():
setattr(dataquery, name, value)
and since you seem to want to update only such fields, call save specifing which fields should be updated
dataquery.save(update_fields=data.keys())
Note: please refer to #Sayse's answer in case you need to update more than one record at a time
class Portfolio:
def read(self, pathfilename):
.... stuff ....
self.portfolio[comp_symbol] = {'name': comp_name , 'holdings': comp_holdings}
def save_portfolio(self, port_collection):
port_collection.insert(self.portfolio)
def list_tickers(self):
return (self.portfolio.keys())
def __init__(self):
self.portfolio = {}
self.id = None
Here is how to call it:
port = Portfolio()
print "==================================================================================="
print port.id
print port.portfolio
print "==================================================================================="
port.read(portfolio_file)
print port.id
print port.portfolio
print port.portfolio.keys()
print "==================================================================================="
print port.list_tickers()
port.save_portfolio(port_collection)
print port.list_tickers()
print port.portfolio
The problem is that on performing the insert with pymongo, the property called portfolio changes, and there is an extra key added. For example: print port.list_tickers() is different before and after the insert procedure and I do not see why this should be the case. Before the insert, I get ['CSCO', 'RSA', 'ARO'] and after the insert, I get: ['CSCO', 'RSA', '_id', 'ARO'], but I should still be reading from the same class property. The additional _id is obviously the id from MongoDB.
The _id attribute is mandatory for records in MongoDB - it serves as the unique identifier for a record. MongoDB will create this automatically upon insertion of new records. There is no way to avoid its inclusion in the keys. However, since it's guaranteed to be in each record, you could safely pop it from the list if it really irritates you.