A lot of times, I wrote queries like the following:
pony.orm.select(u for u in User if u.available and u.friends > 0 and ...)
So, I would like to write my own version of select, an alternative of it. Something like to avoid me writing every time the first part of the predicate, if u.available and u.friends > 0.
My question is more general: how can I write a function like select that it accepts arguments like those that select method or count method can accept.
Let's define a User entity in the following way:
from datetime import datetime, timedelta
from pony.orm import *
db = Database('sqlite', ':memory:')
class User(db.Entity):
username = Required(str, unique=True)
password = Required(str)
friends = Set("User", reverse='friends') # many-to-many symmetric relation
online = Required(bool, default=False) # if user is currently online
last_visit = Optional(datetime) # last visit time
disabled = Required(bool, default=False) # if user is disabled by administrator
sql_debug(True)
db.generate_mapping(create_tables=True)
Now we can define some convenient functions to retrieve most frequently used types of users. The first function will return the users who are not disabled by the admin:
def active_users():
return User.select(lambda user: not user.disabled)
In that function I use select method of User entity which accepts a lambda function, but the same function can be written using global select function which accepts a generator expression:
def active_users():
return select(u for u in User if not user.disabled)
The result of active_users function is a query object. You can call filter method of the query object to produce more specific query. For example, I can use active_users function to select active users whose names start with the 'A' letter:
users = active_users().filter(lambda user: user.name.startswith('A')) \
.order_by(User.name)[:10]
Now I want to find users who visit the site in a few last days. I can define another function which uses the query returned from the previous function and augment it in the following way:
def recent_users(days=1):
return active_users().filter(lambda u: u.last_visit > datetime.now() - timedelta(days))
In this example I pass the days argument to the function and use its value inside the filter.
You can define a set of such functions which will form data access layer of your application. Some more examples:
def users_with_at_least_n_friends(n=1):
return active_users().filter(lambda u: count(u.friends) >= n)
def online_users():
return User.select(lambda u: u.online)
def online_users_with_at_least_n_online_friends(n=1):
return online_users().filter(lambda u: count(f for f in u.friends if f.online) >= n)
users = online_users_with_at_least_n_online_friends(n=10) \
.order_by(User.name)[:10]
In the examples above I define global functions. Another option is to define these functions as a classmethods of the User entity:
class User(db.Entity):
username = Required(str, unique=True)
...
#classmethod
def name_starts_with(cls, prefix):
return cls.select(lambda user: not user.disabled
and user.name.startswith(prefix))
...
users = User.name_starts_with('A').order_by(desc(User.last_visit))[:10]
If you might want to have a generic function which can be applied to different entity classes, then you need to pass the entity class as a parameter. For example, if a number of different classes have the deleted attribute, and you want to have a generic method to select only non-deleted objects, you can write something like that:
def select_active(cls):
return cls.select(lambda obj: not obj.deleted)
select_active(Message).filter(lambda msg: msg.author == current_user)
All functions provided above have one drawback - they are not composable. You cannot get a query from one function and augment it with another function. If you want to have a function which can augment existing query, that function should accept the query as an argument. Example:
def active_users():
return User.select(lambda user: not user.disabled)
def name_starts_with(query, prefix):
return query.filter(lambda user: user.name.startswith('prefix'))
The name_starts_with function can be applied to another query:
users1 = name_starts_with(active_users(), 'A').order_by(User.last_visited)
users2 = name_starts_with(recent_users(), 'B').filter(lambda user: user.online)
Also we are working on the query extension API which will allow a programmer to write custom query methods. When we release this API it will be possible to just chain custom query methods together in the following way:
select(u for u in User).recent(days=3).name_starts_with('A')[:10]
Hope I answered your question. If this is the case, please accept the answer as a correct one.
Related
I am relatively new to Django and very new to writing unit tests. I'd like to ask for assistance but I'm a bit stuck with where to even begin. The app I'm working with allows a teacher to assign multiple assignments to a student. On the student dashboard, an assignment should only be available if the start date <= today's date. The student should only see the first assignment in the list.
I need to compose a unit test to cover this scenario:
manually assign multiple assignments to a student
use the same query that is used for the student dashboard to check that the only assignments returned are the ones with a
start date <= today's date
check that the student only sees the first assignment (with the earliest start date) in the list.
Below I have posted the relevant code that is pulling what displays on the student dashboard. Please let me know if additional code is needed to help me get started with this. Thanks very much for any help you can offer!
Edit: I would like to only use the built in django.test features for now, if possible
from my home/views.py file
#login_required
def index(request):
user_type = request.user.type.text
if user_type == 'Student':
""" Only return the first test so the student sees one test at a time"""
assignment = Assignment.objects.filter(
student=request.user,
start_date__lte=datetime.date.today(),
completed=False).first()
if (assignment):
context = {
'test_pk': assignment.test.pk,
}
else:
context = {}
return render(request, 'home/student.html', context)
Basics of testing stuff like this goes roughly like:
Create the desired data manually
Create the action/conditions that are happening in the view(maybe send a request to the view)
Check the result with the previously manually created data.
So, start with creating some Assignment objects for students.
Run your view(send a request to your view logged in as the previously created user)
Check if the desired outcome exist in the returned html.
I would suggest you to use pytest and factoryboy for that, there's a lot of great tutorials online to use it with Django.
For you example it would be something like this
You need first to init the session, we can create fixture for that
import pytest
import factory
#pytest.fixture
def client():
from django.test.client import Client
return Client(HTTP_USER_AGENT='pytest')
then we should init the session, another fixture:
#pytest.fixture
def session(client):
# your custom session her
user = #use factory for the user
client.user = user
# your defaults for
# client.GET
# client.POST
# client.META
return client
class AssignmentFactory(factory.django.DjangoModelFactory):
class Meta:
model = Assignment
django_get_or_create = ('any attribute you like',)
# default the attributes you want to create here
# For example
name = "assignment one"
Then the test can be something like this
#pytest.mark.django_db
def test_retrieve_assignment_success(session):
path = reverse("view_name")
assignment = AssignmentFactory()
res = session.get(path=path, data={}, follow=False)
json_res = res.json()
assert json_res.get('context') is not None
assert assigment.pk == json_res.get('context').get('test_pk')
#pytest.mark.django_db
def test_retrieve_assignment_fail(session):
path = reverse("view_name")
res = session.get(path=path, data={}, follow=False)
json_res = res.json()
assert json_res.get('context') is not None
assert json_res.get('context') == {}
I am using flask-restful this is
My class I want to insert
class OrderHistoryResource(Resource):
model = OrderHistoryModel
schema = OrderHistorySchema
order = OrderModel
product = ProductModel
def post(self):
value = req.get_json()
data = cls.schema(many=True).load(value)
data.insert()
In my model
def insert(self):
db.session.add(self)
db.session.commit()
schema
from config.ma import ma
from model.orderhistory import OrderHistoryModel
class OrderHistorySchema(ma.ModelSchema):
class Meta:
model = OrderHistoryModel
include_fk = True
Example Data I want to insert
[
{
"quantity":99,
"flaskSaleStatus":true,
"orderId":"ORDER_64a79028d1704406b6bb83b84ad8c02a_1568776516",
"proId":"PROD_9_1568779885_64a79028d1704406b6bb83b84ad8c02a"
},
{
"quantity":89,
"flaskSaleStatus":true,
"orderId":"ORDER_64a79028d1704406b6bb83b84ad8c02a_1568776516",
"proId":"PROD_9_1568779885_64a79028d1704406b6bb83b84ad8c02a"
}
]
this is what i got after insert method has started
TypeError: insert() takes exactly 2 arguments (0 given)
or there is another way to do this action?
Edited - released marshmallow-sqlalchemy loads directly to instance
You need to loop through the OrderModel instances in your list.
You can then use add_all to add the OrderModel objects to the session, then bulk update - see the docs
Should be something like:
db.session.add_all(data)
db.session.commit()
See this post for brief discussion on why add_all is best when you have complex ORM relationships.
Also - not sure you need to have all your models/schemas as class variables, it's fine to have them imported (or just present in the same file, as long as they're declared before the resource class).
You are calling insert on list cause data is list of model OrderHistoryModel instances.
Also post method doesn't need to be classmethod and you probably had an error there as well.
Since data is list of model instances you can use db.session.add_all method to add them to session in bulk.
def post(self):
value = req.get_json()
data = self.schema(many=True).load(value)
db.session.add_all(data)
db.session.commit()
For my site for auth I'm using https://flask-httpauth.readthedocs.io/en/latest/ . Now I'm trying to make it that it's using data from database. To do that i created database named Users and created columns named username and password.
To get data from this table after defining its class as model I've made get_user functions which looks like it:
#staticmethod
def get_user():
query = (Users
.select())
user = []
for s in query:
user.append(s)
return user
(I'm not sure if it's correct)
Next I had to modify get_pw function but I also wasn't sure how to modify it so I made it look like it:
#auth.get_password
def get_pw(username):
if username in Users.get_user():
return users.get(Users.get_user())
return None
Now after running the site I get prompt to give login and password but those that I set up in my database doesn't seem to work so there must be a problem with get_pw function. Also I'm using peewee SQL to manage database : http://docs.peewee-orm.com/en/latest/peewee/querying.html
You can get rid of your get_user method since you are issuing a very large select query that fetches all records from user table. The get_pw can be redefined as:
def get_pw(username):
user = Users.get(Users.name == username) #assuming you have a username field in model
return user.password #assuming a password field
Also, its a good practice to define your model class as a singular noun rather than plural. So, its better to call it User rather than Users.
This'll help you get started in no time: http://docs.peewee-orm.com/en/latest/peewee/quickstart.html#quickstart
I want to drop all user sessions when user resets his password, but I can't find a way to do that.
My idea was to get all UserTokens of the specific user and delete them, but it seems impossible, because of
user = model.StringProperty(required=True, indexed=False)
in UserToken model
Any ideas how to do that?
I see two ways how to do that.
First is to inherit from the UserToken class making user an indexed property. Then you can set the token_model class property to your new token model in your user class. Here is the code:
class MyToken(UserToken):
user = ndb.StringProperty(required=True)
class MyUser(User):
token_model = MyToken
# etc.
Don't forget to set the user model used by webapp2 to your user class if you do not do it already:
webapp2_config = {
"webapp2_extras.auth": {
"user_model": "models.MyUser"
},
# etc.
}
app = webapp2.WSGIApplication(routes, config=webapp2_config)
The second way is to make a complicated datastore query based on the token key name. Since the key names are of the form <user_id>.<scope>.<random>, it is possible to retrieve all the entities starting with a specific user ID. Have a look at the code:
def query_tokens_by_user(user_id):
min_key = ndb.Key(UserToken, "%s." % user_id)
max_key = ndb.Key(UserToken, "%s/" % user_id) # / is the next ASCII character after .
return UserToken.query(UserToken.key > min_key, UserToken.key < max_key)
This uses the fact that the query by key names works in the lexicographical order.
I have a weired problem with couple of queries I am trying to run.
I have built a method which returns a tuple of result from the query-
def get_activeproducts():
query = Product.gql("WHERE active = True")
choices = []
for obj in query:
choices.append((str(obj.key()), obj.name))
return choices
The problem is, the result is same for each call. Even if products are deleted or changed to 'False' in the product attribute 'active'. The result will be refreshed only when I restart the sdk server. In production, it just doesnt change till I change versions.
I have seen similar issue with one more query where the query property is BooleanProperty.
Any idea on how this could be fixed?
EDIT:
I am using the method in a tipfy application. It is used to populate a select field in wtforms. 'choices' basically takes in a list of tuples (value, name) pair.
class InvoiceForm(Form):
product = SelectField('Product', choices=get_activeproducts())
I dont have any issue with editing. WHen I check it from the admin end, I can see that certain products are set to 'False'. And even if I empty(delete) the whole list of products, I get the same list I got the first time.
I am not using caching anywhere in the application.
Your class definition is getting cached by the App Engine runtime when an instance is started, with the default set to what it was when the instance started. To make the choices dynamic, you need to set them at runtime.
Example from the wtforms (which IIRC is what tipfy is using) docs; will need to be adjusted for App Engine queries:
class UserDetails(Form):
group_id = SelectField(u'Group', coerce=int)
def edit_user(request, id):
user = User.query.get(id)
form = UserDetails(request.POST, obj=user)
form.group_id.choices = [(g.id, g.name) for g in Group.query.order_by('name')]
when you create your form, the function is called once.
you can overload the form __init__.py function to do this cleanly
class InvoiceForm(Form):
product = SelectField(u'Group', choices=[])
def __init__(self, product_select, *args, **kwargs)
super(InvoiceForm, self).__init__(*args, **kwargs)
self.product.choices = select_dict
----
form = InvoiceForm(product_select=get_activeproducts())