how to minimize db calls in ndb and optimize python code? - python

I have a collection of properties. some of these properties are given review score. review is saved in AggregateReview table. I have to sort these properties on basis of their score. Property with highest review score will come first.When all properties with review score will be sorted, the i have to append those property which aren't reviewed.
This is the code i have written, It's working fine....but i want to know is there anything in this code, that can be optimized, I am new to app engine and ndb, so any help will be appreciated. ( I think there are so many calls to db )...
sortedProperties = []
sortedProperties.extend(sorted([eachProperty for eachProperty in properties if AggregateReview.query(AggregateReview.property == eachProperty.key).get()],key=lambda property: AggregateReview.query(AggregateReview.property == property.key).get().rating,reverse=True))
sortedProperties.extend([eachProperty for eachProperty in properties if AggregateReview.query(AggregateReview.property == eachProperty.key).get() is None])
return sortedProperties
after bit of workaround i came to this:
return sorted(properties,key=lambda property: property.review_aggregate.get().average,reverse=True)
but it throws an error :
'NoneType' object has no attribute 'average'
because it can not find the review_aggregate for every property. I want it to accept None....

I am not sure what's going on in your code there, to be honest, as the lines are soo long. But I know that you may want to look into Memcached. It can be used to minimize hits on a database, and is widely used indeed. It is actually built into Google app engine. Read the docs on it here

Related

Listing the title of created items in a choice field of another item in plone

I have a problem with my Plone item I cannot solve. 'Car' is supposed to create a list of all instances of 'Colour'.
All 'Colour' instances are in a given container. I cannot make it static because I want to add more 'Colour' instances in the future.
I tried selecting each item in my container and add it to my vocabularylist. I only need the id/title of my object, but I always end up with a giant stacktrace of failures.
In the end I want to choose a colour out of the given instances on creating a new 'Car' instance similar to a dropdown.
I have read the docs but cannot find a solution and this is my best idea.
I am also not a python programmer and this is my first plone project. I can add the complete failure list later if you need it.
I appreciate every bit of help. Thank you.
```colour= schema.Choice(
title=u"Colour",
description=u"Paintjob",
vocabulary=givecolour(),
required=False
)
#provider(IContextSourceBinder)
def givecolour():
colourlist = self.context.portal_catalog(path={"query" : "/madb-entw/it/colourcontainer", "depth" : 1})
list = []
i = 0
for colour in colourlist:
list.append(
SimpleVocabulary.createTerm(
colourlist.[i].getObject().id
)
)
i += 1
return SimpleVocabulary(list)```
Please always add your traces, so that we can help you better.
There is also the official community.plone.org forum, where are more people can help you.
I recommend you to use the plone.api to find your objects, this is a bit easier and well doumented.
something like this:
from plone import api
color_brains = api.content.find(context=api.content.get(path='/madb-entw/it/colourcontainer'), depth=1, portal_type='Color')
# no need to do getOject() here, get the id/title directly from the catalog brain
colors = [(color.id, color.Title) for color in color_brains]
One note to your query:
colourlist = self.context.portal_catalog(path={"query" :
"/madb-entw/it/colourcontainer", "depth" : 1})
Path has to be absolute, which means it includes the Plone site id and this can be different in another Plone site.
So an absolute path is not a good idea here, better get the portal object and traverse your path relative from there.
If madb-entw is your Plone site id:
portal.restrictedTraverse('it/colourcontainer')
or better as above, use plone.api.content.get(path='/it/colourcontainer')
Which is cleaner and easier.

How can I specify a default time for a ndb.TimeProperty()?

I find myself stuck on this problem, and repeated Googling, checking SO, and reading numerous docs has not helped me get the right answer, so I hope this isn't a bad question.
One entity I want to create is an event taking place during a convention. I'm giving it the property start_time = ndb.TimeProperty(). I also have a property date = messages.DateProperty(), and I'd like to keep the two discrete (in other words, not using DateTimeProperty).
When a user enters information to create an event, I want to specify defaults for any fields they do not enter at creation and I'd like to set the default time as midnight, but I can't seem to format it correctly so the service accepts it (constant 503 Service Unavailable response when I try it using the API explorer).
Right now I've set things up like this (some unnecessary details removed):
event_defaults = {...
...
"start_time": 0000,
...
}
and then I try looping over my default values to enter them into a dictionary which I'll use to .put() the info on the server.
data = {field.name: getattr(request, field.name) for field in request.all_fields()
for default in event_defaults:
if data[default] in (None, []):
data[default] = event_defaults[default]
setattr(request, default, event_defaults[default])
In the logs, I see the error Encountered unexpected error from ProtoRPC method implementation: BadValueError (Expected time, got 0). I have also tried using the time and datetime modules, but I must be using them incorrectly, because I still receive errors.
I suppose I could work around this problem by using ndb.StringProperty() instead, and just deal with strings, but then I'd feel like I would be missing out on a chance to learn more about how GAE and NDB work (all of this is for a project on udacity.com, so learning is certainly the point).
So, how can I structure my default time properly for midnight? Sorry for the wall of text.
Link to code on github. The conference.py file contains the code I'm having the trouble with, and models.py contains my definitions for the entities I'm working with.
Update: I'm a dummy. I had my model class using a TimeProperty() and the corresponding message class using a StringField(), but I was never making the proper conversion between expected types. That's why I could never seem to give it the right thing, but it expected two different things at different points in the code. Issue resolved.
TimeProperty expects a datetime.time value
import datetime
event_defaults = {...
...
"start_time": datetime.time(),
...
}
More in the docs: https://cloud.google.com/appengine/docs/python/ndb/entity-property-reference#Date_and_Time
Use the datetime() library to convert it into a valid ndb time property value
if data['time']:
data['time'] = datetime.strptime(data['time'][:10], "%H:%M").time()
else:
data['time'] = datetime.datetime.now().time()
ps: Don't forget to change data['time'] with your field name

python what data type is this?

I'm pretty new to python, and currently playing with the zeroconf library.
when I try to register a service on the network, I'm seeing this in the function definition:
def register_service(self, info, ttl=_DNS_TTL):
"""Registers service information to the network with a default TTL
of 60 seconds. Zeroconf will then respond to requests for
information for that service. The name of the service may be
changed if needed to make it unique on the network."""
self.check_service(info)
self.services[info.name.lower()] = info
if info.type in self.servicetypes:
self.servicetypes[info.type] += 1
else:
self.servicetypes[info.type] = 1
now = current_time_millis()
next_time = now
i = 0
while i < 3:
if now < next_time:
self.wait(next_time - now)
now = current_time_millis()
continue
out = DNSOutgoing(_FLAGS_QR_RESPONSE | _FLAGS_AA)
out.add_answer_at_time(DNSPointer(info.type, _TYPE_PTR,
_CLASS_IN, ttl, info.name), 0)
out.add_answer_at_time(DNSService(info.name, _TYPE_SRV,
_CLASS_IN, ttl, info.priority, info.weight, info.port,
info.server), 0)
out.add_answer_at_time(DNSText(info.name, _TYPE_TXT, _CLASS_IN,
ttl, info.text), 0)
if info.address:
out.add_answer_at_time(DNSAddress(info.server, _TYPE_A,
_CLASS_IN, ttl, info.address), 0)
self.send(out)
i += 1
next_time += _REGISTER_TIME
Anyone know what type info is meant to be?
EDIT
Thanks for providing the answer that it's a ServiceInfo class. Besides the fact that the docstring provides this answer when one goes searching for it. I'm still unclear on:
the process expert python programmers follow when encountering this sort of situation - what steps to take to find the data type for info say when docstring wasn't available?
how does python interpreter know info is of ServiceInfo class when we don't specify the class type as part of the input param for register_service? How does it know info.type is a valid property, and say info.my_property isn't?
It is an instance of ServiceInfo class.
It can be deduced from reading the code and docstrings. register_service invokes check_service function which, I quote, "checks the network for a unique service name, modifying the ServiceInfo passed in if it is not unique".
It looks like it should be a ServiceInfo. Found in the examples of the repository:
https://github.com/jstasiak/python-zeroconf/blob/master/examples/registration.py
Edit
I'm not really sure what to say besides "any way I have to". In practice I can't really remember a time when the contract of the interface wasn't made perfectly clear, because that's just part of using Python. Documentation is more a requirement for this reason.
The short answer is, "it doesn't". Python uses the concept of "duck typing" in which any object that supports the necessary operations of the contract is valid. You could have given it any value that has all the properties the code uses and it wouldn't know the difference. So, per part 1, worst case you just have to trace every use of the object back as far as it is passed around and provide an object that meets all the requirements, and if you miss a piece, you'll get a runtime error for any code path that uses it.
My preference is for static typing as well. Largely I think documentation and unit tests just become "harder requirements" when working with dynamic typing since the compiler can't do any of that work for you.

Best option for alternate/suggested search in Django

I have a Django app that searches for data from a model Classified and displays them using simple querysets based on the input term. This works perfectly and I've got no complains with this method.
However, if someone enters a term that returns no data, I would like to provide an option with alternate/suggested search.
Eg: Someone searches 'Ambulance Services' which doesn't return data. I'd like to suggest 'Ambulance' or 'Services' as alternate search options which may return data from the model depending on the data present in the model.
I Googled suggested search in django and it gave me options of Haystack/elastic search, etc which I'm not sure are really required since the search is across just one model.
Note: SO tells me that my question may be closed as it is subjective. If thats the case, please suggest where I can move it to. Thank you!
This is just an idea, but might work for you:
The user enters the search data: "Ambulance Services"
If the query inside the view returns nothing, redirect to the same view using your selected alternative search data, lets say "Ambulance", and a flag value thats saids the view you're preforming a suggested search.
You must have two things in consideration:
What if the alternate search don't returns anything either? You have to set a recursion limit here.
How I'm going to select the data of the alternate search? Well, thats another question about a completly different topic.
This is this idea in code:
def search(request, data, alternate=False, recursion_level=3):
result = Classified.objects.filter(some_filed=data)
if 0 == result.count() and 0 != recursion_level: # Conditions needed for perform alternate search.
new_data = select_new_data(data) # The logic inside this function is up to you.
r_level = recursion_level - 1 # Decreas recursion level.
return redirect('search', alternate=True, recursion_level=r_level) # Redirecting using view name, you can use others
# forms of redirection see link below for visit
# the Djando redirect API doc.
else:
return_data = {}
if alternate:
# You can use this flag in order to reflect
# to the user you have performed an alternate search.
# Example:
return_data["alternate"] = True
# Build your return data.
# and render.
#return render_to_template()
Django redirect doc: redirect
Haystack is, indeed, a great option, here you will find how to give 'spelling suggestions', you can see an example in this OS question/answer
No matter you have only one model, this tool is really great, simple to install /set up/use, and very flexible.
Perhaps this helps as well.

How to write a generative update in SQLAlchemy

I'm just using SQLAlchemy core, and cannot get the sql to allow me to add where clauses. I would like this very generic update code to work on all my tables. The intent is that this is part of a generic insert/update function that corresponds to every table. By doing it this way it allows for extremely brief test code and simple CLI utilities that can simply pass all args & options without the complexity of separate sub-commands for each table.
It'll take a few more tweaks to get it there, but should be doing the updates now just fine. However, while SQLAlchemy refers to generative queries it doesn't distinguish between selects & updates. I've reviewed SQLAlchemy documentation, Essential SQLAlchemy, stackoverflow, and several source code repositories, and have found nothing.
u = self._table.update()
non_key_kw = {}
for column in self._table.c:
if column.name in self._table.primary_key:
u.where(self._table.c[column.name] == kw[column.name])
else:
col_name = column.name
non_key_kw[column.name] = kw[column.name]
print u
result = u.execute(kw)
Which fails - it doesn't seem to recognize the where clause:
UPDATE struct SET year=?, month=?, day=?, distance=?, speed=?, slope=?, temp=?
FAIL
And I can't find any examples of building up an update in this way. Any recommendations?
the "where()" method is generative in that it returns a new Update() object. The old one is not modified:
u = u.where(...)

Categories

Resources