Django Model with API as data source - python

I want to create a new model which uses the https://developer.microsoft.com/en-us/graph/graph-explorer api as a data source as i want to have additional info on the user.
Using a computed property on the model does not work as it is going to query for each instance in the set.
So, i want to have the model relate to a new model which has the api as it´s data source.
I could not find anything on this topic, besides maybe abusing the from_db() method, if this even works.

It appears that what you're trying to do is to cache data from an external API that relates to, and augments/enriches, your user model data. If so, you can simply use a custom user model (instead of Django's default; this is a highly-recommended practice anyway) and then simply store the external API data in serialized form in a TextField attribute of your custom user model (let's call it user_extras; you can write model methods that serializes and deserializes this field for convenience upon access in your views).
The key challenge then is how to keep user_extras fresh, without doing something terrible performance-wise or hitting some constraint like API call limits. As you said, we can't do API queries in computed properties. (At least not synchronously.) One option then is to have a batch job/background task that regularly goes through your user database to update the user_extras in some controlled, predictable fashion.

Related

Django pure controller functions

I am wondering on how to implement pure controller functions in a Django's' "biased" MVC scheme. Let me explain it on an example.
Let's say I have a model of an Invoice, which has some attributes (say net, gross etc.). I can present it to the user using a view + template. And that's fine and easy.
But now, I want to send this invoice to a client. This is a more complicated thing, inluding more models (i.e. create an addressed Package model, get a number and let's say few other thing including creating and modifying not only Invoice model itself, but also creating and updating few other model types and instances.
I want this "action" to be available in multiple places of my web application, so going by the book I need to create a view with those actions implemented and bind it to some URL. Probably it should be implemented in POST action.
My questions are:
What kind of generic view should it be (just View? DetailView? other?).
Where should this View redirect after succesfull "send"? The simplest answer would be to redirect to the same referring page, but is this a correct way?
What if I want this "action" to be ran in background (say, send all unsend invoices at midnight) using celery or such? Of course I can make this a celery task and call it in a view. But is this clean django'ish solution? Where do you store such pure business methods in an app/project?

In Django, given a model instance, how can I determine if a ForeignKey field has been fully hydrated or not?

Context:
I maintain legacy Django code. In many cases my code receives multiple model objects, each of which has a ForeignKey or a manually cached property that represents the same data entity. The data entities referred to thus do not change. However, not all objects I receive have ever accessed those ForeignKey fields or cached properties, so the data may not be present on those objects, though it will be lazy-loaded on first access.
I do not easily have access to/control over the code that declares the models.
I want to find any one of the objects I have which has a primed cache, so that I can avoid hitting the database to retrieve the data I'm after (if none have it, I'll do it if I have to). This is because the fetch of that data is frequent enough that it causes performance issues.
Django 1.6, Python 2.7.
Problem
I can interrogate our manually cached fields and say internal_is_cached(instance, 'fieldname') without running a query. However, I cannot do this with ForeignKey fields.
Say I have a model class in Django, Foo, like so:
class Foo(models.Model):
bar = models.ForeignKey('BarModel')
Question
If I get an instance of the model Foo from somewhere, but I do not know if bar has ever been called on it, or if it has been eagerly fetched, how do I determine if reading instance.bar will query the database or not?
In other words, I want to externally determine if an arbitrary model has its internal cache primed for a given ForeignKey, with zero other knowledge about the state or source of that model.
What I've Tried
I tried model caching using the Django cache to make an "end run" around the issue. The fetches of the related data are frequent enough that they caused unsustainable load on our caching systems.
I tried various solutions from this question. They work well for modified models, but do not seem to work for models which haven't been mutated--I'm interested in the "lazy load" state of a model, not the "pending modification" state. Many of those solutions are also inapplicable since they require changing model inheritance or behavior, which I'd like to avoid if possible (politics).
This question looked promising, but it requires control over the model initial-reader process. The model objects my code receives could come from anywhere.
Doing the reverse of the after-the-fact cache priming described in [this writeup] works for me in testing. However, it relies on the _default_manager internal method of model objects, which is known to be an inaccurate field reference for some of our (highly customized) model objects in production. Some of them are quite weird, and I'd prefer to stick to documented (or at least stable and not frequently bypassed) APIs if possible.
Thanks #Daniel Roseman for the clarification.
With Django version <= 1.6 (working solution in your case)
You can check if your foo_instance has a _bar_cache attribute:
hasattr(foo_instance, "_bar_cache")
Like explained here.
With Django version > 1.6
The cached fields are now stored in the fields_cache dict in the _state attribute:
foo_instance._state.fields_cache["bar"]

Django create custome admin view with non database fields

I would like to create a custome admin view with fields with data that I fill manually, meaning that there's no database table behind.
In my admin.py I have this so far
class TestAdmin(admin.ModelAdmin):
pass
admin.site.register(Test, TestAdmin)
and in the models.py
class Test(models.Model):
test = models.BooleanField(
default=True
)
But with this I receive an error
django.db.utils.ProgrammingError: relation "..._test" does not exist
Django is looking up the table in the database but in my model I need only fields which data I fill manually
It seems that you don't really get what a Model and ModelAdmin are. A Model class is the representation of a database table, and a ModelAdmin is a component that provides administration features for this table and it's content. IOW, when you write that "there's no database table behind", you're missing the point somehow: you cannot have a ModelAdmin without a Model, and you cannot have a Model without a database table.
To make a long story short: you'll either have to
1/ use a model or
2/ write the whole view(s) and templates by yourself and plug them into the admin.
For the second solution, see this question (and it's answers), or just google for "django admin views without models" (there are a couple 3rd part apps that are supposed to help plugin custom views in the admin).
Now the first solution might - or not, depending on your exact needs - be as good as the second one, depending on who populates your s3bucket and how. Without a model, you'll have to query the bucket for a file listing each and every time someone accesses your custom admin view, which might get quite costly. If all the uploads to the s3 bucket are under your control, you can use a model to keep a local file listing "cache" and you'll only need to hit s3 when effectively downloading a file. This also gives you the opportunity to store metadata about those files - which can be used for searching / filtering - and eventually to link your s3 files to other models if needed. And of course this means you get your admin for free ;)
Actually even if you don't have control on those uploads, you may still want to use a model for the above reasons and have a background task (cron job or equivalent) that updatees your table from the s3 bucket contents (or - if s3 provides web hooks - have an API endpoint called by s3 on uploads that updates your table).
The way that django's ORM works is that all class members of a model class that is an instance of model.Field will map to a column in the database.
It is possible to have model properties that does not map to a database table. You could use a #property instead, for example.
class Test(models.Model):
#property
def test(self):
return get_aws_bucket_value(self.id)
#test.setter
def test(self, value):
set_aws_bucket_value(self.id, value)
You have to implement the aws getter/setter functions yourself.
You can use properties as read-only fields in the django admin class. Be careful about using it in the list view if your getter logic needs to fetch data from a remote server syncronously, since that would be a significant performance bottleneck.
If you want to be able to set values for a setter from the django admin, you would have to write a custom ModelForm for TestAdmin.
It is possible to wrangle Django into doing what you want here. But Django's models and admin are based around the ORM, which means using a relational database as persistant storage. If you don't want that, you can find yourself fighting against and overriding the framework more often than benefiting from Django's "batteries included". This project might be better solved with a less monolithic and less opinionated framework such as Flask instead of Django.

Could someone please show me where I can find Flask-Admin callbacks?

I am attempting to combine Flask-Admin and a Flask S3 Tool Suite written by someone to store images to Amazon S3 simple storage. The tool I am trying to specifically use is the S3Saver functionality and in the article link the user goes on to say that to incorporate this tool, I would ideally use it on a flask-admin callback.
However, I cannot find a list of Flask-Admin callbacks anywhere. I have done my homework and I have literally checked the entire docs and source code for these callbacks Flask-Admin documentation .
In flask_admin.contrib.sqla I found some methods that kind of resembled a callback but its not what I am looking for. I imagine that it is something close to Rails's before_action and after_action. Can someone please point me in the right direction?
By the way this is the actual quote from the article
If you wanted to handle deleting files in the admin as well, you could (for example) use s3-saver, and hook it in to one of the Flask-Admin event callbacks
What callbacks are those?
Flask-Admin Callback Functions (as far as I know)
after_model_change(form, model, is_created)
#Called from create_model after successful database commit.
after_model_delete(model)
#Called from delete_model after successful database commit (if it has any meaning for a store backend).
on_form_prefill(form,id)
# Called from edit_view, if the current action is rendering the form rather than receiving client side input, after default pre-filling has been performed.
on_model_change(form,model,is_created)
#Called from create_model and update_model in the same transaction (if it has any meaning for a store backend).
on_model_delete(model)
# Called from delete_model in the same transaction (if it has any meaning for a store backend).
More info at http://flask-admin.readthedocs.org/en/latest/api/mod_model/ (search for "Called from")
Here is the documentation for Flask Admin events :
https://flask-admin.readthedocs.org/en/latest/api/mod_model/#flask_admin.model.BaseModelView.on_form_prefill
on_form_prefill(form, id) Perform additional actions to pre-fill the edit form.
Called from edit_view, if the current action is rendering the form
rather than receiving client side input, after default pre-filling has
been performed.
By default does nothing.
You only need to override this if you have added custom fields that
depend on the database contents in a way that Flask-admin can’t figure
out by itself. Fields that were added by name of a normal column or
relationship should work out of the box.
Parameters: form – Form instance id – id of the object that is going
to be edited
on_model_change(form, model, is_created) Perform some actions before a model is created or updated.
Called from create_model and update_model in the same transaction (if
it has any meaning for a store backend).
By default does nothing.
Parameters: form – Form used to create/update model model – Model
that will be created/updated is_created – Will be set to True if model
was created and to False if edited
on_model_delete(model) Perform some actions before a model is deleted.
Called from delete_model in the same transaction (if it has any
meaning for a store backend).
By default do nothing.

Passing JSON (dict) vs. Model Object to Templates in Django

Question
In Django, when using data from an API (that doesn't need to be saved to the database) in a view, is there reason to prefer one of the following:
Convert API data (json) to a json dictionary and pass to the template
Convert API data (json) to the appropriate model object from models.py and then pass that to the template
What I've Considered So Far
Performance: I timed both approaches and averaged them over 25 iterations. Converting the API response to a model object was slower by approximately 50ms (0.4117 vs. 0.4583 seconds, +11%). This did not include timing rendering.
Not saving this data to the database does prevent me from creating many-to-many relationships with the API's data (must save an object before adding M2M relationships), however, I want the API to act as the store for this data, not my app
DRY: If I find myself using this API data in multiple views, I may find convenience in putting all my consumption/cleaning/etc. code in the appropriate object __init__ in models.
Thanks very much in advance.
Converting this to a model objects doesn't require storing it in database.
Also if you are sure you don't want to store it maybe placing it in models.py and making it Django models is wrong idea. Probably it should be just normal Python classes e.g. in resources.py or something like that, not to mistake it with models. I prefer such way because maybe converting is slower (very tiny) but it allows to add not only custom constructor but others methods and properties as well which is very helpful. It also is just convenient and organizes stuff when you use normal classes and objects.
Pass the list of dictionaries directly to the template. When you need to use it with models, use .values() to get a list of dictionaries from it instead.

Categories

Resources