I have a rethinkdb. Data will get in database for every five minutes.
I want to create a website to real-time inspect this data flow from rethinkdb.
That is, when surfing the webpage, the data from db on webpages can update automatically without refreshing the webpage.
I know there are several ways to make it real-time such as django channels or websockets. However, model in django does not support rethinkdb.
Sorry I am a layman of making website and may express things inaccurately.
Can someone give me a keyword or hint?
If you make your question more specific, the community here will be able to offer you better support.
However, here is a general solution to your problem.
You will need to do two things:
Create a backend API that allows you to:
Check if new data has been added to the database
Fetch new data via a REST api request
Make frontend AJAX requests to this api
Fetch data
Periodically (every 30sec) check if there is new data
Fetch data again if new data is detected
To do this using Django as the backend, I would recommend using the Django Rest Framework to create your API.
This API should have two endpoints:
ListView of your data
Endpoint returning the id and timestamp of the last datapoint
Next you will have to create a frontend that uses javascript to make requests to these endpoints. When you fetch data, store the id and timestamp of the most recent data point. Use this to check if there is new data.
I would recommend using a Javascript framework such as Angular or react but depending on your needs these may be overkill.
EDIT:
Now that you have updated your answer to be more specific, here is my advice. It sounds like your number one priority is rethinkDB and real time data. Django is not well suited this because it is not compatible with rethinkDB. Real time support has come a long way in Django with Django channels however.
It sounds like you are early on in your project and have little to no codebase in Django. I would recommend using horizon along with rethink db. Horizon is a javascript backend built for real time data from rethinkdb.
Related
I've set up a fairly simple Django web app on Google Cloud to handle POSTed API calls that import JSON data into my database. I currently use the standard DRF models to handle serialization as well as validation. However, I've noticed some of the POSTs with larger JSONs timeout after 60 seconds on upload. I am guessing I can just increase the timeout settings on the Google Cloud instance, but was wondering from the perspective of Django what the best step would be. This is a learning project, as I'm new to Django, so I had several ideas on how to proceed. Was wondering if anyone had advice on this topic.
Use the Django FileUpload parser. However, I'm not sure this would avoid the timeout issue, and I would have to write my own custom file validation script.
Some sort of asynchronous processing where the file is uploaded, some ID is returned to the user which can used with a different endpoint to see when the upload has been processed and validated successfully. The downside of this is to my knowledge Django has no about-the-box solution for async support yet so it might be out-of-scope for a relative beginner.
Switch to using Flask. If I'm largely learning on my own should I even be using Django? I imagine Flask might be more efficient for handling large amounts of data but I don't know. So far my current solution has worked well, but if its best to switch I'd rather do that sooner than later.
Looking for a bit of advice.
I have a current architecture of Django and PostgreSQL, where a whole lot of activity is happening to the data via the ORM, through scheduled jobs. The data on the backend is being processed and updated on roughly 30 second intervals.
The data is available to the front-end through a bunch of DRF serialisers (basic REST API). This is just being piped to standard HTML templates at the moment.
I'd like the React front-end to mirror this behaviour, and am looking for best-practice advice on how this is typically done. I know in practice how this works in other frameworks but am not certain of doing this well (namely, connecting React's DOM automation to server-side updates).
(I don't want to get involved with websockets, at all.)
Theoretically, I understand there is two ways to do this:
Front-end AJAX polling the API for new data
HTTP/2 Server Push
Something built into React that will load stuff in incrementally?
Appreciate the advice - (short examples would be really helpful if possible).
First use Django channels, documentation is great btw.
Django Channels
Next what is for you is connect React on some event from models, when you save something in model or create new instance after method save, call channels to expose that object in some group. Of course you need to write URL-s where you will be able to get response from channels.
I am currently getting a lot of data via API and I would like to display it on a dynamic dashboard.
So far, I saw that I could use Grafana, but it seems to require a database such as InfluxDB.
Is it possible to use Grafana without storing the data I get via API into a database, and then display only the data I get each with each request?
You can define a RESTful API endpoint as a datasource using SimpleJson datasource plugin. In this way, you are able to remove the direct dependency to a database. However, your back-end needs to implement certain URLs and conforms to the plugin's request/response formats. I would recommend that you have a look at this link for a sample implementation, and see if it really meets your specific requirement.
I have developed a RESTful API using the Django-rest-framework in python. I developed the required models, serialised them, set up token authentication and all the other due diligence that goes along with it.
I also built a front-end using Angular, hosted on a different domain. I setup CORS modifications so I can access the API as required. Everything seems to be working fine.
Here is the problem. The web app I am building is a financial application that should allow the user to run some complex calculations on the server and send the results to the front-end app so they can be rendered into charts and other formats. I do not know how or where to put these calculations.
I chose Django for the back-end as I expected that python would help me run such calculations wherever required. Basically, when I call a particular api link on the server, I want to be able to retrieve data from my database, from multiple tables if required, and use the data to run some calculations using python or a library of python (pandas or numpy) and serve the results of the calculations as response to the API call.
If this is a daunting task, I at least want to be able to use the API to retrieve data from the tables to the front-end, process the data a little using JS, and send it to a python function located on the server with this processed data, and this function would run the necessary complex calculations and respond with results which would be rendered into charts / other formats.
Can anyone point me to a direction to move from here? I looked for resources online but I think I am unable to find the correct keywords to search for them. I just want a shell code kind of a thing to integrate into my current backed using which I can call some python scripts that I write to run these calculations.
Thanks in advance.
I assume your question is about "how do I do these calculations in the restful framework for django?", but I think in this case you need to move away from that idea.
You did everything correctly but RESTful APIs serve resources -- basically your model.
A computation however is nothing like that. As I see it, you have two ways of achieving what you want:
1) Write a model that represents the results of a computation and is served using the RESTful framework, thus your computation being a resource (can work nicely if you store the results in your database as a way of caching)
2) Add a route/endpoint to your api, that is meant to serve results of that computation.
Path 1: Computation as Resource
Create a model, that handles the computation upon instantiation.
You could even set up an inheritance structure for computations and implement an interface for your computation models.
This way, when the resource is requested and the restful framework wants to serve this resource, the computational result will be served.
Path 2: Custom Endpoint
Add a route for your computation endpoints like /myapi/v1/taxes/compute.
In the underlying controller of this endpoint, you will load up the models you need for your computation, perform the computation, and serve the result however you like it (probably a json response).
You can still implement computations with the above mentioned inheritance structure. That way, you can instantiate the Computation object based on a parameter (in the above case taxes).
Does this give you an idea?
I have a django application hosted on a server running on Apache + Ubuntu. I deployed the application using mod_wsgi. Is there any way to find out the number of visitors to my web site.
I realize that this query might have little to do with django and more do with the server. Any help would be appreciated.
Why not just use Google Analytics? You can easily monitor user behavior, traffic source, time spend on each page, etc.
If you really want to do this with Django you could write a context processor to record each request, but then you would have to write the user's IP and check if the user has not visited before and this would be incredibly imprecise since there might be different users sharing the same IP, etc.
How about using some free statistics provider like Statcounter or Google Analytics?
If you don't want to use Google Analytics or similar, but do it all yourself, you have two options:
One is to alter all views, if you are using class-based view then add a mixin (see this SO question for more information about mixins,) or if you are using the old function-based view you have to manually call another function to keep track.
The other alternative, and probably best one, is to write a middleware class, and keep track through that.
There's also this free and powerful Django app Chartbeat that you could try to work with.
Chartbeat provides real-time analytics to websites and blogs. It shows visitors, load times, and referring sites on a minute-by-minute basis. The service also provides alerts the second your website crashes or slows to a crawl.
https://django-analytical.readthedocs.io/en/latest/services/chartbeat.html