I want to calculate income of a commerce site with services'sales and also products'sales for financial dashboard with python.but I dont know how to get real data from the site.
If your question is that you want to get real-time data from your server(Database) to the Browser(Client), then leverage WebSockets. If you use Django on the backend, use django Channels and Redis to create WebSockets, and then connect to them using Javascript on the frontend. Refer to https://channels.readthedocs.io/en/stable/tutorial/part_1.html for a very simple practical implementation of this.
If you don't want bidirectional communication and more complexity, then you can also consider using Server-Sent Events. It is very useful when only unidirectional communication is required i.e. only one-sided communication from Server to the Browser(Client) is required. (Which is exactly what you want in this case, I think)
Related
Looking for a bit of advice.
I have a current architecture of Django and PostgreSQL, where a whole lot of activity is happening to the data via the ORM, through scheduled jobs. The data on the backend is being processed and updated on roughly 30 second intervals.
The data is available to the front-end through a bunch of DRF serialisers (basic REST API). This is just being piped to standard HTML templates at the moment.
I'd like the React front-end to mirror this behaviour, and am looking for best-practice advice on how this is typically done. I know in practice how this works in other frameworks but am not certain of doing this well (namely, connecting React's DOM automation to server-side updates).
(I don't want to get involved with websockets, at all.)
Theoretically, I understand there is two ways to do this:
Front-end AJAX polling the API for new data
HTTP/2 Server Push
Something built into React that will load stuff in incrementally?
Appreciate the advice - (short examples would be really helpful if possible).
First use Django channels, documentation is great btw.
Django Channels
Next what is for you is connect React on some event from models, when you save something in model or create new instance after method save, call channels to expose that object in some group. Of course you need to write URL-s where you will be able to get response from channels.
How does e-commerce usually handle integrations with ERP software?
We are working on a project for a client, who previously planned to use an ERP system that had a REST API.
This API allowed us to:
Place orders
Inform the ERP if the order was paid for
Get order status
Get all of the items available
Check item availability
Get user data
That would allow us to build a fairly complex online store with a lot of features.
Now the client wants to use another ERP system:
http://www.netsuite.com/portal/platform.shtml
I researched it, and the difficulty of integration surprised me. No REST API, some weird SOAP protocol to communicate with the system, and you have to write a lot of logic using SuiteScript. A whole new, different programming language just to build an integration with an online store? Why not just give developers access to an API to place orders and fetch items? And there are absolutely no docs available online for the thing. People on forums are saying that the system lacks in documentation and one has figure it out himself, along the way.
Magento and Shopify integration is done by third parties and looks dodgy. Same thing with SAP ERP. Am I missing something? Why is such a basic thing as a REST API for e-commerce not available for those systems?
Why develop using Python Django for the back-end and using React.js for the frontend. What is the right way to integrate them with the ERP system?
NetSuite does have a REST API and webservices. "you have to write a lot of logic using SuiteScript" is true but it's just JavaScript and there are many talented developers out there.
I'm not sure there is a "right way" but there are many ways to connect to the data.
My suggestion would be to contact a partner company, such as SWK Technologies. http://swktech.com
NetSuite has two main APIs, SuiteTalk and SuiteScript.
SuiteTalk is the Web Services API, which is SOAP based and allows for pulling data from and updating NetSuite. The SuiteScript API is JavaScript based and allows you to customize accounts and export data at the appropriate event during your business process. The term "SuiteCloud" encompasses all APIs and integration tools.
As for documentation, this is mostly only available to clients and partners. If you have a client who provides you with access to their account, you will gain access to the NetSuite Help Center and all relevant documentation.
Your options for integrating with the e-commerce platform depends on the exact platform. This ranges from Webhooks to HTTP requests.
You can't say NetSuite is delimiting developers in any way. It depends on how you look at it. As I see it, NetSuite provides two main method for developers - SuiteTalk and SuiteScript.By this, developer can create his/her own API, define what kind of acces those API should have.
SuiteTalk is SOAP based.
I would suggest using SuiteScript to create your own API using either NS RESTlet or NS Suitelet.
They have the feature for External URL. By sending request to this external URL you can trigger your own custom functions written on the SuiteScript. By SuiteScript, you can create your own API and define your own function. Ie, developer is in full control.
The only problem I see with NetSuite is its higher barrier for entry. There is no way you can access NetSuite Help Centre without having a Client/Partner/Test account.
But obviously, those who need some kind of integration with NetSuite have NS account.
What is the best way to update a UI element depending on a change in the database? For example, whenever someone comments on a post, Facebook automatically updates the element for each user - how is this done?
I know that data pulling is one way to do it but are there any better procedures?
I would like to know how something like this can be done with Python (Django) but any other generic solution is welcome as well.
What you're looking for is the websockets protocol:
WebSocket is a computer communications protocol, providing full-duplex
communication channels over a single TCP connection
and usually the default way to do with Django is to use the django-channels project:
Channels augments Django to bring WebSocket, long-poll HTTP, task
offloading and other async support to your code, using familiar Django
design patterns and a flexible underlying framework that lets you not
only customize behaviours but also write support for your own
protocols and needs.
You will probably need to spend some time to configure the channels setup and modify your application to use it, but if you're looking to "push" data from backend to the frontend after a certain action has finished, this is probably the way to do it.
I would recommend looking into the channels-examples repository for an example chat implementation which uses django-channels.
I have a rethinkdb. Data will get in database for every five minutes.
I want to create a website to real-time inspect this data flow from rethinkdb.
That is, when surfing the webpage, the data from db on webpages can update automatically without refreshing the webpage.
I know there are several ways to make it real-time such as django channels or websockets. However, model in django does not support rethinkdb.
Sorry I am a layman of making website and may express things inaccurately.
Can someone give me a keyword or hint?
If you make your question more specific, the community here will be able to offer you better support.
However, here is a general solution to your problem.
You will need to do two things:
Create a backend API that allows you to:
Check if new data has been added to the database
Fetch new data via a REST api request
Make frontend AJAX requests to this api
Fetch data
Periodically (every 30sec) check if there is new data
Fetch data again if new data is detected
To do this using Django as the backend, I would recommend using the Django Rest Framework to create your API.
This API should have two endpoints:
ListView of your data
Endpoint returning the id and timestamp of the last datapoint
Next you will have to create a frontend that uses javascript to make requests to these endpoints. When you fetch data, store the id and timestamp of the most recent data point. Use this to check if there is new data.
I would recommend using a Javascript framework such as Angular or react but depending on your needs these may be overkill.
EDIT:
Now that you have updated your answer to be more specific, here is my advice. It sounds like your number one priority is rethinkDB and real time data. Django is not well suited this because it is not compatible with rethinkDB. Real time support has come a long way in Django with Django channels however.
It sounds like you are early on in your project and have little to no codebase in Django. I would recommend using horizon along with rethink db. Horizon is a javascript backend built for real time data from rethinkdb.
Currently I am working on a portal which is exposed to end users. This portal is developed using Python 2.7, Django 1.6 and MySQL.
Now we want to expose this portal as a mobile app. But current design does not support that as templates, views and database are tightly coupled with each other. So we decided to re-architect the whole portal. After some research I found following:
Client side: AngularJS for all client side operations like show data and get data using ajax.
Server side: Rest API exposed to AngularJS. This Rest API can be developed using either Tastypie or Django Rest Framework (still not decided). Rest API will be exposed over Django.
I have few questions:
What you guys think about architecture? Is this is a good or bad design? How it can be improved?
Will performance of portal will go down after adding above layers in architecture?
In the above architecture whether 2 servers should be used to run this (like one for client and other for serving the API's) or one server will be enough. Currently Heroku is used for deployment.
Currently portal is getting 10K hits in a day and it is expected to go to 100K a day in 6 months. Will be happy to provide more information if needed.
If i got an opportunity to architect the portal which you mentioned than i would really love to design the architecture which i have already explained here.
What you guys think about architecture?
This is a common Service Oriented Architecture with decoupled Clients. You just have REST endpoints on your backend, and any Client can consume those endpoints.
You should also think about:
Do you need RESTful service (RESTful == stateless, will you store any state on the server?)
How to scale the service in the future? (this is a legit thing as you already aware of huge traffic increase and assume 2 servers)
How it can be improved?
Use scala instead of python :)
Will performance of portal will go down after adding above layers in architecture?
It depends.
It will get some performance penalty (any additional abtract layer has it's tax), but most probably you won't event notice it. But still, you should measure it using some stress tests.
In the above architecture whether 2 servers should be used to run this (like one for client and other for serving the API's) or one server will be enough. Currently Heroku is used for deployment.
Well, as usual, it depends.
It depends on the usage profile you have right now and on the resources available.
If you are interested in whether the new design will perform better than the old one? - there are a number of parameters.
Resume
This is a good overall approach for the system with different clients.
It will allow you:
Totally decouple mobile app and frontend development from backend development. (It could be different independent teams, outsourceable)
Standardize your API layer (as all clients will consume the same endpoints)
Make you service scalable easier (this includes the separate webserver for static assets and many more).