At first I want to find some API, but I have searched on the internet and didn't find anything
really helpful.
"Real time" I mean live stream the stock price on a webpage without a refresh.
If there is not such API, would my following method be a good way to implement this?
1. Python side, call yahoo finance api to get the most recent price.
2. Browser side, use ajax to constantly call server side to get the price and display the price. More specifically, I am thinking to use setInterval in jquery to achieve this.
How does this approach look?
Actually this is not specific to stock price data, any website that need to constantly retrieve data from server side need to consider this problem. For example google chat, facebook news feed, and so on. Can anybody tell me in general how to achieve live streaming data from server to browser?
Another way would be to use a push architecture. You could take a look at APE - Ajax Push Engine.
You could also take a look at Socket.IO, a realtime application framework for Node.JS.
Hope this helps!
You should definitely use a Push API. These days you should probably use http://www.websocket.org/
You don't want to use a rest API for real time, its inefficient to constantly "pull" the live price. Instead you want a service that will "push" changes to you whenever new trades are executed on the exchange. This is done with a websocket, which is a type of API but it is definitely different from a rest API. This article discusses the difference.
Intrinio provides a real-time websocket and you can access it via Python using this SDK on Github. You can access the same data via rest API using this package in Python. If you try them both you will see the architecture doesn't make sense with a rest API.
This video shows the trades coming in- trades don't execute on the market at regular intervals, it's completely sporadic. Instead of constantly "asking" the server for the data, it's better to "listen". This is called top of the book, meaning you get the newest trades as they come in from the top.
Related
I would like to automate this process of viewing logs in dashboard and typing the information (Total messages sent in a time period, total errors, CPU usage, memory usage), this task is very time consuming at the moment.
The info is gathered from mulesoft anypoint platform. I'm currently thinking of a way to extract all of the data using python webscraping but I don't know how to use it perfectly.
You'll find here a screenshot of the website i'm trying to get the data off of, you can choose to see the logs specific to a certain time and date. My question is, do I start learning python webscrapping or is there another way of doing things that I am just unaware of ?
Logs website example
It doesn't make any sense to use web scrapping. All services in Anypoint Platform have a REST API. Most of them are documented at https://anypoint.mulesoft.com/exchange/portals/anypoint-platform/. Scrapping may broke with any minor change to the UI. The REST API is stable.
The screenshot seems to be from Anypoint Monitoring. I see in the catalog Anypoint Monitoring Archive API. I'm not sure if the API for getting Monitoring Dashboards data is documented. You could alternatively use the older CloudHub Dashboards API. It is probably not exactly the same but it will approximate.
I want to calculate income of a commerce site with services'sales and also products'sales for financial dashboard with python.but I dont know how to get real data from the site.
If your question is that you want to get real-time data from your server(Database) to the Browser(Client), then leverage WebSockets. If you use Django on the backend, use django Channels and Redis to create WebSockets, and then connect to them using Javascript on the frontend. Refer to https://channels.readthedocs.io/en/stable/tutorial/part_1.html for a very simple practical implementation of this.
If you don't want bidirectional communication and more complexity, then you can also consider using Server-Sent Events. It is very useful when only unidirectional communication is required i.e. only one-sided communication from Server to the Browser(Client) is required. (Which is exactly what you want in this case, I think)
How does e-commerce usually handle integrations with ERP software?
We are working on a project for a client, who previously planned to use an ERP system that had a REST API.
This API allowed us to:
Place orders
Inform the ERP if the order was paid for
Get order status
Get all of the items available
Check item availability
Get user data
That would allow us to build a fairly complex online store with a lot of features.
Now the client wants to use another ERP system:
http://www.netsuite.com/portal/platform.shtml
I researched it, and the difficulty of integration surprised me. No REST API, some weird SOAP protocol to communicate with the system, and you have to write a lot of logic using SuiteScript. A whole new, different programming language just to build an integration with an online store? Why not just give developers access to an API to place orders and fetch items? And there are absolutely no docs available online for the thing. People on forums are saying that the system lacks in documentation and one has figure it out himself, along the way.
Magento and Shopify integration is done by third parties and looks dodgy. Same thing with SAP ERP. Am I missing something? Why is such a basic thing as a REST API for e-commerce not available for those systems?
Why develop using Python Django for the back-end and using React.js for the frontend. What is the right way to integrate them with the ERP system?
NetSuite does have a REST API and webservices. "you have to write a lot of logic using SuiteScript" is true but it's just JavaScript and there are many talented developers out there.
I'm not sure there is a "right way" but there are many ways to connect to the data.
My suggestion would be to contact a partner company, such as SWK Technologies. http://swktech.com
NetSuite has two main APIs, SuiteTalk and SuiteScript.
SuiteTalk is the Web Services API, which is SOAP based and allows for pulling data from and updating NetSuite. The SuiteScript API is JavaScript based and allows you to customize accounts and export data at the appropriate event during your business process. The term "SuiteCloud" encompasses all APIs and integration tools.
As for documentation, this is mostly only available to clients and partners. If you have a client who provides you with access to their account, you will gain access to the NetSuite Help Center and all relevant documentation.
Your options for integrating with the e-commerce platform depends on the exact platform. This ranges from Webhooks to HTTP requests.
You can't say NetSuite is delimiting developers in any way. It depends on how you look at it. As I see it, NetSuite provides two main method for developers - SuiteTalk and SuiteScript.By this, developer can create his/her own API, define what kind of acces those API should have.
SuiteTalk is SOAP based.
I would suggest using SuiteScript to create your own API using either NS RESTlet or NS Suitelet.
They have the feature for External URL. By sending request to this external URL you can trigger your own custom functions written on the SuiteScript. By SuiteScript, you can create your own API and define your own function. Ie, developer is in full control.
The only problem I see with NetSuite is its higher barrier for entry. There is no way you can access NetSuite Help Centre without having a Client/Partner/Test account.
But obviously, those who need some kind of integration with NetSuite have NS account.
I failed to get approval for my application that I started to write against the TradeMe API. My API access was not approved. I'm therefore looking for alternatives.
Any NZ property for sale APIs out there? I have seen realestate.co.nz which according to the github repo, might provide something in PHP and Ruby, but the Ruby repo hasn't been touched in several years. Google API perhaps?
I'm specifically interested in obtaining geo-location information for the properties on sale.
The sandbox should let you access trademe without the need to access the main server.
realestate.co.nz seems to have both Javascript and Ruby APIs. I'm going to investigate the possibility of building a Python port as their code is on github/realestate.co.nz
I have no financial interest in either TradeMe or realestate.co.nz, for the record. Just a guy trying to avoid screen scraping.
I have a rethinkdb. Data will get in database for every five minutes.
I want to create a website to real-time inspect this data flow from rethinkdb.
That is, when surfing the webpage, the data from db on webpages can update automatically without refreshing the webpage.
I know there are several ways to make it real-time such as django channels or websockets. However, model in django does not support rethinkdb.
Sorry I am a layman of making website and may express things inaccurately.
Can someone give me a keyword or hint?
If you make your question more specific, the community here will be able to offer you better support.
However, here is a general solution to your problem.
You will need to do two things:
Create a backend API that allows you to:
Check if new data has been added to the database
Fetch new data via a REST api request
Make frontend AJAX requests to this api
Fetch data
Periodically (every 30sec) check if there is new data
Fetch data again if new data is detected
To do this using Django as the backend, I would recommend using the Django Rest Framework to create your API.
This API should have two endpoints:
ListView of your data
Endpoint returning the id and timestamp of the last datapoint
Next you will have to create a frontend that uses javascript to make requests to these endpoints. When you fetch data, store the id and timestamp of the most recent data point. Use this to check if there is new data.
I would recommend using a Javascript framework such as Angular or react but depending on your needs these may be overkill.
EDIT:
Now that you have updated your answer to be more specific, here is my advice. It sounds like your number one priority is rethinkDB and real time data. Django is not well suited this because it is not compatible with rethinkDB. Real time support has come a long way in Django with Django channels however.
It sounds like you are early on in your project and have little to no codebase in Django. I would recommend using horizon along with rethink db. Horizon is a javascript backend built for real time data from rethinkdb.