I am trying to create a web interface to display some data that is being extracted live from a piece of hardware.
I have already written a program using pyQt that extracts data every second. I was wondering if it is possible to simultaneously push this data to a web interface with Python.
I want the webpage to not continuously refresh as data is coming in every second. Charts are essential, I have to have the ability to plot the data that is being pushed.
Can anyone suggest me how I would go about doing this?
Probably the most conventional way to go about this would be to write some server-side code to communicate with the device, and save responses (all of them? the most recent n?) to a database. You'd expose this to a REST API, which you'd then call at a specified interval using AJAX from the browser.
Related
I am building a car server with Raspberry Pi 3.
I am reading the speed variable from serial ports using Python.
This variable continuously changes as the car speed changes.
I want to display that value live, in a HTML file.
How can I do this?
I tried creating a text file from the Python script which constantly overwrites a speed.txt and prints on the HTML but as I read it is not secure and fast enough for monitoring.
Your Html page must be dynamic and communicate with a web server.
You have 2 options for updating the car speed :
use XHR and read the speed periodically from the web server
use websocket and let the server push the latest speed to your page.
You can use a framework like Flask for both options
I'll briefly explain what I'm trying to achieve: We have a lot of servers behind ipvsadm VIPs (LVS load balancing) and we regularly move servers in/out of VIPs manually. To reduce risk (junior ops make mistakes...) I'd like to abstract it to a web interface.
I have a Python daemon which repeatedly runs "ipvsadm -l" to get a list of servers and statistics, then creates JSON from this output. What I'd now like to do is server this JSON, and have a web interface that can pass commands. For example, selecting a server in a web UI and pressing remove triggers an ipvsadm -d <server>... command. I'd also like the web UI to update every 10 seconds or so with the statistics from the list command.
My current Python daemon just outputs to a file. Should I somehow have this daemon also be a web server and serve its file and accept POST requests with command identifiers/arguments? Or a second daemon for the web UI? My only front end experience is with basic Bootstrap and jQuery usually backed by Laravel, so I'm not sure if there's a better way of doing this with sockets and some fancy JS modern-ism.
If there is a more appropriate place for this post, please move it if possible or let me know where to re-post.
You don't need fancy js application. To take the path of least resistance, I would create some extra application - if you like python, I recommend flask for this job. If you prefer php, then how about slim?
In your web application, if you want to make it fast and easy, you can even implement ajax mechanism fetching results based on interval to refresh servers' data every 10 seconds. You will fetch it from json served by independent, already existing deamon.
Running commands clicked on Web UI can be done by your web application.
Your web application is something extra and I find it nice to be separated from deamon which fetch data about servers and save it as json. Anytime you can turn off the page, but all statistics will be still fetching and available for console users in json format.
I have a rethinkdb. Data will get in database for every five minutes.
I want to create a website to real-time inspect this data flow from rethinkdb.
That is, when surfing the webpage, the data from db on webpages can update automatically without refreshing the webpage.
I know there are several ways to make it real-time such as django channels or websockets. However, model in django does not support rethinkdb.
Sorry I am a layman of making website and may express things inaccurately.
Can someone give me a keyword or hint?
If you make your question more specific, the community here will be able to offer you better support.
However, here is a general solution to your problem.
You will need to do two things:
Create a backend API that allows you to:
Check if new data has been added to the database
Fetch new data via a REST api request
Make frontend AJAX requests to this api
Fetch data
Periodically (every 30sec) check if there is new data
Fetch data again if new data is detected
To do this using Django as the backend, I would recommend using the Django Rest Framework to create your API.
This API should have two endpoints:
ListView of your data
Endpoint returning the id and timestamp of the last datapoint
Next you will have to create a frontend that uses javascript to make requests to these endpoints. When you fetch data, store the id and timestamp of the most recent data point. Use this to check if there is new data.
I would recommend using a Javascript framework such as Angular or react but depending on your needs these may be overkill.
EDIT:
Now that you have updated your answer to be more specific, here is my advice. It sounds like your number one priority is rethinkDB and real time data. Django is not well suited this because it is not compatible with rethinkDB. Real time support has come a long way in Django with Django channels however.
It sounds like you are early on in your project and have little to no codebase in Django. I would recommend using horizon along with rethink db. Horizon is a javascript backend built for real time data from rethinkdb.
I am new to python. I want to read JSON data from URL and if there is any change in JSON data on server, I want to update in my JSON file which is on client. How can I do that through python?
Actually i am ploting graph on django using JSON data which is on another server. That JSON data is updated frequently. So here i want to update my charts based on updated json data. For that i has to listen to URL link for change. So how can i do that.....i know with select() system call i can but need some another way
There's no way to "listen" for changes other than repeatedly requesting that URL to check if something has changed; i.e. classic "pull updates". To get actual live notifications of changes, that other server needs to offer a way to push such a notification to you. If it's just hosting the file and is not offering any active notifications of any sort, then repeatedly asking is the best you can do. Try not to kill the server when doing so.
At first I want to find some API, but I have searched on the internet and didn't find anything
really helpful.
"Real time" I mean live stream the stock price on a webpage without a refresh.
If there is not such API, would my following method be a good way to implement this?
1. Python side, call yahoo finance api to get the most recent price.
2. Browser side, use ajax to constantly call server side to get the price and display the price. More specifically, I am thinking to use setInterval in jquery to achieve this.
How does this approach look?
Actually this is not specific to stock price data, any website that need to constantly retrieve data from server side need to consider this problem. For example google chat, facebook news feed, and so on. Can anybody tell me in general how to achieve live streaming data from server to browser?
Another way would be to use a push architecture. You could take a look at APE - Ajax Push Engine.
You could also take a look at Socket.IO, a realtime application framework for Node.JS.
Hope this helps!
You should definitely use a Push API. These days you should probably use http://www.websocket.org/
You don't want to use a rest API for real time, its inefficient to constantly "pull" the live price. Instead you want a service that will "push" changes to you whenever new trades are executed on the exchange. This is done with a websocket, which is a type of API but it is definitely different from a rest API. This article discusses the difference.
Intrinio provides a real-time websocket and you can access it via Python using this SDK on Github. You can access the same data via rest API using this package in Python. If you try them both you will see the architecture doesn't make sense with a rest API.
This video shows the trades coming in- trades don't execute on the market at regular intervals, it's completely sporadic. Instead of constantly "asking" the server for the data, it's better to "listen". This is called top of the book, meaning you get the newest trades as they come in from the top.