I need a sort of consultation. I am building a web app in django(hosted in heroku) which need to communicate with 100 of embedded devices(writing in C++/C). The embedded devices send data(50kb) to the web app and the web app present this information in a form of graphs.
My concern is , is it wise to build a python polling system(Socket communication) in the server side ?
Or is it Error-prone and I should use services like CloudMQTT?
Thank you in advance for your answers.
Unless you wanna have another project on your hands, it'd be best to use a dedicated library.
This isn't python specific, but I feel like you could achieve your goal relatively easy with a simple ajax request.
Check out The WebSocket API.
EDIT:
Upon further reading I found the Python port of websockets
Related
I built a machine learning model of binary classification in python.
It works on my laptop (e.g. command line tool). Now I want to deploy it in production on a separate server in my company. It has to take inputs from another server (C# application), make some calculations and return outputs back to it.
My question is what are the best practices of doing such thing in production? As I know it can be done through TCP/IP connection.
I am new in this field and I don't know the terms used here.
So can anybody guide me?
Thanks.
I would say it depends on your infrastructure and how can the other application (C#) can communicate.
The easiest way in my opinion would be through a REST API (http request). There are some tools in different languages to create REST endpoints easily and request REST endpoints.
For example, in python, you can request the content of a URL like this:
What is the quickest way to HTTP GET in Python?
But it depends on what you have on the C# side. Can you update the C# code?
Here are a range of solutions:
REST API: need to expose REST endpoints on the communicating "service".
in C#: https://learn.microsoft.com/en-us/aspnet/web-api/overview/older-versions/build-restful-apis-with-aspnet-web-api
in python, I would recommend django framework if you need to create a server (but if the python only request things and don't serve as a server, you may not need it)
message queue like rabbitmq or zeromq, but it requires an external service to manage queues and messages
TCP/IP socket like you suggested, but it requires to manage yourself those connections
I have a long-running process written in Python 2.7 that I would like to send KML files to my GWT application asynchronously as the KML files are generated.
I have been trying to determine what Python web framework I could use as the back-end with the Python process that could possibly allow the webapp to be hosted on Google AppEngine.
I was able to write a simple python webserver using Cherrypy that sent the kml using JSON from the back-end to GWT using an http request; however, I would like the files to be sent to GWT as they are generated since it may be several minutes between each one. What would be a relatively simple but effective way to achieve this? (Comet? Long-polling? Websockets?)
After researching more python web frameworks, I started experimenting with Tornado because it is non-blocking and seems like it could return data as it is generated possibly using long-polling as mentioned in this answer. However, it looks like GAE requires WSGI which would not allow a Tornado webserver to be non-blocking.
I have read answers to similar questions such as this one. However, I am not sure if updates in web frameworks, GWT, or GAE has changed what is the best option today, or whether some of these answers apply to my case.
What Python web framework would you recommend I use to send data to my asynchronous GWT app using long-polling or another method relatively simply? Could I use this web framework with GAE, or would I need to use something else?
If I understood the problem correctly you might don't need any special framework and you can solve it with what you have: Tasks API and Channel API.
With Tasks API you can perform long task and when the task is complete you can get a notification. You can combine it with the Channel API to push messages directly to the client when a particular task is complete.
You could use also the deferred library to simplify your life with tasks and maybe even using the PubNub for your push notifications, since the setup is easier and you can have many subscribers at the same time.
I'm looking for a bit of web development advice. I'm fairly new to the area but I'm sure there are some gurus out there willing to part with some wisdom.
Objective: I'm interested in controlling a Python application on my computer from my personal web hosted site. I know, this question has been asked several times before but in each case the requirements were a bit different from my own. To reduce the length of this post I'll summarize my objective in a few bullet points:
Personal site is hosted by a web hosting company
Site uses HTML, PHP, MySQL, Python and JavaScript, the majority of everything is coded by me from the ground up
An application that is coded in Python will run on a PC within my home and will communicate with an Arduino board
The app will receive commands from the internet to control actuation via the Arduino, and will transmit sensor data back to the site (such as temperature)
Looking for the communication to be bi-directional, fast and secure
Securing the connection between site and Python app would be most ideal
I'm not looking to connect to the Python application directly, the web server must serve as the 'middle man'
So far I've considered HTTP Post and HTML forms, using sockets (Python app would run as a web server), an IRC bot and reading/writing to a text file stored on the web server.
I was also hoping to have a way to communicate with the Python app without needing to refresh the webpage, perhaps using AJAX or JavaScipt? Maybe with Flash?
Is there something I'm not considering? I feel like I'm missing something. Thanks in advance for the advice!
Just thinking out loud for how I would start out with this. First, regarding the website itself, you can just use what's easiest to you, or to the environment you're in. For example, a basic PHP page will do just fine, but if you can get a site running in Python as well, I'd prefer using the same language all over.
That said, I'm not sure why you would need to use a hosted website? Given that you're already forced to have a externally accessible PC at home for the communication, why not run a webserver on that directly (Apache, Nginx, or even something like CherryPy should do)? That webserver can then communicate with the python process that is running to control your Arduino (by using e.g. Python's xmlrpclib). If you would run things via the hosting company, you would still need some process that can handle external requests securely... something a webserver is quite good at. Just running it yourself gives you all the freedom you want, and simplifies things by lessening the number of components in your solution.
The updates on your site I'd keep quite basic: commands you want to run can be handled in the request handlers of the webserver by just calling the relevant (xmlrpclib) calls. Dynamically updating the page is best done by some AJAX calls I reckon. Based on your story, these updates are easily put in a JSON object, suitable for periodically updating only the relevant segments of your page.
I am currently working on a project to create simple file uploader site that will update the user of the progress of an upload.
I've been attempting this in pure python (with CGI) on the server side but to get the progress of the file I obviously need send requests to the server continually. I was looking to use AJAX to do this but I was wondering how hard it would be to, instead of changing to some other framerwork (web.py for instance), just write my own web server for receiving the XML HTTP Requests?
My main problem is that sending the request is done from HTML and Javascript so it all seems like magic trickery at the moment.
Can anyone advise me as to the best way to go about receiving these requests on the server?
EDIT: It seems that a framework would be the way to go. Would web.py be a good route to take?
I would recommend to use a microframework like Sinatra for Ruby. There seem to be some equivalents for Python. What python equivalent of Sinatra would you recommend?
Such a framework allows you to simply map a single method to a route.
Writing a very basic HTTP server won't be very hard (see http://docs.python.org/library/simplehttpserver.html for an example), but you will be missing many features that are provided by real servers and web frameworks.
For your project, I suggest you pick one of the many Python web frameworks and run your application behind Apache/mod_wsgi.
There's absolutely no need to write your own web server. Plenty of options exist, including lightweight ones like nginx.
You should use one of those, and either your own custom WSGI code to receive the request, or (better) one of the microframeworks like Flask or Bottle.
I want to develop a tool for my project using python. The requirements are:
Embed a web server to let the user get some static files, but the traffic is not very high.
User can configure the tool using http, I don't want a GUI page, I just need a RPC interface, like XML-RPC? or others?
Besides the web server, the tool need some background job to do, so these jobs need to be done with the web server.
So, Which python web server is best choice? I am looking at CherryPy, If you have other recommendation, please write it here.
what about the internal python webserver ?
just type "python web server" in google, and host the 1st result...
Well, I used web frameworks like TurboGears, my current projects are based on Pylons. The last ist fairly easy to learn and both come with CherryPy.
To do some background job you could implement that in pylons too.
Just go to your config/environment.py and see that example:
(I implemented a queue here)
from faxserver.lib.myQueue import start_queue
...
def load_environment(global_conf, app_conf):
...
start_queue()
It depends on your need if you simply use CherryPy or start to use something more complete like Pylons.
Use the WSGI Reference Implementation wsgiref already provided with Python
Use REST protocols with JSON (not XML-RPC). It's simpler and faster than XML.
Background jobs are started with subprocess.
Why dont you use open source build tools (continuous integration tools) like Cruise. Most of them come with a web server/xml interface and sometimes with fancy reports as well.
This sounds like a fun project. So, why don't write your own HTTP server? Its not so complicated after all, HTTP is a well-known and easy to implement protocol and you'll gain a lot of new knowledge!
Check documentation or manual pages (whatever you prefer) of socket(), bind(), listen(), accept() and so on.