I want to integate django with aiohttp/asyncio
for asynchronous programming and for websockets handling.
I know django has celery & django-channels to do asynchronous task and websocket server respectively but aiohttp is having both asynchronous and websocket server pre built in it and I found that framework more scalable and easy, compared to celery/django channels while creating a function to webscraping (I don't know if webscraping can be possible in celery I, havent tried it yet).
And it also supports async and await perfectly.
But my question is: How can we implement both django and aiohttp in a project? Instead using django's development server can we use aiohttp server to serve the site.
And are we able to integrate django with aiohttp function (like lets take an example: If I want to scrape a website of user submmited input to my database. Can I use await calls in my function while fetching the website and posting the following website to my django database? Or post the function results to another django function?)
And I want to know the disadvantages of integrating, if any?
And while posting your answer please could you post a sample practical example of integration instead suggesting me those libraries over github.
Maybe is time for considering Django >= 4.1 wich already have built-in Asynchronous Support
From the docs:
Django has support for writing asynchronous (“async”) views, along with an entirely async-enabled request stack if you are running under ASGI. Async views will still work under WSGI, but with performance penalties, and without the ability to have efficient long-running requests.
We’re still working on async support for the ORM and other parts of Django. You can expect to see this in future releases. For now, you can use the sync_to_async() adapter to interact with the sync parts of Django. There is also a whole range of async-native Python libraries that you can integrate with.
Related
My problem is basically that I am currently making a customized management system based on Django(3.1) Python(v3.7.9) in which I am pulling the data from a third-party tool. The tool is not giving me the webhooks of every data that I want for visualization and analysis.
The webhook is giving me bits of information and I have to perform a GET request to their API to fetch the rest details if those are not in my database. They are asking for a successful response of webhook within 5 secs otherwise it will trigger a retry.
If I try to do a get request within the function of webhook the time of 5 second will get exceeded the solutions that I came up with was to Django Middleware or Django Triggers so which would be best suitable for my problem I am bit confused.
Note: I can not lower the Django version as I have to use Async Functions
This would be a good use case for a task scheduler like Celery.
Django-triggers is an interface to the Celery scheduler, so it might be a good fit.
Keep in mind, Celery has to be run as a separate process next to django.
Another popular task scheduler is rq-scheduler.
This offers a simple implementation using Redis as a message queue. *Note that Loadbalanced/multi-instance applications are not easily setup with RQ.
What is the best way to update a UI element depending on a change in the database? For example, whenever someone comments on a post, Facebook automatically updates the element for each user - how is this done?
I know that data pulling is one way to do it but are there any better procedures?
I would like to know how something like this can be done with Python (Django) but any other generic solution is welcome as well.
What you're looking for is the websockets protocol:
WebSocket is a computer communications protocol, providing full-duplex
communication channels over a single TCP connection
and usually the default way to do with Django is to use the django-channels project:
Channels augments Django to bring WebSocket, long-poll HTTP, task
offloading and other async support to your code, using familiar Django
design patterns and a flexible underlying framework that lets you not
only customize behaviours but also write support for your own
protocols and needs.
You will probably need to spend some time to configure the channels setup and modify your application to use it, but if you're looking to "push" data from backend to the frontend after a certain action has finished, this is probably the way to do it.
I would recommend looking into the channels-examples repository for an example chat implementation which uses django-channels.
I have a long-running process written in Python 2.7 that I would like to send KML files to my GWT application asynchronously as the KML files are generated.
I have been trying to determine what Python web framework I could use as the back-end with the Python process that could possibly allow the webapp to be hosted on Google AppEngine.
I was able to write a simple python webserver using Cherrypy that sent the kml using JSON from the back-end to GWT using an http request; however, I would like the files to be sent to GWT as they are generated since it may be several minutes between each one. What would be a relatively simple but effective way to achieve this? (Comet? Long-polling? Websockets?)
After researching more python web frameworks, I started experimenting with Tornado because it is non-blocking and seems like it could return data as it is generated possibly using long-polling as mentioned in this answer. However, it looks like GAE requires WSGI which would not allow a Tornado webserver to be non-blocking.
I have read answers to similar questions such as this one. However, I am not sure if updates in web frameworks, GWT, or GAE has changed what is the best option today, or whether some of these answers apply to my case.
What Python web framework would you recommend I use to send data to my asynchronous GWT app using long-polling or another method relatively simply? Could I use this web framework with GAE, or would I need to use something else?
If I understood the problem correctly you might don't need any special framework and you can solve it with what you have: Tasks API and Channel API.
With Tasks API you can perform long task and when the task is complete you can get a notification. You can combine it with the Channel API to push messages directly to the client when a particular task is complete.
You could use also the deferred library to simplify your life with tasks and maybe even using the PubNub for your push notifications, since the setup is easier and you can have many subscribers at the same time.
This question already has answers here:
Closed 10 years ago.
Possible Duplicate:
Recommendations of Python REST (web services) framework?
I'm looking for a RESTful Python (preferably Python 3) web framework. It should have the following things:
configurable URLs
URL generation
support for file uploads
authentication (http basic auth, cookie based)
content-negotiation
based on WSGI
ability to answer requests with HTTP verbs not supported by the requested resource correctly (example: if someone sends PUT but the resource only supports POST and GET, the application should answer with the allowed methods POST and GET)
support for caching headers
transform/render results
What would you recommend?
pyramid 1.3 has python 3.2 support
http://www.pylonsproject.org/projects/pyramid/about
docs: http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/
requests: http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/narr/webob.html#request
view config decorator: http://docs.pylonsproject.org/projects/pyramid/en/1.3-branch/narr/viewconfig.html
gives the ability to write specific views for each request method to the same route e.g.
#view_config(route_name='wiki', renderer='base.pt', request_method='POST')
def view(request):
return {'a': None}
#view_config(route_name='wiki', renderer='base.pt', request_method='PUT')
def view(request):
return {'a': None}
you should glance at this link, Recommendations of Python REST (web services) framework?
in this link #martin has gave really good example for developing your own rest-api. i dont know any RESTful framework which meets your all needs but you can develop your own.
and you can check Flask and Bottle. they are fast, simple and lightweight WSGI micro web-framework for Python...
It sounds like you have a good bit of experience with HTTP. You should check out CherryPy, which is much more of an HTTP framework than a web framework. That point of view allows you to leverage HTTP in ways that the other frameworks generally try to hide from you. CherryPy can do all of the things you requested: flexible configuration is one of its selling points, and it ships with tools for caching, the Allow header, auth, and negotiation. Version 3.2 abandoned the restrictive cgi module for processing uploads and now supports upload temp files, streaming, and automatic pre-processing based on media type.
The non-blocking webserver and framework Tornado looks promising. It's a bit like web.py with an event driven model like the JavaScript framework node.js (but with a more convenient language). But I have not tested it yet.
I was wondering how to create a django webservice (responds with XML) with websockets.
I have already a django webservice which accepts xml requests, parse those requests, makes a database query, creates a response xml and send that xml back to the requester/browser. Just a normal HTTP XML request, where the response is shown as xml within the browser.
But how would i now create a websocket django webservice? Lets say i would like to send a xml response to the requester/browser with the latest data from database whenever a new magical event occurs.
I have read a lot of posts and blogs but it was kinda all too general. Can i solve this only with django + apache or do i need something else next to django and another server only to handle websockets?
I am right now using django 1.3, Apache + wsgi but i would be ready to switch any configuration that would work.
Update:
There are many possible websockets out there,
http://pypi.python.org/pypi?:action=search&term=websocket&submit=search
but which one could be used in my case?
Sorry but django handles async requests very very poorly as it is wsgi. You'll be limited by your number of parallel instance if you have to handle real users. The best solution is to use tornado or node.js.
Tornado handles websocket and long polling brilliantly. Here is my wrapper to allow getting user and sessions from a parallel tornado thread:
https://gist.github.com/1939836
It's adapted from a more complex source, I didn't tested this gist, it's long polling but tornado handlse WebSocket as well.
http://www.tornadoweb.org/documentation/websocket.html
update:
Avoid django-websocket for production use. Even the main developer recommends against it.
I recommend Tornado because it's an awesome technology which is damningly faster/lighter than django. It may be useful for some simple cases. You'll need to configure apache/nginx anyway so, at least get "faster web pages" feature available.
Django-Desktop-Notification focuses on chrome browser and require node.js.
update (01/2016):
Mozilla gave money to django in late 2015 to solve this particular issue, the current most promizing implementation made by a django core dev is this one:
https://github.com/andrewgodwin/channels
It will probably be part of django 1.11 or 2.0
Although it's a bit complicated to setup (but probably the way to go), you could use gunicorn + gevent + socket.io .
I used this article to guide my way through it.
You might also look at server sent events (the article mentioned above looks at that too). If they suit your needs, it would be a bit easier to set up - since you don't have to set up socket.io and you don't need a client library. One catch though - SSE are not supported in IE.
Yeah, django is not all that great when it comes to asynchronous stuffs. My advice for you would be to use twisted as it has a lot of websocket libraries. If you really need to use django..you can make django act just as a pass through, for all the api stuff you build using twisted.