High level Python library for using Websocket and Comet "fallback" transparently - python

I'm looking for a High Level Python library for establishing HTTP connections to a Web server.
The connections should ideally remain open (persistant) for sending and receiving two-ways messages, so Websocket are great to me.
As I want it to be compatible will most HTTP proxies, I think about a "fallback" mode with HTTP polling (Comet style).
My problem is I can't find a library for managing these two kinds of connections transparently. Ideally, I would establish the connection to the server with one of the techniques (Websocket or Comet), then simply send/receive messages using the same functions for both types of connections.
I found many Python servers and some Js clients for that purpose, but not in Python.
I looked at : Twisted, Tornado, ZeroMQ, py4ws

Have you taken a look at socket.io? It mainly works with websockets but has plenty of fallbacks and is thus supposed to be supported by all browsers.
For the server side, I've used flask together with gevent-socketio. Miquel Gringberg has recently released flask-socketio extension which is a nice abstraction for working with flask and gevent-socketio. Gevent-socketio is built on the nice gevent library.
gevent-socketio should work fine with other Python frameworks, such as Django and Bottle.
I'm not entirely sure if this fits your bill but probably worth a look.

I think Python Socket-IO client could be a good solution :
https://github.com/invisibleroads/socketIO-client
He can interact easily with a Socket.io NodeJs server, with same paradigms.
I tested it and it can default with Websocket connection and fallbacks to xhr-polling, which is great (I actually tested this features through a proxy).
Example :
with SocketIO('http://127.0.0.1', 7777, Namespace, transports=["websocket", "xhr-polling"], proxies={'http': 'http://localhost:8888'}) as socketIO:
socketIO.on('foo',some_callback_function)
socketIO.emit('bar')
socketIO.wait()

Related

How do I use a browser as a client in python sockets?

I tried to search in the internet for this subject, But I didn't found some answers.
If some know how can I use a browser as a client in python sockets it will be very good.
To use the browser as a client to a python (server) socket, you simply need to point it to the right endpoint.
Assuming you are running the browser and the python server on the same machine, and that you're opening port 1234 on the server socket, you simply need to open the localhost:1234 URL in your browser.
Of course, what happens next is entirely dependent on how you handle the communication in your program. Most browsers will understand plain text put directly on the socket, but you probably want to talk HTTP.
It's worth mentioning that using a plain socket to communicate with a browser is, at best, uncommon. There may be better solutions, depending on what, exactly, you want to do:
If you just want to quickly serve a few files from a directory (i.e.: often
called a "directory listing"), you can use SimpleHTTPServer
If you're trying to build a website, you should look into a web framework, such as Django, Flask or CherryPy
If you want a lower-level highly asynchronous scalable communication, Tornado is a popular choice
You might want to consider using websockets. They essentially function like regular TCP sockets, but are initiated with a HTTP handshake, making them suitable for browsers. They are supported in recent versions of all major browsers. There are many libraries available that adapt common python webservers to serve websockets as well, for example:
https://pypi.python.org/pypi/gevent-websocket/
if you like gevent.
They also support an SSL layer, which is called using a url starting with "wss://" on the browser side. More information here:
https://www.websocket.org/

Vertx SockJS server vs sockjs-tornado

I've been inspecting two similar solutions for supporting web sockets via sockJS using an independent Python server, and so far I found two solutions.
I need to write a complex, scalable web socket based web application, and I'm afraid it will be hard to scale Tornado, and it seems Vertx is better with horizontal scaling of web sockets.
I also understand that Redis can be used in conjunction with Tornado for scaling a pub/sub system horizontally, and HAproxy for scaling the SockJS requests.
Between Vertx and Tornado, what is the preferred solution for writing a scalable system which supports SockJS?
Vertx has build-in clustering support. I haven't tried it with many nodes, but it seemed to work well with a few. Internally it uses hazelcast to organise the nodes.
Vertx also runs on a JVM, which has already many monitoring/admin tools which might be useful. So Vertx seems to me like the "batteries included" solution.
You can also use Sockjs Tornado + Rabbit MQ + Memcached in order to scale horizontally.
RabbitMQ brocker will play role of messaging bus from physical Server A to physical Server B.
All information about servers may be stored in memcache. For instance you need to send message M from client-socket C1(A) to client-socket C2(B):
if receiver of A hosted on the same server (by checking memcache), send msg directly using SockJS Router
otherwise send M via RabbitMQ brocker B1(A) (by using routing logic) to B2(A), where SockJS router B can directly send your message to original receiver C2(B).
Since AMQP protocol of RabbitMQ utilizing Erlang, message passing is very stable and quite good for high-load distributed applications. To support my words, look here: http://www.rabbitmq.com/blog/2012/04/25/rabbitmq-performance-measurements-part-2/
Each physical server (with the following power Xenon 4-nodes, MEM 4Gb, HDD- 140 -1000Gb) can handle 3-5 sockjs tornado instances. SockJS implementation also quite good utilizes reverse proxy (HaProxy) via additional params in url.
For distributed testing you can use gemetr, or tsung (based on erlang).
I used this approach in couple of distributed apps.
In addition, don't forget to use Tornado memory as L1 cache.

Python Application that Always Listen to Specific Port Number

I want to create a python application that is always listening to a parametrized port. Whenever there is a request coming from the port, the application will parse the request and do tasks based on the request.
Is this type of application called services? (I have 0 knowledge on services). Where can I find beginner's tips and guides on this type of development?
This is called a server, there are examples at the bottom of the Python socket documentation page.
HTH.
This is socket programming. Writing sockets is cumbersome, you can use any web server written in python. My recommendation is use werkzeug, it is very simple. Meanwhile have a look at Flask which is built on top of werkzeug.
In case you are trying to build your own protocol engine twisted is one which will help you to achieve that.
You can using threads or the Twisted (arguably an easier option) framework to create a server.

How can I implement a secure WebSocket (wss://) server in Python?

I want to serve a real-time stream that has to be securely encrypted due to sensitive data.
I've successfully got normal WebSockets streaming using both gevent and gunicorn as direct frontends, but now I need to make it secure, and am looking for either of these:
A server that can serve secure WebSocket connections that are proxied to (for example) gunicorn which listens for non-secure WebSocket connections.
A framework that can serve secure WebSocket connections directly. I've been looking at Tornado and believe it can handle it, but I'm still open to suggestions.
I use ZeroMQ for the PUB/SUB pattern. If there is a good WebSocket protocol implementation for ZeroMQ, that would be great.
Speed is not super important here as the number of connections will be low. However, the integrity of the data is important.
Assuming that you have your app running correctly over non-SSL Tornado WebSockets, change the listen call from:
app.listen(args.listen_port, args.listen_interface)
to:
app.listen(args.listen_port, args.listen_interface, ssl_options={
"certfile": os.path.join(lib_dir, "mydomain.crt"),
"keyfile": os.path.join(lib_dir, "mydomain.key"),
})
where "mydomain.crt" and "mydomain.key" are your usual SSL certificate files, and lib_dir is the directory they live in.
Don't forget to change the client to use "wss:"
Also note that the port you specify in the listen call will still be used if you specify ssl_options. i.e. it will not revert to listening on port 443.
You can check out the websockify project. Websockify is a proxy that allows a WebSockets capable browser to communicate with a raw binary TCP server. It does this by base64 encoding all traffic to/from the browser. However, the project is modular and the websocket.py file is a general WebSocket server that is designed to be extended (and there a couple of included tests that show how this works). It would be fairly easy to disable the base64 encoding if that is not needed for you project.
Websockify also includes a Javascript library 'websock.js' which is designed to interact with websockify. It will transparently fallback to using web-socket-js (Flash based) if the browser does not have native WebSocket support.
Websockify supports secure (TLS/wss) connections and also is able to answer Flash security policy requests inline on the same port.
Disclaimer: I made websockify.
Take a look at the standalone websockets server of the pywebsocket project supported by Google.
Note that this Python module uses CGIHTTPServer so you need to tweak it to make it secure. I had a similar requirement for a project I was involved in some months ago, so I forked the standalone.py module and removed the dependencies with CGI stuff but I haven't tested secure connections very much.
Maybe you can import OpenSSL.SSL and set up a WebSocketServer as it is in my script. It should use a WebSocketRequestHandler with the proper configuration of use_tls, private_key and certificate in order to implement TLS (Transport Layer Security).
Read the source code. I think you can extend it to meet your needs.
We use Tornado and Tornadio for our realtime app, and I just switched on SSL for websockets, as well as all the other realtime socket.io protocols. It took me just over an hour! more info here:
http://devblog.resolversystems.com/?p=1084
In server side add this to Tornado:
tornadio2.server.SocketServer(application, ssl_options={
"certfile": "server.crt",
"keyfile": "server.key",
})
In client side, refer to this link:
wss://www.example.com:2201/ws, where the 2201 is the secure Websocket's TLS port.

Implement a Web based Client that interacts with a TCP Server

EDIT:Question Updated. Thanks Slott.
I have a TCP Server in Python.
It is a server with asynchronous behaviour. .
The message format is Binary Data.
Currently I have a python client that interacts with the code.
What I want to be able to do eventually implement a Web based Front End to this client.
I just wanted to know , what should be correct design for such an application.
Start with any WSGI-based web server. werkzeug is a choice.
The Asynchronous TCP/IP is a seriously complicated problem. HTTP is synchronous. So using the synchronous web server presenting some asynchronous data is always a problem. Always.
The best you can do is to buffer things and have two processes in your web application.
TCP/IP process that collects data from the remove server and buffers it in a file (or files) somewhere.
WSGI web process which handles GET/POST processing.
GET requests will fetch some or all of the buffer and display it.
POST requests will send a message to the TCP/IP server.
For Web-based, talk HTTP. Use JSON or XML as data formats.
Be standards-compliant and make use of the vast number of libraries out there. Don't reinvent the wheel. This way you have less headaches in the long run.
if you need to maintain a connection to a backend server across multiple HTTP requests, Twisted's HTTP server is an ideal choice, since it's built to manage multiple connections easily.

Categories

Resources