I have a service provider who is transmitting data thro' UDP. I want to establish a connection to them, receive & process data (will be with in the 30 sec limit/request)
Is it possible to get & process UDP data in appengine. I am looking for some simple example.
Unfortunately I don't believe it's possible. The docs on the GAE python runtime say this:
An App Engine application cannot:
open a socket or access another host directly. An application can use the App Engine URL fetch service to make HTTP and HTTPS requests to other hosts on ports 80 and 443, respectively.
You would most likely use the socket module for UDP communication, which you can import on GAE, but does not contain any socket functionality (the module is empty).
Update for GAE 1.7.7:
Outbound sockets moved to Preview
Outbound sockets is now in preview
in this release for Java and Python. With outbound sockets,
billing-enabled App Engine applications can now make outbound
connections with TCP or UDP sockets.
Note the word "outbound" above -- you still cannot create a listen socket.
You could run a separate agent on a cloud host like DigitalOcean or Amazon EC2 that proxies this protocol and makes itself available to Google App Engine via ordinary HTTP or web sockets.
Related
I'm currently developing a multiplayer gaming app using flutter framework and AWS services. Connections are established through web sockets. I really don't understand how to establish these connections with ec2. Do I need to develop api using node.js or python?
Please note down if there are any useful links.
WebSockets, on the other hand, allow for sending message-based data, similar to UDP, but with the reliability of TCP. WebSocket uses HTTP as the initial transport mechanism, but keeps the TCP connection alive after the HTTP response is received so that it can be used for sending messages between client and server. WebSockets allow us to build “real-time” applications without the use of long-polling.
https://sookocheff.com/post/networking/how-do-websockets-work/
Seems like something that will work fine for AWS ingress.
I am having trouble in sending data from tcp client in my esp32 board to my python django server,I am not familiar with setting channels in Django ,is there a way so that i can send the data and display in my page?
in order for your microcontroller (esp32) communicate with your own server side code first you need to define protocol you're going to use:
A. TCP:
TCP relies on IP which provides address to communicate between computers. TCP/IP is a basis for internet and other networks.
B. HTTP:
HTTP mostly used by browser (IE, Google Chrome). It rides on top of TCP which provides a safe and reliable link between two computers because if packet get lost - it can be safely re-transmitted.
After deciding protocol that you're going to use now you need suitable server side code. In python there are several library / framework that you can use:
A. HTTP:
Django, Flask, AIOHTTP (all of this supports sending and receiving JSON (REST)), I preferably use one of this framework for my IoT Projects.
B. TCP: If your microcontroller is very minimal and doesn't support HTTP/JSON, you can use a simple SocketServer or Tornado TCP Server. Don't worry even though communication between your board and server done through TCP you can still import django's libraries and use django's ORM.
I'm creating an extremely simple Vega visualization viewer: it's a one file module that serves a base HTML page containing just the Vega graphic and an HTML5 EventSource of updates. The user (me) is working in a Python shell through ssh, creates an object representing the viewer, which prints its IP and port for the user to paste into their (my) web browser. This HTTP server doesn't serve files or take input from clients, so I don't see any security concerns.
The part I'm unsure of is how to set (host, port) such that my web browser can find the HTTP server running in the remote Python. I've been experimenting all afternoon, and I don't know if I'm misunderstanding what's supposed to happen or if the servers I use have changed their access policies.
Here's a minimal example:
import SimpleHTTPServer
import SocketServer
Handler = SimpleHTTPServer.SimpleHTTPRequestHandler
httpd = SocketServer.TCPServer((host, port), Handler)
print(httpd.server_address)
httpd.serve_forever()
If I'm running this locally and want to ensure that outside viewers cannot access it, do I set host to "127.0.0.1" because that means a client would have to access it as 127.0.0.1, which can only happen locally? In this case, port can be 0 to get any open port.
If I'm running this remotely want to to ensure that outside viewers can access it, do I set host to "" or "0.0.0.0" because that means that a client can access it as any address that makes its way to the server? In this case, I might not be able to set port to 0 because many of those ports might be blocked, or is the OS smarter about this?
Basically, how is access control in Python's SocketServer supposed to work?
This is basic TCP. Nothing to do with Python.
If you listen at 127.0.0.1, only clients running in the same host can connect.
If you listen at 0.0.0.0, anybody can connect, firewalls permitting.
I recently discovered autobahn python and js as a comfortable method to establish a pub/sub server and corresponding client even with rpc-calls.
After looking through the tutorials, I set up a test version with a websocket server and a webserver running on the same port. The server sends periodically data to the client via websockets. The html the user gets lies on the localhost root. All that works fine.
However, what I want to accomplish is: Setup a pub/sub server and a webserver listening on the same port.
The tutorials show only how to setup these on two different ports (as shown at http://autobahn.ws/python/tutorials/pubsub).
Im very new to python in general and autobahn and twisted especially.
Any advice would be really nice!
Thanks very much!
Marc
Sure. You can run a WAMP/WebSocket server and a plain old Web server on one port using Autobahn. Here is an example for pure WebSocket and here is one for WAMP.
Disclaimer: I am author of Autobahn and work for Tavendo.
When using WAMP while having HTTP and WS servers listening on the same port you will need to start your instance of WampServerFactory manually as explained here.
factory = WampServerFactory("ws://localhost:8080")
factory.protocol = YourServerProtocolClass
factory.startFactory() # <--- need to call this manually
resource = WebSocketResource(factory)
root = File(".")
root.putChild("ws", resource)
For more details please see this complete example.
I would put nginx as a frontend that forwards each call either to pubsub or to web... Recent Nginx supports WebSocket forwarding.
Or you man write something similar with Twisted :)
Another alternative would be to adapt autobahn.websocket.WebSocketServerProtocol and its subclass autobahn.wamp.WampServerProtocol to Twisted.web. It should be possible.
I want to serve a real-time stream that has to be securely encrypted due to sensitive data.
I've successfully got normal WebSockets streaming using both gevent and gunicorn as direct frontends, but now I need to make it secure, and am looking for either of these:
A server that can serve secure WebSocket connections that are proxied to (for example) gunicorn which listens for non-secure WebSocket connections.
A framework that can serve secure WebSocket connections directly. I've been looking at Tornado and believe it can handle it, but I'm still open to suggestions.
I use ZeroMQ for the PUB/SUB pattern. If there is a good WebSocket protocol implementation for ZeroMQ, that would be great.
Speed is not super important here as the number of connections will be low. However, the integrity of the data is important.
Assuming that you have your app running correctly over non-SSL Tornado WebSockets, change the listen call from:
app.listen(args.listen_port, args.listen_interface)
to:
app.listen(args.listen_port, args.listen_interface, ssl_options={
"certfile": os.path.join(lib_dir, "mydomain.crt"),
"keyfile": os.path.join(lib_dir, "mydomain.key"),
})
where "mydomain.crt" and "mydomain.key" are your usual SSL certificate files, and lib_dir is the directory they live in.
Don't forget to change the client to use "wss:"
Also note that the port you specify in the listen call will still be used if you specify ssl_options. i.e. it will not revert to listening on port 443.
You can check out the websockify project. Websockify is a proxy that allows a WebSockets capable browser to communicate with a raw binary TCP server. It does this by base64 encoding all traffic to/from the browser. However, the project is modular and the websocket.py file is a general WebSocket server that is designed to be extended (and there a couple of included tests that show how this works). It would be fairly easy to disable the base64 encoding if that is not needed for you project.
Websockify also includes a Javascript library 'websock.js' which is designed to interact with websockify. It will transparently fallback to using web-socket-js (Flash based) if the browser does not have native WebSocket support.
Websockify supports secure (TLS/wss) connections and also is able to answer Flash security policy requests inline on the same port.
Disclaimer: I made websockify.
Take a look at the standalone websockets server of the pywebsocket project supported by Google.
Note that this Python module uses CGIHTTPServer so you need to tweak it to make it secure. I had a similar requirement for a project I was involved in some months ago, so I forked the standalone.py module and removed the dependencies with CGI stuff but I haven't tested secure connections very much.
Maybe you can import OpenSSL.SSL and set up a WebSocketServer as it is in my script. It should use a WebSocketRequestHandler with the proper configuration of use_tls, private_key and certificate in order to implement TLS (Transport Layer Security).
Read the source code. I think you can extend it to meet your needs.
We use Tornado and Tornadio for our realtime app, and I just switched on SSL for websockets, as well as all the other realtime socket.io protocols. It took me just over an hour! more info here:
http://devblog.resolversystems.com/?p=1084
In server side add this to Tornado:
tornadio2.server.SocketServer(application, ssl_options={
"certfile": "server.crt",
"keyfile": "server.key",
})
In client side, refer to this link:
wss://www.example.com:2201/ws, where the 2201 is the secure Websocket's TLS port.