I use PHP, JS, HTML, CSS. I'm willing to learn ruby or python if that is the best option.
My next project will involve live data being fed to users from the server and vice versa. I have shell access on my shared server, but I'm not sure about access to ports. Is it possible to use websockets or any other efficient server-client connection on a shared hosting account, and if so, what do I need to do?
For having the best performance and full control of your setup you need "your own" server.
Today there are a huge amount of virtual server providers which means you get full control over your IP but where the physical server is still shared between many clients, meaning cheaper prices and more flexibility.
I recommend utilizing the free tier program at Amazon EC2, you can always resign after the free period. And they have many geographical locations to choose from.
Another provider in Europe that I have been satisfied with is Tilaa
You can probably find many more alternatives that suits your needs on the Webhosting talk forum
Until some weeks ago, websockets deployment required either a standalone server running on a different port, or server side proxies like varnish/haproxy to listen on port 80 and redirecting normal http traffic. The latest nginx versions added built-in support for websockets, but unless your hosting provider uses it, you're out of luck. (note that I don't have personal experience with this nginx feature)
Personally I find that for most applications, websockets can be replaced with Server-sent events instead - a very lightweight protocol which is basically another http connection that stays open on the server side and sends a stream of plaintext with messages separated by double newlines.
It's supported in most decent browsers, but since this excludes internet explorer there are polyfills available here and here
This covers one side of the connection, the one that is usually implemented with long-polling. The other direction can be covered the usual way with XHR. The end result is very similar to websockets IMO, but with a bit higher latency for client->server messages.
Related
I am attempting to create a basic chatroom in python, and I would like to know how I could transmit data from one script to another, preferably without using google drive. If needed to, I could create a webserver on Replit, but I don't do well with HTML or PHP.
Side note: I can't port forward, as my google wifi doesn't accept any level of port forwarding.
I would send messages of about 50 characters every couple seconds
Since you mention port forwarding, I assume you want two chat clients that run on different local networks to talk to each other, for example your own and the chat client of a friend in a remote location, over the internet.
If you (or your counterpart) cannot set up port forwarding, then direct communication between the script on your computer and theirs is hard, if not impossible. The solution is to set up a third computer or service on the internet that can be reached by both clients and use it for relaying messages between them.
A network is typically protected by a firewall of sorts and will typically be behind a router that performs network address translation (NAT) to help multiple devices on a network to simultaneously access services on the internet, whilst all using the same IP address on the internet. Port forwarding fits into that by connecting a specific port from the outside directly to a port on a machine on the inside - without that, an outside computer might be able to reach your IP address, but they could never connect to a computer or program on the inside of the network, as the router wouldn't know what computer to contact, also the firewall might disallow the connection to begin with.
But if your computer on the inside establishes a connection with an accessible server on the internet, expecting a response, that creates a temporary conduit through the router and firewall that can be used by the server to send messages (look up 'hole punching' for more information). And if both computers do this, the server can relay message between both clients. Only the server then needs to run in an environment that doesn't have firewall restrictions or NAT that prevent this.
You could write a simple Python server, that accepts incoming connections and can send several responses and a simple client that connects to it, identifying itself and joining a chatroom, or having a direct conversation with another connected client. There are many techniques that would allow you to do this, but I think web sockets might be a good starting point, as long as you don't plan to do advanced fast or high volume stuff that would require something like a UDP connection.
A library like websockets could be a good starting point, but you may want to start out by figuring out where you would have this service hosted first, since there may be limitations on what you're able and allowed to do.
Also, if all you're looking to do is send simple messages, you may want to stay away from writing your own server an protocols at all - have a look around for open source message servers written in a language you are comfortable with, or that just work out of the box without any development, in which case the language doesn't even really matter, as long as you can connect to it and exchange messages from Python.
I'm writing a websocket connection between a chrome plugin and a desktop application that exclusively talk to each other locally. I know that you can't have two sockets listening on the same port (is this true for all ports?)
Port clashes won't happen for the majority of my users, but the application will run on a few thousand computers; I think I can reasonably expect there to be clashes in rare situations, where another application already occupies the port.
How do bigger applications deal with this potential issue? I'm fairly unfamiliar with networking and (web)socket programming in general.
I guess there are a few non-technical workarounds like these:
Log the issue well, and let support deal with it
Let the user configure a port
But I'd rather have a sound technical way to deal with it, if possible. I could come up with some algorithm that tries a few preconfigured ports, but I expect that this is a fairly well known problem in the industry, and am hoping to learn what the consensus is for this problem.
The question is primarily technology agnostic, but if it matters, my stack is python with this websockets library on the desktop side, and of course JavaScript on the client side.
I don't think there's a reason to overthink it. A list of seemingly unoccupied fallback ports and finally a prompt to specify a port should be fine. If both parties are capable of HDD IO (I don't know how much can a chrome plugin can do), then it's even easier for the plugin and the app to meet.
For two apps on a same host, you may also add an additional loopback interface. Example.
If to speak about large networks, there's a class of so-called service discovery protocols that may help dealing with this kind of issue. Basically they are IP- or UDP-based multicast protocols that advertise different services across the network. Usually these protocols have a standardized port for communications, so it is unlikely to get occupied by another app. Examples are DHCP, Bonjour.
However, dealing with port conflicts are not their primary function. In large networks machines should be under some sort of control and should not have lots of apps grabbing random ports. If a conflict happens with third party apps, the ideal solution is to look for a setting that allows to set ports manually.
Well I don't know how bigger applications actually do it but a way could be to try till you find a free port?
found_port = False
ports = [port1, port2, port3 ....]
for port in ports
try:
mySocket.bind ( ( ip_address, port ) )
found_port = True
except:
pass
if found_port:
break
I'm working on a project to expose a set of methods from various client machines to a server for the purpose of information gathering and automation. I'm using Python at the moment, and SimpleXMLRPCServer seems to work great on a local network, where I know the addresses of the client machines, and there's no NAT or firewall.
The problem is that the client/server model is backwards for what I want to do. Rather than have an RPC server running on the client machine, exposing a service to the software client, I'd like to have a server listening for connections from clients, which connect and expose the service to the server.
I'd thought about tunneling, remote port forwarding with SSH, or a VPN, but those options don't scale well, and introduce more overhead and complexity than I'd like.
I'm thinking I could write a server and client to reverse the model, but I don't want to reinvent the wheel if it already exists. It seems to me that this would be a common enough problem that there would be a solution for it already.
I'm also just cutting my teeth on Python and networked services, so it's possible I'm asking the wrong question entirely.
What you want is probably WAMP routed RPC.
It seems to address your issue and it's very convenient once you get used to it.
The idea is to put the WAMP router (let's say) in the cloud, and both RPC caller and RPC callee are clients with outbound connections to the router.
I was also using VPN for connecting IoT devices together through the internet, but switching to this router model really simplified things up and it scales pretty well.
By the way WAMP is implemented in different languages, including Python.
Maybe Pyro can be of use? It allows for many forms of distributed computing in Python. You are not very clear in your requirements so it is hard to say if this might work for you, but I advise you to have a look at the documentation or the many examples of Pyro and see if there's something that matches what you want to do.
Pyro abstracts most of the networking intricacy away, you simply invoke a method on a (remote) python object.
I tried to search in the internet for this subject, But I didn't found some answers.
If some know how can I use a browser as a client in python sockets it will be very good.
To use the browser as a client to a python (server) socket, you simply need to point it to the right endpoint.
Assuming you are running the browser and the python server on the same machine, and that you're opening port 1234 on the server socket, you simply need to open the localhost:1234 URL in your browser.
Of course, what happens next is entirely dependent on how you handle the communication in your program. Most browsers will understand plain text put directly on the socket, but you probably want to talk HTTP.
It's worth mentioning that using a plain socket to communicate with a browser is, at best, uncommon. There may be better solutions, depending on what, exactly, you want to do:
If you just want to quickly serve a few files from a directory (i.e.: often
called a "directory listing"), you can use SimpleHTTPServer
If you're trying to build a website, you should look into a web framework, such as Django, Flask or CherryPy
If you want a lower-level highly asynchronous scalable communication, Tornado is a popular choice
You might want to consider using websockets. They essentially function like regular TCP sockets, but are initiated with a HTTP handshake, making them suitable for browsers. They are supported in recent versions of all major browsers. There are many libraries available that adapt common python webservers to serve websockets as well, for example:
https://pypi.python.org/pypi/gevent-websocket/
if you like gevent.
They also support an SSL layer, which is called using a url starting with "wss://" on the browser side. More information here:
https://www.websocket.org/
I've been inspecting two similar solutions for supporting web sockets via sockJS using an independent Python server, and so far I found two solutions.
I need to write a complex, scalable web socket based web application, and I'm afraid it will be hard to scale Tornado, and it seems Vertx is better with horizontal scaling of web sockets.
I also understand that Redis can be used in conjunction with Tornado for scaling a pub/sub system horizontally, and HAproxy for scaling the SockJS requests.
Between Vertx and Tornado, what is the preferred solution for writing a scalable system which supports SockJS?
Vertx has build-in clustering support. I haven't tried it with many nodes, but it seemed to work well with a few. Internally it uses hazelcast to organise the nodes.
Vertx also runs on a JVM, which has already many monitoring/admin tools which might be useful. So Vertx seems to me like the "batteries included" solution.
You can also use Sockjs Tornado + Rabbit MQ + Memcached in order to scale horizontally.
RabbitMQ brocker will play role of messaging bus from physical Server A to physical Server B.
All information about servers may be stored in memcache. For instance you need to send message M from client-socket C1(A) to client-socket C2(B):
if receiver of A hosted on the same server (by checking memcache), send msg directly using SockJS Router
otherwise send M via RabbitMQ brocker B1(A) (by using routing logic) to B2(A), where SockJS router B can directly send your message to original receiver C2(B).
Since AMQP protocol of RabbitMQ utilizing Erlang, message passing is very stable and quite good for high-load distributed applications. To support my words, look here: http://www.rabbitmq.com/blog/2012/04/25/rabbitmq-performance-measurements-part-2/
Each physical server (with the following power Xenon 4-nodes, MEM 4Gb, HDD- 140 -1000Gb) can handle 3-5 sockjs tornado instances. SockJS implementation also quite good utilizes reverse proxy (HaProxy) via additional params in url.
For distributed testing you can use gemetr, or tsung (based on erlang).
I used this approach in couple of distributed apps.
In addition, don't forget to use Tornado memory as L1 cache.