I am learning Python; specifically, I'm learning about network architecture and HTTP requests. The course example below demonstrates how to write a simple web browser.
import socket
mysock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
mysock.connect(('data.pr4e.org', 80))
cmd = 'GET http://data.pr4e.org/romeo.txt HTTP/1.0\r\n\r\n'.encode()
mysock.send(cmd)
while True:
data = mysock.recv(512)
if (len(data) < 1):
break
print(data.decode(),end='')
mysock.close()
I understand that the .connect() function starts the two-way communication, effectively "opening the tunnel". The syntax is socketname.connect(('address'),port)
However, I am wondering if there is a difference between that function and the CONNECT method described here. That syntax is:
CONNECT serverurl: port
Specifically, when is it appropriate to use one or the other?
The .connect() function is connecting the TCP socket to the remote server, which allows data to be sent to and received from the server. In your case you're using the TCP socket to send HTTP commands, and receive corresponding HTTP responses.
The HTTP CONNECT method is something completely different. It's a type of HTTP command, which include GET (the one you're using), HEAD, POST, PUT, etc that can be sent over your connected socket. CONNECT is related to HTTP tunneling (i.e. having the web server proxy your requests to another server).
When you send a CONNECT command, you're basically instructing the server to forward future HTTP commands to a different HTTP server. So it is sort of like being 'connected' to that other server in a way.
Related
I'm sending data via sockets between godot and python like this:
godot:
var socket = PacketPeerUDP.new()
socket.set_dest_address("127.0.0.1", 6000)
var data={...}
socket.put_packet(JSON.print(data).to_ascii())
python server:
s= socket.socket(socket.AF_INET,socket.SOCK_DGRAM)
s.bind(("127.0.0.1", 6000))
while True:
data = s.recvfrom(1024)
but the problem is even when the python server is not running the godot code sends the data instead of giving an error that the server is not available
I even tried var err=socket.set_dest_address("127.0.0.1", 6000) hopin this would print out the error
but it always prints 0 whether the python server is running or not
so how do I check if the server is available or not?
This is UDP we are talking about. So there isn't really a session or connection established. Also there isn't really an acknowledged package. So at the end the only solution is to implement your own reliability protocol on top of it (e.g. have the server respond and the client wait for the response). Try searching dor UDP reliability on the gamedev site.
The return values for set_dest_address are ERR_CANT_RESOLVE (the IP is not valid) or OK.
The returns values of put_packet. It can return ERR_BUSY (send buffers are full), FAILED (the socket is otherwise in use) or OK.
I am using the Kaldi speech recognition toolkit's "online 2-tcp-nnet3-decode-faster". The server receives raw audio and sends the text corresponding to this audio live. In other words, when using such a server, the idea is to start transcribing audio as soon as it is sent.
If the server is busy serving one client request, it cannot handle a second one. The second request will remain idle until the first transcription completes and the first client closes the connexion.
I would like to build a python client to communicate with the TCP server via websockets. I am able to create a socket connexion, however, I am still not able to determine whether the server is already serving another client so that I can try other servers on other ports or, create a new server instance on the fly.
I am using something like the snippet below. The call to connect succeeds even when the server is serving another client.
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((HOST, PORT))
I have this code:
import socket
import socks
socks.setdefaultproxy(socks.PROXY_TYPE_SOCKS4, "IP_SOCK", PORT_SOCK, True)
s = socks.socksocket(socket.AF_INET, socket.SOCK_STREAM)
s.connect((target,port))
s.close()
print("Connection tested.")
It works, but the lenght of the packets is 0. I'd like to add a payload data, how can I do that?
For example, here I have this working code with scapy library:
import socket
import socks
from scapy.all import *
def send(src_ip, dst_ip, dst_port):
ip = IP(src=src_ip, dst=dst_ip)
src_port = random.randint(20, 65000)
transport = TCP(sport=src_port, dport=dst_port, flags="S")
send(ip/transport/b"mydata")
if __name__ == '__main__':
send('SRC_IP', 'DST_IP', DST_PORT)
And it send indeed the "mydata" payload. If I intercept it with tcpdump, I can see the lenght of data field and "mydata" in the syn packet.
I would like to send it through a sock or a http proxy, but I don't know how can I do it.
The fact is that I don't need to establish the connection, because I want to send packets even if the port is closed.
Some people continue to tell me that I can't use socks to send tcp packets. It's not true. SOCKS acts in layer 4, so it's perfectly doeable, the thing is that I don't know how to do it in Python since I'm new to it.
Some people continue to tell me that I can't use socks to send tcp packets. It's not true.
It is true even if you don't like it. A proxy (HTTP proxy or Socks proxy) is not a packet forwarder on the network level. It is instead a payload forwarder at the application level. If you want a packet forwarder you need a real VPN (with the focus on "N", i.e. network level) instead.
When using a Socks or HTTP proxy one needs to establish a TCP connection to the proxy first. Inside this TCP connection one then does some initial handshaking like authentication against the proxy and specification of the target one likes to reach. The proxy then connects to the target and once this is done there are two TCP connections: one between client and proxy and another between proxy and server. The proxy will then read on both of these connections and forward the data received to the peer, i.e. data received from the client will be forwarded to the server etc. The proxy might also modify data in transit.
Since the proxy only forwards the data at the application level there is no way to send crafted TCP packets trough the proxy. And since the proxy maintains two independent TCP connections and just forwards application level data even the incoming and outgoing packet sizes might be different, i.e. two incoming packets can be merged in one outgoing packet or might result in three etc. Since TCP is just byte stream packet sizes don't matter at the application level.
you can use sendall(); for example: s.sendall("GET / HTTP/1.1 ...")
For more details, see https://pypi.org/project/PySocks/ , where I obtained this example from, as well as https://docs.python.org/3/library/socket.html (socks mimics python's built in sockets)
Is it possible to have one protocol connect to another protocol on the same server? My goal is to accept a request for one protocol and then pass that request to a different protocol and have that second protocol return values to whatever client is connected to it.
I'm thinking you would transfer the onMessage request to the other protocol some how.
I don't have any code to show as I don't know where to start, but any code examples would be appreciated.
What you're asking for sounds like a proxy server. A proxy can simply be a middleman that speaks the same protocol out both ends (as in a typical http proxy) or it can be some sort of translator that has one protocol coming in and another protocol going out.
So, supposed you want a browser to be able to use a webSocket connection to speak to some other server that doesn't speak the webSocket protocol. You could implement a proxy server yourself that allows the browser to connect to it and then, via your proxy, it could send/receive messages with the other server that speaks a different protocol.
To implement a proxy server like this, you would do the following:
Create a server process that listens for incoming webSocket connections. This would allow the browser to connect to your proxy.
Once connected, the browser would send a message (of your own design) over the webSocket.
Your proxy would receive that message and translate it to the protocol of the other server,
Your proxy would then connect to that other server and send the message to the other server.
Your proxy could then receive a response from that other server and then, if needed, send a translated response message back to the browser over the webSocket.
It would be the proxy's responsibility to translate each message data from what it received over the webSocket to whatever format/protocol the other server speaks.
It would be an implementation choice whether you maintained a dedicated connection between the proxy and the other server for each webSocket connection or whether you directed all requests over one dedicated connection or whether you created a new connection upon demand only for the duration of a given request. Which makes the most sense depends entirely upon the characteristics of the other server, number of requests and the work that is being done.
There're basically two issues I'd like to resolve:
Client side send query string when initializing the connection to server
Server side validate user token in handshake (not after the connection is established and then validate streaming message that contains the token) and set user session accordingly.
I read a article (https://auth0.com/blog/2014/01/15/auth-with-socket-io/) that talks about this process implemented in nodejs, just wonder if the same function can be achieved by using python. (Currently I'm doing some research on twisted but haven't found anything similar)
PS: guess it's helpful to demo the use case as well. A user may login to your server over normal http then server will issue him/her an valid accessToken. Then this user may need to establish a socket connection with the server (or some other server), then the server needs to figure out who the user is and validate before establishing the socket connection.
Query strings are part of HTTP URLs.
If you're building a TCP socket server instead of an HTTP server, you don't get URLs—or headers, or anything else out-of-band.* All you get is a stream of data. You need to come up with a protocol for your data that you can fit the token into.
This means the server can't "figure out who the user is and validate before establishing the socket connection". It has to establish the socket connection, read the first message, parse it, validate the token, and then drop or continue the connection. (You can, of course, put up a front-end server that accepts connections, validates them, and then migrates or proxies them to the real back-end server. But someone has to accept, read, and parse.)
Note that this is exactly what HTTP does—it can't see the query string until it accepts the connection and reads the first line of data.
Meanwhile, the example you're looking at appears to be using WebSockets. A WebSockets client can't talk to a socket server (well, unless you build a WebSockets server on top of your socket server, or a proxy in front of it) in the first place.
* This isn't quite true. You can cram 40 bytes of options into TCP header extensions. But then you have to go below the level people are usually talking about when they say "socket server"—and there's a good chance it won't make it through the internet. Also, TCP does have a concept of "out-of-band" data, but that isn't relevant here; you still have to accept the connection and read from it to get an OOB data.