I have a simple Python HTTP server which also connects other HTTP servers to fetch some data. While, connecting to other servers, my server acts as an http client, but the socket created for incoming connection requests still keeps listening from port 8080 (I have a different socket for the client).
The list of other servers that I need to connect and fetch data is stored in a JSON file and I have code like this
with open(myjsonfile, 'r') as json_file:
entries = json.load(json_file)
for entry in entries.keys():
address = entries[entry]['address']
port = int(entries[entry]['port'])
client_port = config.server_port + 50000
host = gethostname()
# request the TXT file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.txt_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = ''
response = sock.recv(2048).decode()
pos = response.find("\r\n\r\n")
txt_data = response[pos+4:]
# processing the received data here
sock.close()
# request the IMG file
sock = socket(AF_INET,SOCK_STREAM)
# sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.bind((host, client_port))
sock.connect((address, port))
reqmsg = "GET /" + config.img_name + " HTTP/1.0\r\n\r\n"
sock.sendall(reqmsg.encode())
response = b''
while True:
recvdata = sock.recv(2048)
if (len(recvdata) < 1):
break
response = response + recvdata
pos = response.find(b"\r\n\r\n")
img_data = response[pos+4:]
# working with the image here
sock.close()
I have to use a set port number for my client because this is how the server identifies me. However, I sometimes get an "Address already in use" error for the second socket.bind() call (the one for the image). Without the bind() calls, my code works fine.
I tried setting socket options (commented out in the code above) and using pycurl with the LOCALPORT property set to client_port value above, but still getting the same error.
What could be the reason behind the error message? I think I open and close the sockets so the operating system should free the port for further use (I think)?
Thanks
PS : This is a small project, not a production system, hence do not bother with "why use port numbers to identify clients"
There is a TIME_WAIT after the session is shutdown to make sure that there are still no live packets in the network.When you re-create the same tuple and one of those packets shows up, it would be treated as a valid packet for your connection this will cause an error state.Usually 2xpacket max age, before the packet is discarded
Before you create a connection with the same tuple, all the packets from the previous session must be dead.
Try using;
...
sock.setsockopt(SOL_SOCKET, SO_REUSEADDR, 1)
sock.listen([backlog])
sock.bind((host, client_port))
...
socket.listen([backlog])
Related
I'm reading about socket module in a web learning site about python, they gave us a simple steps to use socket module like follows:
import socket
with socket.socket() as client_socket:
hostname = '127.0.0.1'
port = 9090
address = (hostname, port)
client_socket.connect(address)
data = 'Wake up, Neo'
data = data.encode()
client_socket.send(data)
response = client_socket.recv(1024)
response = response.decode()
print(response)
when executing I got the error message:
ConnectionRefusedError: [WinError 10061] No connection could be made because the target machine actively refused it.
when I searched about this some sites was talking about server listening and I see in most of tutorials about server socket and they use it along with client one.
so Is the error message related to the fact that I'm not using a server socket and is it a must to use them both
Update:
after reading the answers I got, I went to the test.py file that the course instructors use to evaluate our codes and I see that they make the server socket in it , so the server is already made by them. that take me back to the Error I got why does it happen then.
def server(self):
'''function - creating a server and answering clients'''
self.sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
self.sock.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
self.sock.bind(('localhost', 9090))
self.ready = True
try:
self.sock.listen(1)
conn, addr = self.sock.accept()
self.connected = True
conn.settimeout(15)
while True:
data = conn.recv(1024)
self.message.append(data.decode('utf8'))
if len(self.message) > 1_000_000:
conn.send(
json.dumps({
'result': 'Too many attempts to connect!'
}).encode('utf8'))
break
if not data:
break
Each connection requires a client, which initiates the connection, and a server, which listens for the incoming connection from the client. The code you have shown is for the client end of the connection. In order for this to run successfully you will need a server listening for the connection you are trying to create.
In the code you showed us you have the lines
hostname = '127.0.0.1'
port = 9090
address = (hostname, port)
client_socket.connect(address)
These are the lines that define what server you are connecting to. In this case it is a server at 127.0.0.1 (which is localhost, the same machine you are running the code on) listening on port 9090.
If you want to make your own server then you can look at the documentation for Python sockets and the particular functions you want to know about are bind, listen, and accept. You can find examples at the bottom of that same page.
Given that you appear to have found this code as part of a course, I suspect they may provide you with matching server code at some point in order to be able to use this example.
I need to create a communication between a client and a server with TCP. But I'd like to send and work with "headers". So from the client I'd like to send a header "COMMAND1" and the server returns me something.
I have the following code:
Server
import socket
import threading
bind_ip = '0.0.0.0'
bind_port = 9998
server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind((bind_ip, bind_port))
server.listen(5) # max backlog of connections
print ('Listening on {}:{}'.format(bind_ip, bind_port))
def handle_client_connection(client_socket):
request = client_socket.recv(1024)
print ('Received {}'.format(request))
client_socket.send('Response1!'.encode('utf-8'))
client_socket.close()
while True:
client_sock, address = server.accept()
print ('Accepted connection from {}:{}'.format(address[0], address[1]))
client_handler = threading.Thread(
target=handle_client_connection,
args=(client_sock,) # without comma you'd get a... TypeError: handle_client_connection() argument after * must be a sequence, not _socketobject
)
client_handler.start()
Client
import socket
hostname, sld, tld, port = 'www', 'integralist', 'co.uk', 80
target = '{}.{}.{}'.format(hostname, sld, tld)
# create an ipv4 (AF_INET) socket object using the tcp protocol (SOCK_STREAM)
client = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
# connect the client
# client.connect((target, port))
client.connect(('0.0.0.0', 9998))
# send some data (in this case a HTTP GET request)
client.send('hi'.encode('utf-8'))
# receive the response data (4096 is recommended buffer size)
response = client.recv(4096)
print (response)
Anyone knows the best way to return "Response1!" when the header is "COMMAND1" and "Response2!" when the header is "COMMAND2"?
I can't find examples on how to use headers
EDIT: It doesn't have to be "COMMAND1" or "COMMAND2" it can be a "0" or "1", or anything else.
If you want to add your own header, you just have to:
Make sure your programm finds the start of your message (like, every message beginns "!?&")
Send your own header-data just after the start-symbol of your message.
Maybe mark the end of your message with something or pass a length in your header.
Since TCP will give you a stream of data, it might come to a case, where it just gives you 2 or 3 messages at once. You have to separate these messages by yourself (e.g. by using "?!&" as start of every message).
You can always create your own protocoll as payload of another protocoll. Just as TCP is just payload from the ethernet point of view.
You can do something i have done with my program to accept such headers
Use pickle library to encode a dict headers and send it through socket.
Code will look something like this.
import pickle
def handleSocket(headers:dict):
message = pickle.dumps(headers)
socket.send(message)
For server side, you will be handling it
Gonna initialise the socket recv to 100 kb
def handleReceivedSocket(socket):
message:dict = pickle.loads(socket.recv(102400))
Another way to do this. Is sending a raw json string to the server (just change pickle.dumps,pickle.loads by json.dumps,json.loads
But it will be in raw and less secure.
Last way you can do it is uri encoding. Check w3schools
I am trying to use this code to create an HTTP proxy cache server. When I run the code it starts running and connects to the port and everything but when I try to connect from the browser, for example, it opens a port on 55555 if I type in localhost:52523/www.google.com it works fine but when I try other sites specifically HTTP, for example, localhost:52523/www.microcenter.com or just localhost:52523/google.com it will display localhost didn’t send any data.
ERR_EMPTY_RESPONSE and shows an exception in the console though it creates the cache file on my computer.
I would like to find out how to edit the code so that I can access any website just as I would normally do on the browser without using the proxy server. It should be able to work with www.microcenter.com
import socket
import sys
import urllib
from urlparse import urlparse
Serv_Sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # socket.socket
function creates a socket.
port = Serv_Sock.getsockname()[1]
# Server socket created, bound and starting to listen
Serv_Sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM) # socket.socket
function creates a socket.
Serv_Sock.bind(('',port))
Serv_Sock.listen(5)
port = Serv_Sock.getsockname()[1]
# Prepare a server socket
print ("starting server on port %s...,"%(port))
def caching_object(splitMessage, Cli_Sock):
#this method is responsible for caching
Req_Type = splitMessage[0]
Req_path = splitMessage[1]
Req_path = Req_path[1:]
print "Request is ", Req_Type, " to URL : ", Req_path
#Searching available cache if file exists
url = urlparse(Req_path)
file_to_use = "/" + Req_path
print file_to_use
try:
file = open(file_to_use[5:], "r")
data = file.readlines()
print "File Present in Cache\n"
#Proxy Server Will Send A Response Message
#Cli_Sock.send("HTTP/1.0 200 OK\r\n")
#Cli_Sock.send("Content-Type:text/html")
#Cli_Sock.send("\r\n")
#Proxy Server Will Send Data
for i in range(0, len(data)):
print (data[i])
Cli_Sock.send(data[i])
print "Reading file from cache\n"
except IOError:
print "File Doesn't Exists In Cache\n fetching file from server \n
creating cache"
serv_proxy = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
host_name = Req_path
print "HOST NAME:", host_name
try:
serv_proxy.connect((url.host_name, 80))
print 'Socket connected to port 80 of the host'
fileobj = serv_proxy.makefile('r', 0)
fileobj.write("GET " + "http://" + Req_path + " HTTP/1.0\n\n")
# Read the response into buffer
buffer = fileobj.readlines()
# Create a new file in the cache for the requested file.
# Also send the response in the buffer to client socket
# and the corresponding file in the cache
tmpFile = open(file_to_use, "wb")
for data in buffer:
tmpFile.write(data)
tcpCliSock.send(data)
except:
print 'Illegal Request'
Cli_Sock.close()
while True:
# Start receiving data from the client
print 'Initiating server... \n Accepting connection\n'
Cli_Sock, addr = Serv_Sock.accept() # Accept a connection from client
#print addr
print ' connection received from: ', addr
message = Cli_Sock.recv(1024) #Recieves data from Socket
splitMessage = message.split()
if len(splitMessage) <= 1:
continue
caching_object(splitMessage, Cli_Sock)
There is a few errors in the code :-
The first is that a GET request does not expect the protocol to be passed in as part of the call, nor does it expect the host, instead the GET should be restricted to only the path + query string.
An additional HOST header should be added which specifies which host you are using (i.e www.google.com ) some web servers may be setup to ignore this and instead send you a default page, but results are intermittent.
You should have a peek at the HTTP RFC which gives some other headers that can be passed via HTTP.
You could also install something like Fiddler or Wireshark and monitor some sample HTTP calls and see how the payload is supposed to look.
"Firts of all this stuff is for my university(a challeng) nothing bad is happening in RL"!
Im not quite familier with phyton but, i give it a trie.
The task is to performe a ssl Stripping attack through a phython script.
The output off the script musst be Username and PW (stdout)
My problem is now:
When i download the ssl strip binarie i see a python script which should include the relevant stuff (sslstrip) but, im not soure where i should start here?
I noticed that the two classes ClientsRequest & Stripping Proxy are essential so they are an the bottom.
My problem is how can i i connect the client request class so that it reads the data from the client ?
Im not sure but this could be the config for the port:8080
(the script must listen on 8080)
# Standard socket stuff:
host = '' # do we need socket.gethostname() ?
port = 8080
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.bind((host, port))
sock.listen(1) # don't queue up any requests
# Loop forever, listening for requests:
while True:
csock, caddr = sock.accept()
print "Connection from: " + `caddr`
req = csock.recv(1024) # get the request, 1kB max
I need a little spark in the right direction.
best regards
Rolento
# Loop forever, listening for requests:
while True:
csock, caddr = sock.accept() #< - this is a blocking call
print "Connection from: " + `caddr`
#you never end up getting to this instruction
req = csock.recv(1024) # get the request, 1kB max
you have some problems
sock.accept will block waiting for a connection ... once you get the connection you print their data and loop right back to the waiting for connection ... when in reality you want to do something with that connection
while True:
csock, caddr = sock.accept() #< - this is a blocking call
print "Connection from: " + `caddr`
req = csock.recv(1024) # get the request, 1kB max
will at least recieve a message from the client after accepting the incomming connection
what you probably need to do is hand the connection off to a seperate thread to work with so you can continue to accept new connections , while processing the current connection
There are some other posts about this issue but none did help me with mine.
I'm trying to build a total simple server - client relationship in python
server.py
#!/usr/bin/python
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(("127.0.0.1",8889))
s.listen(1)
try:
while True:
client, add = s.accept()
data = client.recv(1024)
if not data:
print 'No data'
print data
finally:
s.close()
client.py
#!/usr/bin/python
import socket
s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.connect(("127.0.0.1",8889))
try:
while True:
message = 'Foo'
s.send(message)
ans = s.recv(1024)
print ans
finally:
s.close()
I start by running the server first, but when I try to run the client I'm getting this Errno 10054 --> An existing connection was forcibly closed by the remote host
While request the browser with the ip and the related port, I receive some data.
I'm quiet new to networking, so please explain what might be obvious wrong in my code.
EDIT* Main issue is, that the client is somehow wrong, because it returns also an empty string on recv
Thank you in advance
Main issue is, that the client is somehow wrong, because it returns also an empty string on recv
The client isn't receiving anything from the server because the server is not sending anything.
On the server side, after print data, adding client.send(data) will send the string back to the client.
I am guessing:
The server accepts one socket and then does
client, add = s.accept()
data = client.recv(1024)
...
client, add = s.accept()
The client does this in the mean time:
s.send(message)
ans = s.recv(1024) # blocks until timeout
If now an other client connects to the server then client is replaced, the socket garbage collected and closed. s.recv(1024) will then tell that the connection is reset.
Have a look at import select or twisted (google around) to handle multiple connections at once.