Since I'm new to programming, there might be other mistakes in the code I haven't detected, so if you notice any rookie mistakes or bad style please point out. I'm trying to implement a simple web server, but keep getting an Indentation Error at the part where server creates an HTTP response message consisting of the requested file preceded by header lines.
from socket import *
import sys
host = 'localhost'
serverPort = int(sys.argv[1])
serverSocket = socket(AF_INET, SOCK_STREAM)
serverSocket.bind((host,serverPort))
serverSocket.listen(1)
while True:
connectionSocket, addr = serverSocket.accept()
try:
#if received correct stuff
File_Wanted = connectionSocket.recv(1024)
File_Name = File_Wanted.split()[1]
#open and read line in index.htm
File = open('index.htm','r')
output = File.read()
#should I close it here or somewhere else?
File.close()
#take care of header first
connectionSocket.send('HTTP/1.0 200 OK\r\n')
#looks good, then send to client(browser)
for i in range(0, len(output)):
connectionSocket.send(output[i])
connectionSocket.close()
IOError:
#Send response message for file not found
connectionSocket.send('404 Not Found')
#Close client socket
connectionSocket.close()
serverSocket.close()
When I build on sublime, I got.
connectionSocket.send(output[i])
^
IndentationError: expect an idented block
Python uses whitespace for defining blocks, you need to indent the lines after the for statement:
for i in range(0, len(output)):
connectionSocket.send(output[i])
connectionSocket.close()
and you need an except statement before checking the error:
except IOError:
Related
I’m trying to send and receive file through TCP socket
There are a lot of problems
1. When the client connects to the server. The server does not “print Client connected ..” but it prints after using the command.
2. When I use the ‘put’ command at the server occur an error socket.error: [Errno 107] Transport endpoint is not connected but the file image is already uploaded to the server.
3. When I use the ‘get’ command at the client. I can’t continue to use another command.
4. The last problem is the client can’t quit and list file from the server. It shows AttributeError: 'module' object has no attribute 'send'
Server
import socket
import sys
import os
HOST = 'localhost'
PORT = 3820
socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket.bind((HOST, PORT))
socket.listen(1)
while (1):
conn, addr = socket.accept()
print 'Client connected ..'
reqCommand = conn.recv(2048)
print 'Client> %s' %(reqCommand)
if (reqCommand == 'quit'):
break
#list file on server
elif (reqCommand == 'lls'):
start_path = os.listdir('.') # server directory
for path,dirs,files in os.walk(start_path):
for filename in files:
print os.path.join(filename)
else:
string = reqCommand.split(' ', 1) #in case of 'put' and 'get' method
reqFile = string[1]
if (string[0] == 'put'):
with open(reqFile, 'wb') as file_to_write:
while True:
data = socket.recv(1024)
# print data
if not data:
break
# print data
file_to_write.write(data)
file_to_write.close()
break
print 'Receive Successful'
elif (string[0] == 'get'):
with open(reqFile, 'rb') as file_to_send:
for data in file_to_send:
conn.sendall(data)
print 'Send Successful'
conn.close()
socket.close()
Client
import socket
import sys
import os
HOST = 'localhost'
PORT = 3820
def put(commandName):
socket1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket1.connect((HOST, PORT))
socket1.send(commandName)
string = commandName.split(' ', 1)
inputFile = string[1]
with open(inputFile, 'rb') as file_to_send:
for data in file_to_send:
socket1.sendall(data)
print 'PUT Successful'
socket1.close()
return
def get(commandName):
socket1 = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
socket1.connect((HOST, PORT))
socket1.send(commandName)
string = commandName.split(' ', 1)
inputFile = string[1]
with open(inputFile, 'wb') as file_to_write:
while True:
data = socket1.recv(2048)
# print data
if not data:
break
# print data
file_to_write.write(data)
file_to_write.close()
print 'GET Successful'
socket1.close()
return
msg = raw_input('Enter your name: ')
while(1):
print 'Instructions'
print '"put [filename]" to upload the file to the server '
print '"get [filename]" to download the file from the server '
print '"ls" to list all files in the client'
print '"lls" to list all files in the server'
print '"quit" to exit'
sys.stdout.write('%s> ' % msg)
inputCommand = sys.stdin.readline().strip()
if (inputCommand == 'quit'):
socket.send('quit')
break
#list file on client
elif (inputCommand == 'ls'):
start_path = os.listdir('.') # client directory
print start_path
#list file on server
elif (inputCommand == 'lls'):
socket.send('lls')
else:
string = inputCommand.split(' ', 1)
if (string[0] == 'put'):
put(inputCommand)
elif (string[0] == 'get'):
get(inputCommand)
socket.close()
Among other things, you need to add "framing" to your transfer protocol. When you do a send on a stream socket, the data gets added to a buffer that will eventually be delivered to the other side. However, the size of that buffer is not transmitted to the other side. In other words, say you send an initial segment with the command "put myfile.txt". Then you send the data from myfile.txt. Now because you are using the file object iterator (for data in file_to_send), you are actually sending it a line at a time (arguably, for a file transfer protocol, it would make more sense to read and send fixed chunks but this would work too). Let's assume the first line of myfile.txt is "The quick brown fox\n"
When the server does its first receive, it could receive "put " or "put myfile.txt" or "put myfile.txtThe quick brown fox\n" or the put command plus the entire file contents. That's because the stream protocol (TCP) does not maintain message boundaries for you.
Now in practice, you may be receiving only the "put myfile.txt" in the first receive, but it's very unwise to count on that because it's dependent on the timing of all sorts of factors on both sending and receiving systems that are outside your control.
So, there are two common ways of handling this:
Add a length at the beginning that delineates the size of the command and any command argument (so that you know where in the stream the actual file data to be transferred begins). (Most binary file transfer protocols work this way.)
Add some known delimiter at the end of your command -- for example, '\n'. HTTP, for example, works this way.
And likewise your receiving side needs to ensure that it is reading exactly the amount needed at each point in order to preserve the full content of the file being transferred.
That means you either (a) are careful to recv exactly the number of bytes you need for the command data, then separately process the file content, or (b) recv an initial chunk of data into a buffer, then carve off exactly what you need for the "command", and ensure the rest will be processed later as file data. Option (b) can often be accomplished by building a file object on top of the socket (see socket.makefile), then using the file object's readline method to obtain only the first line.
Other issues with your code:
The line socket = socket.socket(socket.AF_INET, socket.SOCK_STREAM) is a very bad idea. You have just hidden the entire socket module by creating a variable with the same name. For example, if you attempted to refer to socket.AF_INET again on the subsequent line, you'd get the exception AttributeError: '_socketobject' object has no attribute 'AF_INET'. You should name the variable something else, say socket1 as you did on the client side.
On the client side, you have the opposite problem. You're attempting to use a socket object method, but providing the socket module object instead. (That's why you get AttributeError: 'module' object has no attribute 'send'.) You need to reorganize your code so that you call the send method on a connected socket object as you are doing in the put and get functions.
The error socket.error: [Errno 107] Transport endpoint is not connected occurs because you are attempting to recv on the listening socket, not the connected one (conn -- which is returned by socket.accept). The only thing you can do with a listening socket, is accept new connections (or close).
You should be using sendall instead of send to ensure that every byte gets sent. Generally, all the data will get sent with send too, but there are corner cases where that doesn't happen.
Your file-receiving loop in the server begins with while True: but then always breaks. Hence it will only receive the first chunk of data (up to 1024 bytes). If the file is larger than that, you will certainly end up truncating it.
Your server file listing function (lls command) is not sending its output back to the client, it's only printing it to the server side's console. (And don't forget, you will need to provide a delimiter of some kind between the file names you send back or they will end up all concatenated into a single big string.)
I'm just starting out with Python and I'm trying to code a simple web server. Everything seems to be work except I'm having one small problem. When I request a specific file like Test.html from my web server, the data in the HTML file is repeated over and over in my client like its stuck in a loop. So instead of seeing just "Test" displayed in the web client once, I see "Test Test Test Test Test Test...many times over". This is probably a simple error but I was hoping someone could point me in the right direction.
Thanks for you help!
import socket
import sys
serversocket = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print("Socket Created!!")
try:
#bind the socket
#fill in start
server_address = ('localhost', 6789)
serversocket.bind(server_address)
#fill in end
except socket.error as msg:
print("Bind failed. Error Code: " + str(msg[0]) + "Message: " +
msg[1])
sys.exit()
print("Socket bind complete")
#start listening on the socket
#fill in start
serversocket.listen(1)
#fill in end
print('Socket now listening')
while True:
#Establish the connection
connectionSocket, addr = serversocket.accept()
print('source address:' + str(addr))
try:
#Receive message from the socket
message = connectionSocket.recv(1024)
print('message = ' + str(message))
#obtian the file name carried by the HTTP request message
filename = message.split()[1]
print('filename = ' + str(filename))
f = open(filename[1:], 'rb')
outputdata = f.read()
#Send the HTTP response header line to the socket
#fill in start
connectionSocket.send(bytes('HTTP/1.1 200 OK\r\n\r\n','UTF-
8'))
#fill in end
#Send the content of the requested file to the client
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata)
#close the connectionSocket
#fill in start
connectionSocket.close()
#fill in end
print("Connection closed!")
except IOError:
#Send response message for file not found
connectionSocket.send(bytes("HTTP/1.1 404 Not
Found\r\n\r\n","UTF-8"))
connectionSocket.send(bytes("<html><head></head><body><h1>404
Not Found</h1></body></html>\r\n","UTF-8"))
#Close the client socket
#fill in start
connectionSocket.close()
serverSocket.close()
You are stuck in a loop :)
#fill in end
#Send the content of the requested file to the client
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata)
You're sending the contents of outputdata the number of times however long the length of the file is.
You only need connectionSocket.send(outputdata) without the for..loop to send it once.
Also, make sure you close the file you read the contents from. (f.close())
I have a homework assignment which involves implementing a proxy cache server in Python for web pages. Here is my implementation of it
from socket import *
import sys
def main():
#Create a server socket, bind it to a port and start listening
tcpSerSock = socket(AF_INET, SOCK_STREAM) #Initializing socket
tcpSerSock.bind(("", 8030)) #Binding socket to port
tcpSerSock.listen(5) #Listening for page requests
while True:
#Start receiving data from the client
print 'Ready to serve...'
tcpCliSock, addr = tcpSerSock.accept()
print 'Received a connection from:', addr
message = tcpCliSock.recv(1024)
print message
#Extract the filename from the given message
filename = ""
try:
filename = message.split()[1].partition("/")[2].replace("/", "")
except:
continue
fileExist = False
try: #Check whether the file exists in the cache
f = open(filename, "r")
outputdata = f.readlines()
fileExist = True
#ProxyServer finds a cache hit and generates a response message
tcpCliSock.send("HTTP/1.0 200 OK\r\n")
tcpCliSock.send("Content-Type:text/html\r\n")
for data in outputdata:
tcpCliSock.send(data)
print 'Read from cache'
except IOError: #Error handling for file not found in cache
if fileExist == False:
c = socket(AF_INET, SOCK_STREAM) #Create a socket on the proxyserver
try:
srv = getaddrinfo(filename, 80)
c.connect((filename, 80)) #https://docs.python.org/2/library/socket.html
# Create a temporary file on this socket and ask port 80 for
# the file requested by the client
fileobj = c.makefile('r', 0)
fileobj.write("GET " + "http://" + filename + " HTTP/1.0\r\n")
# Read the response into buffer
buffr = fileobj.readlines()
# Create a new file in the cache for the requested file.
# Also send the response in the buffer to client socket and the
# corresponding file in the cache
tmpFile = open(filename,"wb")
for data in buffr:
tmpFile.write(data)
tcpCliSock.send(data)
except:
print "Illegal request"
else: #File not found
print "404: File Not Found"
tcpCliSock.close() #Close the client and the server sockets
main()
I configured my browsers to use my proxy server like so
But my problem when I run it is that no matter what web page I try to access it returns a 404 error with the initial connection and then a connection reset error with subsequent connections. I have no idea why so any help would be greatly appreciated, thanks!
There are quite a number of issues with your code.
Your URL parser is quite cumbersome. Instead of the line
filename = message.split()[1].partition("/")[2].replace("/", "")
I would use
import re
parsed_url = re.match(r'GET\s+http://(([^/]+)(.*))\sHTTP/1.*$', message)
local_path = parsed_url.group(3)
host_name = parsed_url.group(2)
filename = parsed_url.group(1)
If you catch an exception there, you should probably throw an error because it is a request your proxy doesn't understand (e.g. a POST).
When you assemble your request to the destination server, you then use
fileobj.write("GET {object} HTTP/1.0\n".format(object=local_path))
fileobj.write("Host: {host}\n\n".format(host=host_name))
You should also include some of the header lines from the original request because they can make a major difference to the returned content.
Furthermore, you currently cache the entire response with all header lines, so you should not add your own when serving from cache.
What you have doesn't work, anyway, because there is no guarantee that you will get a 200 and text/html content. You should check the response code and only cache if you did indeed get a 200.
When I run below python script and try to do a GET request for a .html file via safari, I get the content in raw format i.e. along with the HTML tags. How do I retrieve html pages and also if I do a GET for images it says the file is damaged.
from socket import *
serverSocket = socket(AF_INET, SOCK_STREAM) #create a socket
serverPort = 7000
serverSocket.bind(('',serverPort))
serverSocket.listen(1)
while True:
print 'Ready to serve . . .'
connectionSocket, addr = serverSocket.accept() #create socket for client
try:
message =connectionSocket.recv(1024) #receive message from client
print message
filename = message.split()[1]
f = open(filename[1:])
outputdata =f.read()
#Send HTTP header line into socket
connectionSocket.send('\nHTTP/1.x 200 OK\n')
#Send contents of the requested file to the client
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i])
connectionSocket.close()
print 'File Received'
except IOError:
connectionSocket.send('\n404 File Not Found\n')
connectionSocket.close()
serverSocket.close()
You need to tell the client that you are sending HTML back. Before sending the data add:
connectionSocket.send("Content-type: text/html\r\n")
Also, you may be seeing the HTTP response header you're sending back as well, right? If so, that's because you have a leading \n which terminates the headers and makes the rest of the body that gets sent back, so change that line to
connectionSocket.send('HTTP/1.x 200 OK\r\n')
and make sure you put a blank line when you are done with all the headers, and also that end of line in HTTP should be \r\n not just \n though I wouldn't be surprised if browsers handled it with just \n
From command line
client.py Aaron 12000 HelloWorld.html GET
client.py
def main(argv):
serverName = argv[0]
serverPort = int(argv[1])
fileName = argv[2]
typeOfHttpRequest = argv[3]
clientSocket = socket(AF_INET, SOCK_STREAM)
clientSocket.connect((serverName, serverPort))
clientSocket.send(typeOfHttpRequest + " " + fileName + " HTTP/1.1\r\n\r\n")
content = clientSocket.recv(1024)
print content
clientSocket.close()
if __name__ == "__main__":
main(sys.argv[1:])
server.py
while True:
#Establish the connection
print 'Ready to serve....'
connectionSocket, addr = serverSocket.accept()
try:
message = connectionSocket.recv(1024)
typeOfRequest = message.split()[0]
filename = message.split()[1]
print typeOfRequest
print filename
f = open(filename[1:])
outputdata = f.read()
if typeOfRequest == 'GET':
for i in range(0, len(outputdata)):
connectionSocket.send(outputdata[i])
connectionSocket.close()
elif typeOfRequest == 'HEAD':
connectionSocket.send(True)
except IOError:
connectionSocket.send('HTTP/1.1 404 Not Found')
connectionSocket.close()
serverSocket.close()
I have put HelloWorld.html in the same directory as server.py but this always generates an IOError. Anyone know why it might be the case?
The files are located in C:\Networking
os.getcwd shows C:\Networking
HelloWorld.html is located in C:/networking/HelloWorld.html
Filename prints out correctly.
As you might have noticed, you were trying to strip the / from the beginning of the URL, though it was not there. However, there are other errors in your code, which mean that it does not work like a HTTP server:
First of all, recv() is not guaranteed to read all the data - even if there would be total of 1024 bytes written to a socket, recv(1024) could return just 10 bytes, say. Thus it is better to do in a loop:
buffer = []
while True:
data = connection_socket.recv(1024)
if not data:
break
buffer.append(data)
message = ''.join(buffer)
Now message is guaranteed to contain everything.
Next, to handle the header lines of the request, you can use
from cStringIO import StringIO
message_reader = StringIO(message)
first_line = next(message_reader)
type_of_request, filename = message.split()[:2]
With this it is easier to extend your code for more complete HTTP support.
Now open the file with open, with with statement:
with open(filename) as f:
output_data = f.read()
This ensures that the file is closed properly too.
Finally, when you respond to the request, you should answer with HTTP/1.0, not HTTP/1.1 as you are not supporting the full extent of HTTP/1.1. Also, even an OK response needs to respond with full headers, say with:
HTTP/1.1 200 OK
Server: My Python Server
Content-Length: 123
Content-Type: text/html;charset=UTF-8
data goes here....
Thus your send routine should do that:
if typeOfRequest == 'GET':
headers = ('HTTP/1.0 200 OK\r\n'
'Server: My Python Server\r\n'
'Content-Length: %d\r\n'
'Content-Type: text/html;charset=UTF-8\r\n\r\n'
'Connection: close\r\n'
) % len(output_data)
connection_socket.sendall(headers)
connection_socket.sendall(output_data)
Notice how you can use sendall to send all data from a string.