I am using CGIHTTPServer.py for creating simple CGI server. I want my CGI script to take care of response code if some operation goes wrong . How can I do that?
Code snippet from my CGI script.
if authmxn.authenticate():
stats = Stats()
print "Content-Type: application/json"
print 'Status: 200 OK'
print
print json.dumps(stats.getStats())
else:
print 'Content-Type: application/json'
print 'Status: 403 Forbidden'
print
print json.dumps({'msg': 'request is not authenticated'})
Some of the snippet from request handler,
def run_cgi(self):
'''
rest of code
'''
if not os.path.exists(scriptfile):
self.send_error(404, "No such CGI script (%s)" % `scriptname`)
return
if not os.path.isfile(scriptfile):
self.send_error(403, "CGI script is not a plain file (%s)" %
`scriptname`)
return
ispy = self.is_python(scriptname)
if not ispy:
if not (self.have_fork or self.have_popen2):
self.send_error(403, "CGI script is not a Python script (%s)" %
`scriptname`)
return
if not self.is_executable(scriptfile):
self.send_error(403, "CGI script is not executable (%s)" %
`scriptname`)
return
if not self.have_fork:
# Since we're setting the env in the parent, provide empty
# values to override previously set values
for k in ('QUERY_STRING', 'REMOTE_HOST', 'CONTENT_LENGTH',
'HTTP_USER_AGENT', 'HTTP_COOKIE'):
env.setdefault(k, "")
self.send_response(200, "Script output follows") # overrides the headers
decoded_query = query.replace('+', ' ')
It is possible to implement support for a Status: code message header that overrides the HTTP status line (first line of HTTP response, e.g. HTTP/1.0 200 OK). This requires:
sub-classing CGIHTTPRequestHandler in order to trick it into writing the CGI script's output into a StringIO object instead of directly to a socket.
Then, once the CGI script is complete, update the HTTP status line with the value provided in the Status: header.
It's a hack, but it's not too bad and no standard library code needs to be patched.
import BaseHTTPServer
import SimpleHTTPServer
from CGIHTTPServer import CGIHTTPRequestHandler
from cStringIO import StringIO
class BufferedCGIHTTPRequestHandler(CGIHTTPRequestHandler):
def setup(self):
"""
Arrange for CGI response to be buffered in a StringIO rather than
sent directly to the client.
"""
CGIHTTPRequestHandler.setup(self)
self.original_wfile = self.wfile
self.wfile = StringIO()
# prevent use of os.dup(self.wfile...) forces use of subprocess instead
self.have_fork = False
def run_cgi(self):
"""
Post-process CGI script response before sending to client.
Override HTTP status line with value of "Status:" header, if set.
"""
CGIHTTPRequestHandler.run_cgi(self)
self.wfile.seek(0)
headers = []
for line in self.wfile:
headers.append(line) # includes new line character
if line.strip() == '': # blank line signals end of headers
body = self.wfile.read()
break
elif line.startswith('Status:'):
# Use status header to override premature HTTP status line.
# Header format is: "Status: code message"
status = line.split(':')[1].strip()
headers[0] = '%s %s' % (self.protocol_version, status)
self.original_wfile.write(''.join(headers))
self.original_wfile.write(body)
def test(HandlerClass = BufferedCGIHTTPRequestHandler,
ServerClass = BaseHTTPServer.HTTPServer):
SimpleHTTPServer.test(HandlerClass, ServerClass)
if __name__ == '__main__':
test()
Needless to say, this is probably not the best way to go and you should look at a non-CGIHTTPServer solution, e.g. a micro-framework such as bottle, a proper web-server (from memory, CGIHTTPServer is not recommended for production use), fastcgi, or WSGI - just to name a few options.
With the standard library HTTP server you cannot do this. From the library documentation:
Note CGI scripts run by the CGIHTTPRequestHandler class cannot execute redirects (HTTP code 302), because code 200 (script output follows) is sent prior to execution of the CGI script. This pre-empts the status code.
This means that the server does not support the Status: <status-code> <reason> header from the script. You correctly identified the portion in the code that shows this support does not exist: The server sends status code 200 before even running the script. There is no way you can change this from within the script.
There are several tickets related to this in the Python bugtracker, some with patches, see e.g., issue13893. So one option for you would be to patch the standard library to add this feature.
However, I would strongly suggest you switch to another technology instead of CGI (or run a real web server).
Related
When running bash commands from Python to a webpage, succesful output to the webpage occurs when no output options are set. However, i'd like to save the output to an xml file and display the output to the webpage. I get an error when attempting to do this:
Code:
#!/usr/bin/python
import subprocess
import os
# Import modules for CGI handling
import cgi, cgitb; cgitb.enable()
import time
# Create instance of FieldStorage
form = cgi.FieldStorage()
# Get data from fields
niktoValue = form.getvalue('niktoInput')
# Get data from fields
print "Content-type:text/html\r\n\r\n"
print "<html>"
print "<head>"
print "</head>"
print "<body>"
print "<h2>Nikto Scan Information for %s </h2>" % (niktoValue)
def bash(command):
return subprocess.check_output(['bash', '-c', command])
def niktoScan():
res = bash("nikto -h %s -output xml" % niktoValue).splitlines()
print(res)
niktoScan()
print "</body>"
print "</html>"
Error 13 is normally "Permission denied".
I suspect your CGI script is running as the _www user rather than as your regular user when you are logged in at your desktop. You can test by running id in the bash script under the http server or looking in your http server config file - maybe /etc/apache2/httpd.conf for the part that looks like this:
User/Group: The name (or #number) of the user/group to run httpd as.
It is usually good practice to create a dedicated user and group for
running httpd, as with most system services.
User _www
Group _www
I am unfamiliar with the nikto program, so you will need to find out from its author how to allow the www user to run it - maybe by adding that user to a Unix group, maybe via sudo, maybe something else.
I have just made a server (only for localhost) in Python to execute by CGI to execute and try my Python scripts. This is the code of the file that executes the server:
#!/usr/bin/env python
#-*- coding:utf-8 -*-
import BaseHTTPServer
import CGIHTTPServer
import cgitb
cgitb.enable() ## This line enables CGI error reporting
server = BaseHTTPServer.HTTPServer
handler = CGIHTTPServer.CGIHTTPRequestHandler
server_address = ("", 8000)
handler.cgi_directories = ["/"]
httpd = server(server_address, handler)
httpd.serve_forever()
When I access some script in the server (http://127.0.0.1:8000/index.py) there is no problem, but when I access the server (http://127.0.0.1:8000/) it shows:
Error response
Error code 403.
Message: CGI script is not executable ('//').
Error code explanation: 403 = Request forbidden -- authorization will not help.
It's like if index files aren't set as default file to access when access a folder instead of a specific file...
I would like to be able to access to http://127.0.0.1/index.py when I access http://127.0.0.1/. Thanks for everything.
Python's built-in HTTP server is extremely basic, so it doesn't include such feature. However you can implement it yourself by subclassing CGIHTTPRequestHandler, probably replacing the is_cgi function.
If you use handler.cgi_directories = ["/cgi"], you can place an index.html file in "/". And of course, if you want a cgi script index.py as default, you can use the index.html for forwarding...
I did try to modify is_cgi function and it's working!
def is_cgi(self):
collapsed_path = _url_collapse_path(self.path)
if collapsed_path == '//':
self.path = '/index.py'
collapsed_path = _url_collapse_path(self.path)
dir_sep = collapsed_path.find('/', 1)
head, tail = collapsed_path[:dir_sep], collapsed_path[dir_sep + 1:]
if head in self.cgi_directories:
self.cgi_info = head, tail
return True
return False
I put this method into this following class:
class CGIHandlerOverloadIndex(CGIHTTPServer.CGIHTTPRequestHandler):
I'm trying to run python on the web to do some CSS/JSS extraction from websites. I'm using mod_wsgi as my interface for python. I've been following this website to get an idea on getting started.
The below is their sample code.
#! /usr/bin/env python
# Our tutorial's WSGI server
from wsgiref.simple_server import make_server
def application(environ, start_response):
# Sorting and stringifying the environment key, value pairs
response_body = ['%s: %s' % (key, value)
for key, value in sorted(environ.items())]
response_body = '\n'.join(response_body)
status = '200 OK'
response_headers = [('Content-Type', 'text/plain'),
('Content-Length', str(len(response_body)))]
start_response(status, response_headers)
return [response_body]
# Instantiate the WSGI server.
# It will receive the request, pass it to the application
# and send the application's response to the client
httpd = make_server(
'localhost', # The host name.
8051, # A port number where to wait for the request.
application # Our application object name, in this case a function.
)
# Wait for a single request, serve it and quit.
httpd.handle_request()
While this runs fine with python 2.7, I can't get it to run on Python 3. For my CSS/JSS extraction, I have modified the above code and put in my own functionalities which use BeautifulSoup and urllib3. While for using those modules I need python 3, for the WSGI code I need python 2.7. Hence, I can't merge the two. When trying to run BS and urllib in python3, I get an error. But when I try to run the WSGI code with python3, I'm just unable to load the webpage.
Any help would be greatly appreciated! Any workarounds, or suggestions as well.
I have a Python script that I'd like to be run from the browser, it seem mod_wsgi is the way to go but the method feels too heavy-weight and would require modifications to the script for the output. I guess I'd like a php approach ideally. The scripts doesn't take any input and will only be accessible on an internal network.
I'm running apache on Linux with mod_wsgi already set up, what are the options here?
I would go the micro-framework approach just in case your requirements change - and you never know, it may end up being an app rather just a basic dump... Perhaps the simplest (and old fashioned way!?) is using CGI:
Duplicate your script and include print 'Content-Type: text/plain\n' before any other output to sys.stdout
Put that script somewhere apache2 can access it (your cgi-bin for instance)
Make sure the script is executable
Make sure .py is added to the Apache CGI handler
But - I don't see anyway this is going to be a fantastic advantage (in the long run at least)
You could use any of python's micro frameworks to quickly run your script from a server. Most include their own lightweight servers.
From cherrypy home page documentation
import cherrypy
class HelloWorld(object):
def index(self):
# run your script here
return "Hello World!"
index.exposed = True
cherrypy.quickstart(HelloWorld())
ADditionally python provides the tools necessary to do what you want in its standard library
using HttpServer
A basic server using BaseHttpServer:
import time
import BaseHTTPServer
HOST_NAME = 'example.net' # !!!REMEMBER TO CHANGE THIS!!!
PORT_NUMBER = 80 # Maybe set this to 9000.
class MyHandler(BaseHTTPServer.BaseHTTPRequestHandler):
def do_HEAD(s):
s.send_response(200)
s.send_header("Content-type", "text/html")
s.end_headers()
def do_GET(s):
"""Respond to a GET request."""
s.send_response(200)
s.send_header("Content-type", "text/html")
s.end_headers()
s.wfile.write("<html><head><title>Title goes here.</title></head>")
s.wfile.write("<body><p>This is a test.</p>")
# If someone went to "http://something.somewhere.net/foo/bar/",
# then s.path equals "/foo/bar/".
s.wfile.write("<p>You accessed path: %s</p>" % s.path)
s.wfile.write("</body></html>")
if __name__ == '__main__':
server_class = BaseHTTPServer.HTTPServer
httpd = server_class((HOST_NAME, PORT_NUMBER), MyHandler)
print time.asctime(), "Server Starts - %s:%s" % (HOST_NAME, PORT_NUMBER)
try:
httpd.serve_forever()
except KeyboardInterrupt:
pass
httpd.server_close()
print time.asctime(), "Server Stops - %s:%s" % (HOST_NAME, PORT_NUMBER)
What's nice about the microframeworks is they abstract out writing headers and such (but should still provide you an interface to, if required)
I'm new to Python, so forgive me if I am missing something obvious.
I am using urllib.FancyURLopener to retrieve a web document. It works fine when authentication is disabled on the web server, but fails when authentication is enabled.
My guess is that I need to subclass urllib.FancyURLopener to override the get_user_passwd() and/or prompt_user_passwd() methods. So I did:
class my_opener (urllib.FancyURLopener):
# Redefine
def get_user_passwd(self, host, realm, clear_cache=0):
print "get_user_passwd() called; host %s, realm %s" % (host, realm)
return ('name', 'password')
Then I attempt to open the page:
try:
opener = my_opener()
f = opener.open ('http://1.2.3.4/whatever.html')
content = f.read()
print "Got it: ", content
except IOError:
print "Failed!"
I expect FancyURLopener to handle the 401, call my get_user_passwd(), and retry the request.
It does not; I get the IOError exception when I call "f = opener.open()".
Wireshark tells me that the request is sent, and that the server is sending a "401 Unauthorized" response with two headers of interest:
WWW-Authenticate: BASIC
Connection: close
The connection is then closed, I catch my exception, and it's all over.
It fails the same way even if I retry the "f = opener.open()" after IOError.
I have verified that my my_opener() class is working by overriding the http_error_401() method with a simple "print 'Got 401 error'". I have also tried to override the prompt_user_passwd() method, but that doesn't happen either.
I see no way to proactively specify the user name and password.
So how do I get urllib to retry the request?
Thanks.
I just tried your code on my webserver (nginx) and it works as expected:
Get from urllib client
HTTP/1.1 401 Unauthorized from server with Headers
Connection: close
WWW-Authenticate: Basic realm="Restricted"
client tries again with Authorization header
Authorization: Basic <Base64encoded credentials>
Server responds with 200 OK + Content
So I guess your code is right (I tried it with python 2.7.1) and maybe the webserver you are trying to access is not working as expected. Here is the code tested using the free http basic auth testsite browserspy.dk (seems they are using apache - the code works as expected):
import urllib
class my_opener (urllib.FancyURLopener):
# Redefine
def get_user_passwd(self, host, realm, clear_cache=0):
print "get_user_passwd() called; host %s, realm %s" % (host, realm)
return ('test', 'test')
try:
opener = my_opener()
f = opener.open ('http://browserspy.dk/password-ok.php')
content = f.read()
print "Got it: ", content
except IOError:
print "Failed!"