I am currently trying to do simple web stuff with the http.server module in Python.
When I try to POST a form to my script, it does not receive the POST data, $_POST ist empty and file_get_contents('php://input') as well.
This is my post_test.html:
#!/usr/bin/php
<!DOCTYPE html>
<html>
<body>
<form method="post" action="post_test.html">
Name: <input type="text" name="fname">
<input type="submit">
</form>
<?php
if ($_SERVER["REQUEST_METHOD"] == "POST") {
// collect value of input field
echo "RAW POST: " . file_get_contents('php://input') . "<br>";
$name = $_POST['fname'];
if (empty($name)) {
echo "Name is empty";
} else {
echo $name;
}
}
?>
</body>
</html>
And this is my server script:
import urllib.parse
from http.server import CGIHTTPRequestHandler, HTTPServer
hostName = "localhost"
serverPort = 8080
handler = CGIHTTPRequestHandler
handler.cgi_directories.append('/php-cgi')
class MyServer(handler):
def do_GET(self):
# get path without first '/' and without anything besides path and filename
file = self.path[1:].split("?")[0].split("#")[0]
# if the file is in the script list, execute from php-cgi, else load from webapp
php_handle = ["post_test.html"]
if file in php_handle:
self.path = "/php-cgi" + self.path
else:
self.path = "/webapp" + self.path
CGIHTTPRequestHandler.do_GET(self)
def do_POST(self):
# get path without first '/' and without anything besides path and filename
file = self.path[1:].split("?")[0].split("#")[0]
# if the file is in the script list, execute from php-cgi
php_handle = ["post_test.html"]
if file in php_handle:
length = int(self.headers['Content-Length'])
post_data = urllib.parse.parse_qs(self.rfile.read(length).decode('utf-8'))
for key, data in post_data.items():
self.log_message(key + ": " + str.join(", ", data))
self.path = "/php-cgi" + self.path
CGIHTTPRequestHandler.do_POST(self)
def do_HEAD(self):
CGIHTTPRequestHandler.do_HEAD(self)
if __name__ == "__main__":
webServer = HTTPServer((hostName, serverPort), MyServer)
try:
webServer.serve_forever()
except KeyboardInterrupt:
pass
webServer.server_close()
print("Server stopped.")
Okay, I was finally able to solve the problem: #!/usr/bin/php is wrong, it should be #!/usr/bin/php-cgi.
Reason: php does NOT use the POST data, and there is no way giving it to php.
php-cgi is made for the webserver purpose and can handle it.
Hwo to solve the next problem: To run php-cgi successfully you have to create a php.ini in the current directory tho, with two settings.
First one to allow executing it directly, second one to set the directory where the scripts are. If you don't set it, you will be greeted with a 404 not found.
php.ini:
cgi.force_redirect = 0
doc_root = /home/username/server-directory
Where the server-directory folder is the folder containing the php.ini, and the php-cgi folder. Also the name of the folder is not related to the php-cgi binary, it's just bad naming that I did.
If you try to recreate this, it should work perfectly now.
Related
I've been creating a simple invoicing web app using Python and Flask. One of its functionalities is that it automatically generates the PDF of the invoice once and sends it to the designated email address. Here is the route and function for that specific part: (Which is the only route and function I'm running into errors with)
#forms.route("/sendasemail/<int:id>", methods=['GET'])
#login_required
def sendasemail(id):
order = Order.query.get(id)
products = order.products
rendered = render_template("invoice.html", order = order, products = products)
save_location = url_for('static', filename=f'Order_{id}.pdf')
pdf = pdfkit.from_string(rendered, save_location)
msg = EmailMessage()
msg['Subject'] = f"Order Form No. {id}"
msg['From'] = 'DESIGNATED EMAIL'
if current_user.username == 'USERNAME':
msg['To'] = 'DESIGNATED_EMAIL'
else:
msg['To'] = 'DESIGNATED_EMAIL'
msg.set_content(f"Hi, \n\nKindly find Order Form No.{id} attached to this email.\n\nRegards,\nShibam S.P. Trading")
with open(f'Order_{id}.pdf', 'rb') as f:
file_data = f.read()
file_name = f.name
msg.add_attachment(file_data, maintype='application', subtype='octet-stream', filename=file_name)
with smtplib.SMTP_SSL('smtp.gmail.com', 465) as smtp:
smtp.login('DESIGNATED_EMAIL', 'DESIGNATED_PASSWORD')
smtp.send_message(msg)
os.unlink(f'Order_{id}.pdf')
flash("Email sent!", "success")
return render_template('show_data.html', order = order, products = products)
This snippet of code depends on the wkhtmltopdf and PDFKit libraries to function. When I run this code locally, it works perfectly fine. However, when I try running the same function within the application deployed on a production server, it throws an Internal Server Error. I think it might be because the PDF file, once created, cannot be found by the program due to some directory restructuring that I'm missing out on or plainly don't understand.
In the save_location = url_for('static', filename=f'Order_{id}.pdf') you create an absolute URL path, e.g. /static/Order_1.pdf instead of a local relative file path, like Order_1.pdf. But later in the script you use the local file path.
So the quick fix is just to use the local file path for save_location as well:
save_location = f'Order_{id}.pdf'
This assumes that the root directory of the Flask app is writable by the process. If not, you have to use a temporary directory, e.g. /tmp/ or an equivalent on your production server and update each file path in the script accordingly.
I have been facing some difficulty for the better part of today and I finally decided to come over to this fantastic community for help.
I am learning full-stack principles using Python. My issue is creating working with a HTTP server. The idea is to have an 'echo page', meaning, there is a HTML input field, and any text submitted through this field is echoed back.
The HTML input field is rendered by the server's do_GET, and an echo page is returned using the server's do_POST
Following a tutorial on the principles of HTTP, here is some code I wrote to execute a do_GET using Python's http.server module. (I must add that I am using Python 3.9, and I learned that I had to change the imported modules from this:
from BaseHTTPServer import BaseHTTPRequestHandler, HTTPServer
to this:
from http.server import BaseHTTPRequestHandler, HTTPServer
I also leaned that using version 3.X of Python, I needed to encode() the wfile.write() content. These are the tweaks that allowed the do_GET method to work.
Here is my full script:
import cgi
from http.server import BaseHTTPRequestHandler, HTTPServer
class serverHandler(BaseHTTPRequestHandler):
def do_GET(self):
if self.path.endswith('/hello'):
self.send_response(200)
self.send_header('Content-type', 'text/html')
self.end_headers()
output = ""
output += "<html><body> Hey There!"
output += "<form method='POST' enctype='multipart/form-data' action='/hello'>"
output += "<h2> What would you like me to say?</h2>"
output += "<input name = 'message' type = 'text' />"
output += "<input type = 'submit' value = 'Submit'/>"
output += "</form>"
output += "</body></html>"
# must encode for python 3+
self.wfile.write(output.encode())
print(output)
return
else:
self.send_error(404, 'File not found: {}'.format(self.path))
def do_POST(self):
try:
self.send_response(301)
self.send_header('Content-type', 'text/html')
self.end_headers()
message_content = None
param_dict = None
content_type, param_dict = cgi.parse_header(
self.headers.getheader('content-type'))
if content_type == 'multipart/form-data':
fields = cgi.parse_multipart(self.rfile, param_dict)
message_content = fields.get('message')
output = ''
output += '<html><body>'
output += '<h2> You Said: </h2>'
output += '<h1> %s </h1>' % message_content[0]
output += "<form method='POST' enctype='multipart/form-data' action='/hello'>"
output += "<h2> What would you like me to say?</h2>"
output += "<input name = 'message' type = 'text' />"
output += "<input type = 'submit' value = 'Submit'/>"
output += "</form>"
output += '</body></html>'
self.wfile.write(output.encode())
print(output)
except:
pass
def main():
server = None
try:
port = 8080
server = HTTPServer(('', port), serverHandler)
print('Server running on port {}'.format(port))
server.serve_forever()
except KeyboardInterrupt:
print('Server shutting down...')
server.socket.close()
main()
As expected, the server runs on port 8080 as specified in the main function. In the serverHandler class, I specified a \hello path, on which the page with the HTML input field is rendered.
The problem comes in when I type in text to the input field and click on the submit button.
Ideally, the page returned from do_POST should have a HTML h2 element that displays the text that was keyed in on submit, and below it, a blank input field should be shown to allow for new text to be entered and echoed.
This however, as I have mentioned, does not happen, and I instead see a blank page on my browser upon clicking the submit button.
Here is the terminal output when the script is run:
Anthony ~\..\digitization\back-end git: Development ≣ +1 ~1 -0 ! ❯❯❯ python .\webserver.py
Server running on port 8080
127.0.0.1 - - [28/Dec/2020 21:12:36] "GET /hello HTTP/1.1" 200 -
<html><body> Hey There!<form method='POST' enctype='multipart/form-data' action='/hello'><h2> What would you like me to say?</h2><input name = 'message' type = 'text' /><input type = 'submit' value = 'Submit'/></form></body></html>
127.0.0.1 - - [28/Dec/2020 21:12:42] "POST /hello HTTP/1.1" 301 -
There seems to be something off with my do_POST method.
As mentioned, I am in the process of learning, and the original script was written in Python 2.X linked here.
I will appreciate insights on what is happening, and a solution to get round it.
Thank you in advance
:)
I dug a little deeper into the community and found a workaround that proved to solve my particular issue.
Kindly note that this answer worked in my scenario, however if it doesn't work for you, you could follow the thread and try out the various suggestions outlined.
:)
I am trying to use ng-file-upload in my Flask app, but I'm having a hard time send it to Flask main.py to be saved. I could have just done a simple submit using flask's url_for(), but I would like to keep Flask in the backend and AngularJS in the front-end.
Also, I would like to use the drag and drop field feature from this angular plugin instead of an upload file button, but I think that's where my issue is coming from. I don't think flask is recognizing it from request.files. I'm not sure if I'm calling correctly or maybe is just a simple fix and I'm not seeing it.
Flask main.py
#app.route("/", methods = ['GET', 'POST'])
def index():
if request.method == 'POST':
print 'unknown went through'
filestorage = request.files['user_file']
print filestorage
if not filestorage:
return jsonify(fileStatus = False)
if filestorage.filename.endswith('.xlsx'):
print "you uploaded a excel file"
file_new_name = 'dataexcel.xlsx'
file_type = 'excel'
elif filestorage.filename.endswith('.csv'):
print "you uploaded a csv file"
file_new_name = 'datacsv.csv'
file_type = 'csv'
path = "static/uploads/" + file_new_name
if os.path.exists(path):
print 'i exist'
os.remove(path)
filename = docs.save(filestorage, None, file_new_name)
else:
print "i don't exist"
filename = docs.save(filestorage, None, file_new_name)
session['path'] = path
session['file_type'] = file_type
return jsonify(fileStatus = True)
I know the "POST" works because i can see the message from this print statement, print 'unknown went through', after that it fails to recognize the filestorage variable
Here is my angular controller:
myApp.controller('UploadFileCtrl',[
'$scope',
'$http',
'Upload',
function($scope, $http, Upload){
// upload later on form submit or something similar
$scope.submit = function(files) {
if ($scope.files) {
$scope.upload($scope.files);
}
}
// upload on file select or drop
$scope.upload = function (files) {
console.log('it was uploaded');
Upload.upload({
url: '/',
method: 'POST',
data: {file: files}
}).then(function (resp) {
// console.log('Success ' + resp.config.data.file.name + 'uploaded. Response: ' + resp.data);
console.log(resp.data);
}, function (resp) {
console.log('Error status: ' + resp.status);
console.log(resp);
}, function (evt) {
var progressPercentage = parseInt(100.0 * evt.loaded / evt.total);
console.log('progress: ' + progressPercentage + '% ' + evt.config.data.file.name);
});
}
}])
html form:
<form ng-click="submit(files)">
<div ngf-drop ngf-select ng-model="files" class="drop-box"
ngf-drag-over-class="'dragover'" ngf-multiple="true" ngf-allow-dir="true"
name="user_file" type="file">Drop pdfs or images here or click to upload</div>
<div ngf-no-file-drop>File Drag/Drop is not supported for this browser</div>
<button type="submit">submit</button>
I'm still not familiar with Flask, but it's a cool webframework to work with and it will be awesome to integrate it with AngularJS. Your help will be appreciated, thanks in advance!
UPDATE
I actually got it to upload in Flask, I need it to replace filestorage = request.files['user_file'] with filestorage = request.files['file']. Like I said, Flask was not recognizing it by it's provided name in the html, which I don't know why. I failed to provide this line return render_template('index.html') in the main.py above. Since Flask is not longer present in the html, could that be the reason Flask is not recognizing the name in the form?
revised main.py
#app.route("/", methods = ['GET', 'POST'])
def index():
if request.method == 'POST':
print 'unknown went through'
filestorage = request.files['file']
print filestorage
if not filestorage:
return jsonify(fileStatus = False)
if filestorage.filename.endswith('.xlsx'):
print "you uploaded a excel file"
file_new_name = 'dataexcel.xlsx'
file_type = 'excel'
elif filestorage.filename.endswith('.csv'):
print "you uploaded a csv file"
file_new_name = 'datacsv.csv'
file_type = 'csv'
path = "static/uploads/" + file_new_name
if os.path.exists(path):
print 'i exist'
os.remove(path)
filename = docs.save(filestorage, None, file_new_name)
else:
print "i don't exist"
filename = docs.save(filestorage, None, file_new_name)
session['path'] = path
session['file_type'] = file_type
return jsonify(fileStatus = True)
return render_template('index.html')
Also I had to change in the html the ngf-multiple to false, this was sending an array, so Flask was expecting a dict.
revised html:
<form ng-click="submit(files)">
<div ngf-drop ngf-select ng-model="files" class="drop-box"
ngf-drag-over-class="'dragover'" ngf-multiple="true" ngf-allow-dir="true" name="user_file" type="file">Drop pdfs or images here or click to upload</div>
<div ngf-no-file-drop>File Drag/Drop is not supported for this browser</div>
<button type="submit">submit</button>
I'm doing pages in html and python (I'm novice in python), I would like to have IP client address, but I don't know if it is possible. I saw it is possible with PHP language.
So, I execute my code in command line (with Linux) like that:
./code.py client_server app_name app_version
infos.py
def main( client_server, app_name, app_version):
template = open('infoHTML.py').read()
c = string.Template(template).substitute(
app_name = app_name,
app_version = app_version,
os = user,
user = login)
f = tempfile.NamedTemporaryFile(prefix='/tmp/info.html', mode='w', delete=False)
f.write(contenu)
f.close()
webbrowser.open(f.name)
if __name__ == "__main__":
client_server = sys.argv[1]
app_name = sys.argv[2]
app_version = sys.argv[3]
user = sys.platform
sys.argv.append(user)
login = getpass.getuser()
sys.argv.append(login)
main(client_server, app_name, app_version)
I have an html code into python code here: infoHTML.py
<html>
App: ${app_name}<br/><br/>
Version: ${app_version}<br/><br/>
User: ${user}<br/><br/>
<form name="sendData" method="get" action="http://localhost:8000/cgi/display.py">
Project: <input type="text" name="pro"><br/><br/>
Number: <input type="text" name="num"/><br/><br/>
<input type="submit" value="OK"/>
</form>
</body>
</html>
It's possible. You need to do it either by rendering the address on the response body or by requesting it with ajax after the response has already been rendered.
It would be hard to give you a code solution without seeing what web server you are using, but here are a couple of pointers for the first approach. To obtain the address, on the server side (python handler):
import socket
ip = socket.gethostbyname(socket.gethostname())
or if you are using something like Google App Engine:
ip = self.request.remote_addr
you should then write the IP to the response. For example, if you are using a templating engine to render your HTML, your HTML can look like something similar to this:
<html>
<script>
var ip = {{ip}}
</script>
and on the python code that renders the template you should do something like that:
htmlContent = template.render(ip=ip)
self.response.write(htmlContent)
I have lately been improving security on my webserver, which I wrote myself using http.server and BaseHTTPRequestHandler. I have blocked (403'd) most essential server files, which I do not want users to be able to access. Files include the python server script and all databases, plus some HTML templates.
However, in this post on stackoverflow I read that using open(curdir + sep + self.path) in a do_GET request might potentially make every file on your computer readable.
Can someone explain this to me? If the self.path is ip:port/index.html every time, how can someone access files that are above the root / directory?
I understand that the user (obviously) can change the index.html to anything else, but I don't see how they can access directories above root.
Also if you're wondering why I'm not using nginx or apache, I wanted to create my own web server and website for learning purposes. I have no intention to run an actual website myself, and if I do want to, I will probably rent a server or use existing server software.
class Handler(http.server.BaseHTTPRequestHandler):
def do_GET(self):
try:
if "SOME BLOCKED FILE OR DIRECTORY" in self.path:
self.send_error(403, "FORBIDDEN")
return
#I have about 6 more of these 403 parts, but I left them out for readability
if self.path.endswith(".html"):
if self.path.endswith("index.html"):
#template is the Template Engine that I created to create dynamic HTML content
parser = template.TemplateEngine()
content = parser.get_content("index", False, "None", False)
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write(content.encode("utf-8"))
return
elif self.path.endswith("auth.html"):
parser = template.TemplateEngine()
content = parser.get_content("auth", False, "None", False)
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write(content.encode("utf-8"))
return
elif self.path.endswith("about.html"):
parser = template.TemplateEngine()
content = parser.get_content("about", False, "None", False)
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write(content.encode("utf-8"))
return
else:
try:
f = open(curdir + sep + self.path, "rb")
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write((f.read()))
f.close()
return
except IOError as e:
self.send_response(404)
self.send_header("Content-type", "text/html")
self.end_headers()
return
else:
if self.path.endswith(".css"):
h1 = "Content-type"
h2 = "text/css"
elif self.path.endswith(".gif"):
h1 = "Content-type"
h2 = "gif"
elif self.path.endswith(".jpg"):
h1 = "Content-type"
h2 = "jpg"
elif self.path.endswith(".png"):
h1 = "Content-type"
h2 = "png"
elif self.path.endswith(".ico"):
h1 = "Content-type"
h2 = "ico"
elif self.path.endswith(".py"):
h1 = "Content-type"
h2 = "text/py"
elif self.path.endswith(".js"):
h1 = "Content-type"
h2 = "application/javascript"
else:
h1 = "Content-type"
h2 = "text"
f = open(curdir+ sep + self.path, "rb")
self.send_response(200)
self.send_header(h1, h2)
self.end_headers()
self.wfile.write(f.read())
f.close()
return
except IOError:
if "html_form_action.asp" in self.path:
pass
else:
self.send_error(404, "File not found: %s" % self.path)
except Exception as e:
self.send_error(500)
print("Unknown exception in do_GET: %s" % e)
You're making an invalid assumption:
If the self.path is ip:port/index.html every time, how can someone access files that are above the root / directory?
But self.path is never ip:port/index.html. Try logging it and see what you get.
For example, if I request http://example.com:8080/foo/bar/index.html, the self.path is not example.com:8080/foo/bar/index.html, but just /foo/bar/index.html. In fact, your code couldn't possibly work otherwise, because curdir+ sep + self.path would give you a path starting with ./example.com:8080/, which won't exist.
And then ask yourself what happens if it's /../../../../../../../etc/passwd.
This is one of many reasons to use os.path instead of string manipulation for paths. For examples, instead of this:
f = open(curdir + sep + self.path, "rb")
Do this:
path = os.path.abspath(os.path.join(curdir, self.path))
if os.path.commonprefix((path, curdir)) != curdir:
# illegal!
I'm assuming that curdir here is an absolute path, not just from os import curdir or some other thing that's more likely to give you . than anything else. If it's the latter, make sure to abspath it as well.
This can catch other ways of escaping the jail as well as passing in .. strings… but it's not going to catch everything. For example, if there's a symlink pointing out of the jail, there's no way abspath can tell that someone's gone through the symlink.
self.path contains the request path. If I were to send a GET request and ask for the resource located at /../../../../../../../etc/passwd, I would break out of your application's current folder and be able to access any file on your filesystem (that you have permission to read).