I've got a Flask app that I'm trying to deploy using Gunicorn and nginx. However, although it works fine locally, it throws a TemplateNotFound error when I run in with Gunicorn on my remote server.
I'm not sure how to even start debugging this, let alone why it's failing...would love help on the former, if not the latter. I thought maybe it was a permissions issue, so chmod'd the templates folder to 777...no luck. Here's all the relavant details:
install script
Starting with a bare Ubuntu 10.04 install, I run this to set up the server and pull in my code: https://github.com/total-impact/total-impact-deploy/blob/master/deploy.sh. Then I put this nginx config file at /etc/nginx/sites-available/total-impact:
server {
location / {
proxy_pass http://127.0.0.1:8000;
}
}
Finally, I navigate the app directory and run gunicorn web:app, and hit the server's IP address. This generates a 500 in the browser, and this output on the command line:
stack trace:
root#jc:/home/ti/total-impact-webapp/totalimpactwebapp# gunicorn web:app2012-05-28 23:15:06 [15313] [INFO] Starting gunicorn 0.14.3
2012-05-28 23:15:06 [15313] [INFO] Listening at: http://127.0.0.1:8000 (15313)
2012-05-28 23:15:06 [15313] [INFO] Using worker: sync
2012-05-28 23:15:06 [15316] [INFO] Booting worker with pid: 15316
2012-05-28 23:15:12,274 - totalimpactwebapp.core - ERROR - Exception on / [GET]
Traceback (most recent call last):
File "/usr/local/lib/python2.6/dist-packages/flask/app.py", line 1292, in wsgi_app
response = self.full_dispatch_request()
File "/usr/local/lib/python2.6/dist-packages/flask/app.py", line 1062, in full_dispatch_request
rv = self.handle_user_exception(e)
File "/usr/local/lib/python2.6/dist-packages/flask/app.py", line 1060, in full_dispatch_request
rv = self.dispatch_request()
File "/usr/local/lib/python2.6/dist-packages/flask/app.py", line 1047, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "/home/ti/total-impact-webapp/totalimpactwebapp/web.py", line 60, in home
return render_template('index.html', commits=False)
File "/usr/local/lib/python2.6/dist-packages/flask/templating.py", line 120, in render_template
return _render(ctx.app.jinja_env.get_template(template_name),
File "/usr/local/lib/python2.6/dist-packages/jinja2/environment.py", line 719, in get_template
return self._load_template(name, self.make_globals(globals))
File "/usr/local/lib/python2.6/dist-packages/jinja2/environment.py", line 693, in _load_template
template = self.loader.load(self, name, globals)
File "/usr/local/lib/python2.6/dist-packages/jinja2/loaders.py", line 115, in load source, filename, uptodate = self.get_source(environment, name)
File "/usr/local/lib/python2.6/dist-packages/flask/templating.py", line 61, in get_source
raise TemplateNotFound(template)
TemplateNotFound: index.html
Are your templates in [app root]/templates/?
If so, check to be sure your path is correct. Put this as the first line in the view that handles your homepage:
return app.root_path
If that's what you expect to see - or if you're using Blueprints or another method that changes the default Jinja Environment somehow - it's a little more complicated.
Oddly, Jinja doesn't seem to have a jinja2.Environment.FileSystemLoader.get_search_path() method. I assumed it would have one :(
Today, I experienced identical problems after a long period of my Flask app behaving quite normally (ie not throwing TemplateNotFound exceptions). None of the approaches mentioned by others here hit the mark or seemed appropriate (eg app.debug, path manipulation).
Instead, I tracked it down to the standard Flask app initialisation line:
app = Flask(__name__)
I had changed __name__ to another value (to get access to a named logger), not expecting for all this carnage to unfold :-) Don't change this value unless you are very familiar with Flask internals.
I have just spent 2 hours in a very similar situation and thought I'd post what ended up being the solution.
I was suddenly getting TemplateNotFound errors in the Apache logs from my Flask application, in production, which had been running fine until then. This resulted in 500 errors across the site.
First issue was that the TemplateNotFound errors did not show unless I had Flask's "debug" flag on -- there was no sign of any problems at all in the Apache log despite LogLevel set to info.
Running the app "locally" (Flask listens on localhost:5000) was fine (one can test the pages via wget 127.0.0.0:5000). It turned out that a copy of the main web app python code had somehow landed in the directory above where it should have been. This was imported by wsgi first and, as a result, the relative path to templates was incorrect.
Related
We're using a Flask app, served by gunicorn + Nginx in a docker container. Roughly once or twice every 24 ours, our logger displays the following message:
Dec 31 00:27:36 Computer logger: [12/31/2019 08:27:36] backend ERROR: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again. Traceback (most recent call last): File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1949, in full_dispatch_request rv = self.dispatch_request() File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1925, in dispatch_request self.raise_routing_exception(req) File "/usr/local/lib/python3.6/site-packages/flask/app.py", line 1907, in raise_routing_exception raise request.routing_exception File "/usr/local/lib/python3.6/site-packages/flask/ctx.py", line 350, in match_request result = self.url_adapter.match(return_rule=True) File "/usr/local/lib/python3.6/site-packages/werkzeug/routing.py", line 1799, in match import pdb werkzeug.exceptions.NotFound: 404 Not Found: The requested URL was not found on the server. If you entered the URL manually please check your spelling and try again.
What this affects or what it's caused by is totally mysterious, and the traceback is not helpful in the slightest. There is no noticeable change on the user-facing side. I've tried many things, including starting the app. in debug mode. My most recent attempt was to go into /usr/local/lib/python3.6/site-packages/werkzeug/routing.py itself and change the behavior of the error. What happens, though, is that above-logged message just prints out my code changes—but doesn't seem to actually run the changes themselves. (i.e. - Everything looks to run exactly the same way, regardless of how I change that file. Instead, the changes themselves are logged [not the effects of running that changed code]).
How does one debug an unresponsive Werkzeug 404 error like this? What options/settings/methods am I missing? Why would changes to the code of routing.py (for example) be ignored, but printed out to the screen (as if what Python is doing is just printing out code blocks between line x and line y).
I'm using flask as a python framework for my object detection REST API and it works just fine on MacOS. The application takes the uploaded image and it works on that. I'm sending a POST request to upload the image to the server. Just to give you an insight into my code:
#app.route('/post', methods=['GET', 'POST'])
def post():
form = PhotoForm(CombinedMultiDict((request.files, request.form)))
if request.method == 'POST' and form.validate():
with tempfile.NamedTemporaryFile() as temp:
form.input_photo.data.save(temp)
temp.flush()
print(temp.name)
result = detect_objects(temp.name)
photo_form = PhotoForm(request.form)
return render_template('upload.html',
photo_form=photo_form, result=result)
else:
return redirect(url_for('upload'))
However, when I try to run the same application on Windows 10, I get the following server error 500:
127.0.0.1 - - [08/Feb/2018 15:32:24] "GET / HTTP/1.1" 200 -
C:\Users\KUGUOG~1.KAA\AppData\Local\Temp\tmpw6y5g2f9
[2018-02-08 15:32:37,369] ERROR in app: Exception on /post [POST]
Traceback (most recent call last):
File "c:\python36\lib\site-packages\flask\app.py", line 1982, in wsgi_app
response = self.full_dispatch_request()
File "c:\python36\lib\site-packages\flask\app.py", line 1614, in full_dispatch_request
rv = self.handle_user_exception(e)
File "c:\python36\lib\site-packages\flask\app.py", line 1517, in handle_user_exception
reraise(exc_type, exc_value, tb)
File "c:\python36\lib\site-packages\flask\_compat.py", line 33, in reraise
raise value
File "c:\python36\lib\site-packages\flask\app.py", line 1612, in full_dispatch_request
rv = self.dispatch_request()
File "c:\python36\lib\site-packages\flask\app.py", line 1598, in dispatch_request
return self.view_functions[rule.endpoint](**req.view_args)
File "C:\Users\kuguoglu.berk.kaan\Desktop\ObjectDetectionRestApi\app.py", line 188, in post
result = detect_objects(temp.name)
File "C:\Users\kuguoglu.berk.kaan\Desktop\ObjectDetectionRestApi\app.py", line 149, in detect_objects
image = Image.open(image_path).convert('RGB')
File "c:\python36\lib\site-packages\PIL\Image.py", line 2543, in open
fp = builtins.open(filename, "rb")
PermissionError: [Errno 13] Permission denied: 'C:\\Users\\KUGUOG~1.KAA\\AppData\\Local\\Temp\\tmpw6y5g2f9'
127.0.0.1 - - [08/Feb/2018 15:32:37] "POST /post HTTP/1.1" 500 -
I skimmed through quite a few posts and forums and I thought this might be related to file permission in Windows OS. As it explicitly says permission error and all the related posts I've read so far mentions file permissions I do not think that it's related to the program itself. If it was the case, it wouldn't work smoothly on MacOS in the first place.
So far, I have tried listening to different ports, including 80 and 8080, and running the program as an administrator through PyCharm terminal and command prompt. However, these did not solve the issue.
I'm using Python 3.6.3, Flask 0.12.2, in case it helps you figure out the issue. Feel free to ask me for more details, if needed. I'd be happy to provide if I can.
Cheers.
Edit: I tried some of the suggested ways in the following two posts, however, all of them failed.
How to run python script with elevated privileges on Windows
Request UAC elevation from within python scripts - 1
Request UAC elevation from within python scripts - 2
PermissionError: [Errno 13] Permission denied
Further, I disabled read-only option for the directory below to see if it solves the problem but it didn't.
C:\Users\KUGUOG~1.KAA\AppData\
Are you running with pycharm?
If you are running with cmd, run the command first as administrator.
When you find the cmd.exe, right click it and click on run as administrator
Elif you are using pycharm, run it as administrator as well.
I'm pretty inexperienced with gunicorn. I have it installed within a virtual env and am trying to serve a pyramid app with the following:
env/bin/gunicorn --pid /home/staging/gunicorn.pid --bind 0.0.0.0:8000 pyzendoc:main
However everytime a request is sent I get the following trace from gunicorn
2013-10-30 14:16:20 [1284] [ERROR] Error handling request
Traceback (most recent call last):
File "/home/staging/api/env/local/lib/python2.7/site-packages/gunicorn/workers/sync.py", line 126, in handle_request
respiter = self.wsgi(environ, resp.start_response)
TypeError: main() takes exactly 1 argument (2 given)
I'm guessing that main in the gunicorn refers to the main method in pyramids init but that method takes (global_config, **settings) as args so I think that maybe gunicorn is somehow looking at the wrong method. Has anyone seen anything similar before?
Thanks
C
The invocation pyzendoc:main is expecting to find a callable that accepts an (environ, start_response) signature, as a WSGI app, which you don't have until main(global_conf, **settings) returns one. A better option is to use gunicorn_paster, as shown here.
I'm using Django 1.5.1 in virtualenv, Python 2.7 on OS X 10.8.4. I changed from DEBUG = True to DEBUG = False in my Django settings and I got an error:
Traceback (most recent call last):
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/wsgiref/handlers.py", line 85, in run
self.result = application(self.environ, self.start_response)
File "/Users/kilroy/.virtualenvs/project_name/lib/python2.7/site-packages/django/core/handlers/wsgi.py", line 255, in __call__
response = self.get_response(request)
File "/Users/kilroy/.virtualenvs/project_name/lib/python2.7/site-packages/django/core/handlers/base.py", line 178, in get_response
response = self.handle_uncaught_exception(request, resolver, sys.exc_info())
File "/Users/kilroy/.virtualenvs/project_name/lib/python2.7/site-packages/django/core/handlers/base.py", line 224, in handle_uncaught_exception
return callback(request, **param_dict)
File "/Users/kilroy/Sites/PYTHON/project_name/apps/common/views.py", line 436, in handler_500
messages.error(request, '500 - Internal server error.')
File "/Users/kilroy/.virtualenvs/project_name/lib/python2.7/site-packages/django/contrib/messages/api.py", line 102, in error
fail_silently=fail_silently)
File "/Users/kilroy/.virtualenvs/project_name/lib/python2.7/site-packages/django/contrib/messages/api.py", line 22, in add_message
raise MessageFailure('You cannot add messages without installing '
MessageFailure: You cannot add messages without installing django.contrib.messages.middleware.MessageMiddleware
I'm using different settings file for development and production, and on my development machine I set environment variable in /Users/kilroy/.virtualenvs/project_name/bin/postactivate file like this:
export DJANGO_SETTINGS_MODULE=project_name.settings.local
My base.py file looks like this https://dpaste.de/avXmL/, and my local.py settings like this https://dpaste.de/Z0PXv/.
Why is this happening when I want to disable debugging and how can I trace why this problem occurs in the first place?
I had the same stack trace when I set DEBUG false when using the development server. In my case it was related to the ALLOWED_HOSTS setting / check by Django. The symptoms were confusing because a SuspiciousOperation exception was being raised which caused the error.
If I had ALLOWED_HOSTS = ['localhost'] in my settings file and http://localhost:8000 as my browser address all was well. However, if I use the IP address http://127.0.0.1:8000/ directly I got this error.
Can you try this on shell:
from django.conf import settings
print settings.MIDDLEWARE_CLASSES
And if you don't see django.contrib.messages.middleware.MessageMiddleware in middleware_classes you should add it.
I guess you didn't get any 500 error while in development and so this line using messages.error(request, '500 - Internal server error.') was never executed and so you did not face this problem in DEBUG=True.
After updating from 1.7.5 (where everything worked fine) I'm getting a HTTP Error 403: Forbidden when trying to open any sites via localhost. Strange thing is I have pretty much the same setup at home as here at work and everything works there... Might be an issue with proxy server we're using at work, since that's the only difference I can think of? Here's the error log I'm getting, so if anyone knows what's going on please help (;
Traceback (most recent call last):
File "U:\Dev\GAE\lib\cherrypy\cherrypy\wsgiserver\wsgiserver2.py", line 1302, in communicate
req.respond()
File "U:\Dev\GAE\lib\cherrypy\cherrypy\wsgiserver\wsgiserver2.py", line 831, in respond
self.server.gateway(self).respond()
File "U:\Dev\GAE\lib\cherrypy\cherrypy\wsgiserver\wsgiserver2.py", line 2115, in respond
response = self.req.server.wsgi_app(self.env, self.start_response)
File "U:\Dev\GAE\google\appengine\tools\devappserver2\wsgi_server.py", line 246, in __call__
return app(environ, start_response)
File "U:\Dev\GAE\google\appengine\tools\devappserver2\request_rewriter.py", line 311, in _rewriter_middleware
response_body = iter(application(environ, wrapped_start_response))
File "U:\Dev\GAE\google\appengine\tools\devappserver2\python\request_handler.py", line 89, in __call__
self._flush_logs(response.get('logs', []))
File "U:\Dev\GAE\google\appengine\tools\devappserver2\python\request_handler.py", line 220, in _flush_logs
apiproxy_stub_map.MakeSyncCall('logservice', 'Flush', request, response)
File "U:\Dev\GAE\google\appengine\api\apiproxy_stub_map.py", line 94, in MakeSyncCall
return stubmap.MakeSyncCall(service, call, request, response)
File "U:\Dev\GAE\google\appengine\api\apiproxy_stub_map.py", line 320, in MakeSyncCall
rpc.CheckSuccess()
File "U:\Dev\GAE\google\appengine\api\apiproxy_rpc.py", line 156, in _WaitImpl
self.request, self.response)
File "U:\Dev\GAE\google\appengine\ext\remote_api\remote_api_stub.py", line 200, in MakeSyncCall
self._MakeRealSyncCall(service, call, request, response)
File "U:\Dev\GAE\google\appengine\ext\remote_api\remote_api_stub.py", line 226, in _MakeRealSyncCall
encoded_response = self._server.Send(self._path, encoded_request)
File "U:\Dev\GAE\google\appengine\tools\appengine_rpc.py", line 393, in Send
f = self.opener.open(req)
File "U:\Dev\Python\lib\urllib2.py", line 410, in open
response = meth(req, response)
File "U:\Dev\Python\lib\urllib2.py", line 523, in http_response
'http', request, response, code, msg, hdrs)
File "U:\Dev\Python\lib\urllib2.py", line 448, in error
return self._call_chain(*args)
File "U:\Dev\Python\lib\urllib2.py", line 382, in _call_chain
result = func(*args)
File "U:\Dev\Python\lib\urllib2.py", line 531, in http_error_default
raise HTTPError(req.get_full_url(), code, msg, hdrs, fp)
HTTPError: HTTP Error 403: Forbidden
INFO 2013-04-19 12:28:52,576 server.py:561] default: "GET / HTTP/1.1" 500 -
INFO 2013-04-19 12:28:52,619 server.py:561] default: "GET /favicon.ico HTTP/1.1" 304 -
Also, the launcher throws an error when closing:
Traceback (most recent call last):
File "launcher\mainframe.pyc", line 327, in OnStop
File "launcher\taskcontroller.pyc", line 167, in Stop
File "launcher\dev_appserver_task_thread.pyc", line 82, in stop
File "launcher\taskthread.pyc", line 107, in stop
File "launcher\platform.pyc", line 397, in KillProcess
pywintypes.error: (5, 'TerminateProcess', 'Access is denied.')
I had this very same issue with my MacOSX when using a proxy server using Google App Engine Launcher 1.8.6. Apparently there's an issue with "proxy_bypass" on "urllib2.py".
There are two possible solutions:
Downgrade to 1.7.5, but, who wants to downgrade?
Edit "[GAE Instalattion path]/google/appengine/tools/appengine_rpc.py" and look for the line that says
opener.add_handler(fancy_urllib.FancyProxyHandler())
In my computer it was line 578, and then put a hash (#) at the beginning of the line, like this:
`#opener.add_handler(fancy_urllib.FancyProxyHandler())`
Save the file, stop and then restart your application. Now dev_appserver.py shouldn't try to use any proxy server at all.
If your application uses any external resources like a SOAP Webservice or something like that and you can't reach the server without the proxy server, then you'll have to downgrade. Please keep in mind that external javascript files (like facebook SDK or similar) are loaded from your browser, not from your application.
Since I'm not using any external REST or SOAP services it worked for me!
Hopefully it will work for you as well.
Try either:
-Accessing it through a different proxy. I.E a . proxy within a proxy
-Accessing it through your local IP i.e 192.168.1.1
I faced the same issue with version 1.9.5. Seems that the API proxy is sending some RPCs to the proxy server, which are then being rejected with HTTP 403 (since proxy servers are generally configured to reject connection attempts to arbitrary ports). In my case I was using the urlfetch module in my app to access external web pages, so disabling the proxy server was not a choice for me.
This is how I worked around the issue some time back (most probably it was based on comments found under this issue, but I cannot remember the exact sources).
NOTE:
For this approach to work, you'll have to know the hostname/IP address and default port of your proxy server, and change them appropriately in the code if you happen to connect to a different proxy server.
When you are not behind the proxy server, you will have to revert the applied changes in order to return to a working state (if you want internet access inside your app).
Here it goes:
Disable proxy settings for the Python (Google App Engine Launcher) environment in some way. (In my case it was easy since I was launching the dev_appserver.py from a Terminal shell (on Linux), and the unset http_proxy and unset https_proxy commands did the trick.)
Edit {App Engine SDK root}/google/appengine/api/urlfetch_stub.py. Find the code block
if _CONNECTION_SUPPORTS_TIMEOUT:
connection = connection_class(host, timeout=deadline)
else:
connection = connection_class(host)
(lines 376-379 in my case) and replace it with:
if _CONNECTION_SUPPORTS_TIMEOUT:
if host[:9] == 'localhost' or host[:9] == '127.0.0.1':
connection = connection_class(host, timeout=deadline)
else:
connection = connection_class('your_proxy_host_goes_here', your_proxy_port_number_goes_here, timeout=deadline)
else:
if host[:9] == 'localhost' or host[:9] == '127.0.0.1':
connection = connection_class(host)
else:
connection = connection_class('your_proxy_host_goes_here', your_proxy_port_number_goes_here)
replacing the placeholders your_proxy_host_goes_here and your_proxy_port_number_goes_here with appropriate values.
(I believe this code can be written more elegantly, though... any Python geeks out there? :) )
In my case, I also had to delete the existing compiled file urlfetch_stub.pyc (located in the same directory as urlfetch_stub.py) because the SDK didn't seem to pick up the changes until I did so.
Now you can use dev_appserver to launch your app, and use urlfetch-backed services within the app, free from HTTP 403 errors.