Develop Reference

Routing static pages with Flask or Bottle

When using:
#route('/<filename>')
def server_static(filename):
return static_file(filename, root='.')
it allows to serve the request www.example.com/helloworld with the static HTML file /myapp/helloworld.
How to make www.example.com/anything be served by the static HTML file /myapp/html/anything.html, without having to hardcode each static filename anything in the Python code?
Note: the tricky part is that the request is /anything (and not /anything.html), and the static file is /myapp/html/anything.html (there are 20 or more such files)

If it's always .html you could consider simply doing filename += '.html'.
If your needs are more complex, you could write code that examines the directory to match various extensions. For example, if you wanted to match .html files, something like this would work and would be adjustable to work with various/multiple extensions or other conditions.
def is_html(filename):
return filename.lower().endswith('.html')
#route('/<filename>')
def server_static(filename):
root = '.'
all_filenames = os.listdir(root)
html_files = filter(is_html, all_filenames)
for fname in html_files:
if fname.startswith(filename):
return static_file(fname, root=root)
else:
return "No such file"
This can probably be abbreviated using fnmatch or something similar.
Although, you may want to simply consider simply having a webserver like Apache simply serve that path instead of using Flask. This would probably be a more secure option, too.

I want to serve files from a folder with webob

I want to use webob.static.DirectoryApp. I just can't figure out how to do it:
From the example at http://docs.webob.org/en/latest/file-example.html my router looks like:
class Router(object):
def __init__(self, static_path=None):
self.routes = []
self.static_path = static_path if static_path is not None else os.path.join(os.path.dirname(os.path.realpath(__file__)), 'static')
self.static_app = DirectoryApp(self.static_path)
def add_route(self, template, controller, **vars):
if isinstance(controller, basestring):
controller = load_controller(controller)
self.routes.append((re.compile(template_to_regex(template)),
controller,
vars))
def __call__(self, environ, start_response):
req = Request(environ)
for regex, controller, vars in self.routes:
match = regex.match(req.path_info)
if match:
req.urlvars = match.groupdict()
req.urlvars.update(vars)
return controller(environ, start_response)
return exc.HTTPNotFound()(environ, start_response)
To create the application to serve:
def create_app():
router = Router()
#router.add_route('/', controller='app.controllers.default:index')
router.add_route('/', controller=default.index)
return router
This serves routes to my added controllers fine. I also added a self.static_app.
I just have no idea how to use it to serve static files from the static-folder! Can someone please enlighten me?

As #gawel wrote the answer is to add a route with DirectoryApp as the controller. There is another thing to notice for it to work directly from the example.
The extension template_to_regex in the example is adding a $ to the forcing us to write a route expression that covers the entire url to catch. The canbewhatever is never used and can be whatever. The important thing is the regex after the variable.
The add_route should look something like this:
router.add_route('/static{canbewhatever:.*}', controller=static_app)

Get list of all routes defined in the Flask app

I have a complex Flask-based web app. There are lots of separate files with view functions. Their URLs are defined with the #app.route('/...') decorator. Is there a way to get a list of all the routes that have been declared throughout my app? Perhaps there is some method I can call on the app object?

All the routes for an application are stored on app.url_map which is an instance of werkzeug.routing.Map. You can iterate over the Rule instances by using the iter_rules method:
from flask import Flask, url_for
app = Flask(__name__)
def has_no_empty_params(rule):
defaults = rule.defaults if rule.defaults is not None else ()
arguments = rule.arguments if rule.arguments is not None else ()
return len(defaults) >= len(arguments)
#app.route("/site-map")
def site_map():
links = []
for rule in app.url_map.iter_rules():
# Filter out rules we can't navigate to in a browser
# and rules that require parameters
if "GET" in rule.methods and has_no_empty_params(rule):
url = url_for(rule.endpoint, **(rule.defaults or {}))
links.append((url, rule.endpoint))
# links is now a list of url, endpoint tuples
See Display links to new webpages created for a bit more information.

I just met the same question. Those solutions above are too complex.
Just open a new shell under your project:
>>> from app import app
>>> app.url_map
The first 'app' is my project script: app.py,
another is my web's name.
(this solution is for the tiny web with a little route)

I make a helper method on my manage.py:
#manager.command
def list_routes():
import urllib
output = []
for rule in app.url_map.iter_rules():
options = {}
for arg in rule.arguments:
options[arg] = "[{0}]".format(arg)
methods = ','.join(rule.methods)
url = url_for(rule.endpoint, **options)
line = urllib.unquote("{:50s} {:20s} {}".format(rule.endpoint, methods, url))
output.append(line)
for line in sorted(output):
print line
It solves the the missing argument by building a dummy set of options. The output looks like:
CampaignView:edit HEAD,OPTIONS,GET /account/[account_id]/campaigns/[campaign_id]/edit
CampaignView:get HEAD,OPTIONS,GET /account/[account_id]/campaign/[campaign_id]
CampaignView:new HEAD,OPTIONS,GET /account/[account_id]/new
Then to run it:
python manage.py list_routes
For more on manage.py checkout: http://flask-script.readthedocs.org/en/latest/

Apparently, since version 0.11, Flask has a built-in CLI. One of the built-in commands lists the routes:
FLASK_APP='my_project.app' flask routes

Similar to Jonathan's answer I opted to do this instead. I don't see the point of using url_for as it will break if your arguments are not string e.g. float
#manager.command
def list_routes():
import urllib
output = []
for rule in app.url_map.iter_rules():
methods = ','.join(rule.methods)
line = urllib.unquote("{:50s} {:20s} {}".format(rule.endpoint, methods, rule))
output.append(line)
for line in sorted(output):
print(line)

Use cli command in Directory where your flask project is.
flask routes

Since you did not specify that it has to be run command-line, the following could easily be returned in json for a dashboard or other non-command-line interface. The result and the output really shouldn't be commingled from a design perspective anyhow. It's bad program design, even if it is a tiny program. The result below could then be used in a web application, command-line, or anything else that ingests json.
You also didn't specify that you needed to know the python function associated with each route, so this more precisely answers your original question.
I use below to add the output to a monitoring dashboard myself. If you want the available route methods (GET, POST, PUT, etc.), you would need to combine it with other answers above.
Rule's repr() takes care of converting the required arguments in the route.
def list_routes():
routes = []
for rule in app.url_map.iter_rules():
routes.append('%s' % rule)
return routes
The same thing using a list comprehension:
def list_routes():
return ['%s' % rule for rule in app.url_map.iter_rules()]
Sample output:
{
"routes": [
"/endpoint1",
"/nested/service/endpoint2",
"/favicon.ico",
"/static/<path:filename>"
]
}

If you need to access the view functions themselves, then instead of app.url_map, use app.view_functions.
Example script:
from flask import Flask
app = Flask(__name__)
#app.route('/foo/bar')
def route1():
pass
#app.route('/qux/baz')
def route2():
pass
for name, func in app.view_functions.items():
print(name)
print(func)
print()
Output from running the script above:
static
<bound method _PackageBoundObject.send_static_file of <Flask '__main__'>>
route1
<function route1 at 0x128f1b9d8>
route2
<function route2 at 0x128f1ba60>
(Note the inclusion of the "static" route, which is created automatically by Flask.)

You can view all the Routes via flask shell by running the following commands after exporting or setting FLASK_APP environment variable.
flask shell
app.url_map

inside your flask app do:
flask shell
>>> app.url_map
Map([<Rule '/' (OPTIONS, HEAD, GET) -> helloworld>,
<Rule '/static/<filename>' (OPTIONS, HEAD, GET) -> static>])

print(app.url_map)
That, is, if your Flask application name is 'app'.
It's an attribute of the instance of the Flask App.
See https://flask.palletsprojects.com/en/2.1.x/api/#flask.Flask.url_map

django static files versioning

I'm working on some universal solution for problem with static files and updates in it.
Example: let's say there was site with /static/styles.css file - and site was used for a long time - so a lot of visitors cached this file in browser
Now we doing changes in this css file, and update on server, but some users still have old version (despite modification date returned by server)
The obvious solution is to add some version to file /static/styles.css?v=1.1 but in this case developer must track changes in this file and manually increase version
A second solution is to count the md5 hash of the file and add it to the url /static/styels.css/?v={mdp5hashvalue} which looks much better, but md5 should be calculated automatically somehow.
they possible way I see it - create some template tag like this
{% static_file "style.css" %}
which will render
<link src="/static/style.css?v=md5hash">
BUT, I do not want this tag to calculate md5 on every page load, and I do not want to store hash in django-cache, because then we will have to clear after updating file...
any thoughts ?

Django 1.4 now includes CachedStaticFilesStorage which does exactly what you need (well... almost).
Since Django 2.2 ManifestStaticFilesStorage should be used instead of CachedStaticFilesStorage.
You use it with the manage.py collectstatic task. All static files are collected from your applications, as usual, but this storage manager also creates a copy of each file with the MD5 hash appended to the name. So for example, say you have a css/styles.css file, it will also create something like css/styles.55e7cbb9ba48.css.
Of course, as you mentioned, the problem is that you don't want your views and templates calculating the MD5 hash all the time to find out the appropriate URLs to generate. The solution is caching. Ok, you asked for a solution without caching, I'm sorry, that's why I said almost. But there's no reason to reject caching, really. CachedStaticFilesStorage uses a specific cache named staticfiles. By default, it will use your existing cache system, and voilà! But if you don't want it to use your regular cache, perhaps because it's a distributed memcache and you want to avoid the overhead of network queries just to get static file names, then you can setup a specific RAM cache just for staticfiles. It's easier than it sounds: check out this excellent blog post. Here's what it would look like:
CACHES = {
'default': {
'BACKEND': 'django.core.cache.backends.memcached.PyLibMCCache',
'LOCATION': '127.0.0.1:11211',
},
'staticfiles': {
'BACKEND': 'django.core.cache.backends.locmem.LocMemCache',
'LOCATION': 'staticfiles-filehashes'
}
}

I would suggest using something like django-compressor. In addition to automatically handling this type of stuff for you, it will also automatically combine and minify your files for fast page load.
Even if you don't end up using it in entirety, you can inspect their code for guidance in setting up something similar. It's been better vetted than anything you'll ever get from a simple StackOverflow answer.

I use my own templatetag which add file modification date to url: https://bitbucket.org/ad3w/django-sstatic

Is reinventing the wheel and creating own implementation that bad? Furthermore I would like low level code (nginx for example) to serve my staticfiles in production instead of python application, even with backend. And one more thing: I'd like links stay the same after recalculation, so browser fetches only new files. So here's mine point of view:
template.html:
{% load md5url %}
<script src="{% md5url "example.js" %}"/>
out html:
static/example.js?v=5e52bfd3
settings.py:
STATIC_URL = '/static/'
STATIC_ROOT = os.path.join(PROJECT_DIR, 'static')
appname/templatetags/md5url.py:
import hashlib
import threading
from os import path
from django import template
from django.conf import settings
register = template.Library()
class UrlCache(object):
_md5_sum = {}
_lock = threading.Lock()
#classmethod
def get_md5(cls, file):
try:
return cls._md5_sum[file]
except KeyError:
with cls._lock:
try:
md5 = cls.calc_md5(path.join(settings.STATIC_ROOT, file))[:8]
value = '%s%s?v=%s' % (settings.STATIC_URL, file, md5)
except IsADirectoryError:
value = settings.STATIC_URL + file
cls._md5_sum[file] = value
return value
#classmethod
def calc_md5(cls, file_path):
with open(file_path, 'rb') as fh:
m = hashlib.md5()
while True:
data = fh.read(8192)
if not data:
break
m.update(data)
return m.hexdigest()
#register.simple_tag
def md5url(model_object):
return UrlCache.get_md5(model_object)
Note, to apply changes an uwsgi application (to be specific a process) should be restarted.

Django 1.7 added ManifestStaticFilesStorage, a better alternative to CachedStaticFilesStorage that doesn't use the cache system and solves the problem of the hash being computed at runtime.
Here is an excerpt from the documentation:
CachedStaticFilesStorage isn’t recommended – in almost all cases ManifestStaticFilesStorage is a better choice. There are several performance penalties when using CachedStaticFilesStorage since a cache miss requires hashing files at runtime. Remote file storage require several round-trips to hash a file on a cache miss, as several file accesses are required to ensure that the file hash is correct in the case of nested file paths.
To use it, simply add the following line to settings.py:
STATICFILES_STORAGE = 'django.contrib.staticfiles.storage.ManifestStaticFilesStorage'
And then, run python manage.py collectstatic; it will append the MD5 to the name of each static file.

How about you always have a URL Parameter in your URL with a version and whenever you have a major release you change the version in your URL Parameter. Even in the DNS. So if www.yourwebsite.com loads up www.yourwebsite.com/index.html?version=1.0 then after the major release the browser should load www.yourwebsite.com/index.html?version=2.0
I guess this is similar to your solution 1. Instead of tracking files can you track whole directories? For example ratehr than /static/style/css?v=2.0 can you do /static-2/style/css or to make it even granular /static/style/cssv2/.

There is an update for #deathangel908 code. Now it works well with S3 storage also (and with any other storage I think). The difference is using of static file storage for getting file content. Original doesn't work on S3.
appname/templatetags/md5url.py:
import hashlib
import threading
from django import template
from django.conf import settings
from django.contrib.staticfiles.storage import staticfiles_storage
register = template.Library()
class UrlCache(object):
_md5_sum = {}
_lock = threading.Lock()
#classmethod
def get_md5(cls, file):
try:
return cls._md5_sum[file]
except KeyError:
with cls._lock:
try:
md5 = cls.calc_md5(file)[:8]
value = '%s%s?v=%s' % (settings.STATIC_URL, file, md5)
except OSError:
value = settings.STATIC_URL + file
cls._md5_sum[file] = value
return value
#classmethod
def calc_md5(cls, file_path):
with staticfiles_storage.open(file_path, 'rb') as fh:
m = hashlib.md5()
while True:
data = fh.read(8192)
if not data:
break
m.update(data)
return m.hexdigest()
#register.simple_tag
def md5url(model_object):
return UrlCache.get_md5(model_object)

The major advantage of this solution: you dont have to modify anything in the templates.
This will add the build version into the STATIC_URL, and then the webserver will remove it with a Rewrite rule.
settings.py
# build version, it's increased with each build
VERSION_STAMP = __versionstr__.replace(".", "")
# rewrite static url to contain the number
STATIC_URL = '%sversion%s/' % (STATIC_URL, VERSION_STAMP)
So the final url would be for example this:
/static/version010/style.css
And then Nginx has a rule to rewrite it back to /static/style.css
location /static {
alias /var/www/website/static/;
rewrite ^(.*)/version([\.0-9]+)/(.*)$ $1/$3;
}

Simple templatetag vstatic that creates versioned static files urls that extends Django's behaviour:
from django.conf import settings
from django.contrib.staticfiles.templatetags.staticfiles import static
#register.simple_tag
def vstatic(path):
url = static(path)
static_version = getattr(settings, 'STATIC_VERSION', '')
if static_version:
url += '?v=' + static_version
return url
If you want to automatically set STATIC_VERSION to the current git commit hash, you can use the following snippet (Python3 code adjust if necessary):
import subprocess
def get_current_commit_hash():
try:
return subprocess.check_output(['git', 'rev-parse', '--short', 'HEAD']).strip().decode('utf-8')
except:
return ''
At settings.py call get_current_commit_hash(), so this will be calculated only once:
STATIC_VERSION = get_current_commit_hash()

I use a global base context in all my views, where I set the static version to be the millisecond time (that way, it will be a new version every time I restart my application):
# global base context
base_context = {
"title": settings.SITE_TITLE,
"static_version": int(round(time.time() * 1000)),
}
# function to merge context with base context
def context(items: Dict) -> Dict:
return {**base_context, **items}
# view
def view(request):
cxt = context({<...>})
return render(request, "page.html", cxt)
my page.html extends my base.html template, where I use it like this:
<link rel="stylesheet" type="text/css" href="{% static 'style.css' %}?v={{ static_version }}">
fairly simple and does the job

pyfacebook + Google App Engine: can't find new functions in facebook.py

I'm trying to use the pyfacebook functions (https://github.com/sciyoshi/pyfacebook/) in a Google app engine project. I've followed the advice on the Facebook developer forum (http://forum.developers.facebook.net/viewtopic.php?pid=164613) and added the additional functions to the __init__.py file, copied that file to the root directory of my project and renamed it facebook.py. Having imported facebook.py I added the following to the get(self) method for the Python class for the page:
facebookapi = facebook.Facebook(API_KEY, SECRET)
if not facebookapi.check_connect_session(self.request):
path = os.path.join(os.path.dirname(__file__), 'templates/login.html')
self.response.out.write(template.render(path, {'apikey': API_KEY}))
return
user = facebookapi.users.getInfo(
[facebookapi.uid],
['uid', 'name', 'birthday', 'relationship_status'])[0]
template_values = {
'name': user['name'],
'birthday': user['birthday'],
'relationship_status': user['relationship_status'],
'uid': user['uid'],
'apikey': API_KEY
}
path = os.path.join(os.path.dirname(__file__), 'templates/index.html')
self.response.out.write(template.render(path, template_values))
When running it I get the following error:
File "\much\baw08u\Private\IDS\helloworld\helloworld.py", line 54, in get
if not facebookapi.check_connect_session(self.request):
AttributeError: 'Facebook' object has no attribute 'check_connect_session'
So it seems to be loading the facebook API fine, but not the new methods I've added. I copied and pasted the code from the developer forum at the bottom of the Facebook class definition, and made sure all the indentation was right but it still doesn't seem to be picking them up. Does anyone know what might be the problem?
Thanks
Ben

You believe the Facebook class has a certain method but Python is sure it hasn't. Why? Maybe you misspelled the method name, maybe you did not get the indentation right - hard to say without seeing the code.
You could try poking around to validate your assumptions:
import facebook
import logging
logging.warn('Facebook class: %r', dir(facebook.Facebook))
logging.warn('facebook module: %r', dir(facebook))
If you are sure you are operating on the correct file, the you should expect to see check_connect_session as a method of Facebook. If you didn't add enough indentation then you expect to see check_connect_method as a function defined in the facebook module. Too much indentation would make check_connect_method a sub function of which ever method precedes it and it won't show up in the above logging. Pay close attention to indentation.
However, a better way to add some custom methods might be:
import facebook
class Facebook(facebook.Facebook):
def check_connect_session(request):
pass
facebookapi = Facebook(API_KEY, SECRET)
if not facebookapi.check_connect_session(...):
...
Now when Facebook update their code you simply copy the new file into place - no need to merge your customisations.