Develop Reference

Django autoreload: add watched file

When source files in my project change, django server reloads. I want to extend this to non-Python source files. I use native SQL queries, which are stored in separate files (eg. big_select.sql), and I want the server to reload when these files change.
I use django on Windows.
I have tried adding .py extension, which didn't work.

Django>=2.2
The autoreloading was given a major overhaul (thanks to #Glenn who notified about the incoming changes in this comment!), so one doesn't have to use the undocumented Django features and append files to _cached_filenames anymore. Instead, register custom signal listener, listening to autoreloading start:
# apps.py
from django.apps import AppConfig
from django.utils.autoreload import autoreload_started
def my_watchdog(sender, **kwargs):
sender.watch_file('/tmp/foo.bar')
# to listen to multiple files, use watch_dir, e.g.
# sender.watch_dir('/tmp/', '*.bar')
class EggsConfig(AppConfig):
name = 'eggs'
def ready(self):
autoreload_started.connect(my_watchdog)
Django<2.2
Django stores the watched filepaths in the django.utils.autoreload._cached_filenames list, so adding to or removing items from it will force django to start or stop watching files.
As for your problem, this is the (kind of a hacky) solution. For the demo purpose, I adapted the apps.py so the file starts being watched right after django initializes, but feel free to put the code wherever you want to. First of all, create the file as django can watch only files that already exist:
$ touch /tmp/foo.bar
In your django app:
# apps.py
from django.apps import AppConfig
...
import django.utils.autoreload
class MyAppConfig(AppConfig):
name = 'myapp'
def ready(self):
...
django.utils.autoreload._cached_filenames.append('/tmp/foo.bar')
Now start the server, in another console modify the watched file:
$ echo baz >> /tmp/foo.bar
The server should trigger an autoreload now.

The accepted answer did not work in Django 3.0.7 probably due to changes since.
Came up with the following after going through autoreload:
from django.utils.autoreload import autoreload_started
# Watch .conf files
def watch_extra_files(sender, *args, **kwargs):
watch = sender.extra_files.add
# List of file paths to watch
watch_list = [
FILE1,
FILE2,
FILE3,
FILE4,
]
for file in watch_list:
if os.path.exists(file): # personal use case
watch(Path(file))
autoreload_started.connect(watch_extra_files)

how to let user define their own plugins under the scope and make it visible to himself?

I'm using the flask to built a web app ,it hold all the input message(user send it the xml message) and find the right plugin to response and return the response to user, the app had provided the basic plugins, but i want the user to write their own plugins under the my defined apis, below is the architecture:
plugins
common_plugins1.py
common_plugins2.py
templates
actions
myapp.py
but i faced several problems:
i only want the user call his plugin not others and other plugins is invisible to him
i only want the user call the defined functions or modules i defined
i want to make it scalable
if is it possible to let the user upload their plugins write by python? and make the app dynamically load it.
thanks for your help!
below is the plugin example:
#coding: utf-8
import somemodule
from somemodule import *
def do(dmessage,context,default,**option):
import re
try:
_l=default.split('%|placeholder|%')
message=ModuleRequestVoice(dmessage)
r=ModuleResponseMusic()
return r.render(message,_l[0],_l[1],_l[2],_l[2])
except Exception,e:
print 'match voice error:%s'%e
return False

How do I access ${buildout:directory} from Python code?

I have a Pyramid web application managed with zc.buildout. In it, I need to read a file on disk, which is located in a sub-directory of buildout directory.
The problem is with determining the path to the file - I do not want to hard-code the absolute path and just providing a relative path does not work when serving the app in production (supposedly because the working directory is different).
So the promising "hooks" I am thinking about are:
the "root" buildout directory, which I can address in buildout.cfg as ${buildout:directory} - however, I can't figure out how can I "export" it so it can be accessed by the Python code
the location of the Paster's .ini file which starts the app

Like #MartijnPieters suggests in a comment on your own answer, I'd use collective.recipe.template to generate an entry in the .ini. I wondered myself how I could then access that data in my project, so I worked it out :-)
Let's work our way backwards to what you need. First in your view code where you want the buildout directory:
def your_view(request):
buildout_dir = request.registry.settings['buildout_dir']
....
request.registry.settings (see documentation) is a "dictonary-like deployment settings object". See deployment settings, that's the **settings that gets passed into your main method like def main(global_config, **settings)
Those settings are what's in the [app:main] part of your deployment.ini or production.ini file. So add the buildout directory there:
[app:main]
use = egg:your_app
buildout_dir = /home/you/wherever/it/is
pyramid.reload_templates = true
pyramid.debug_authorization = false
...
But, and this is the last step, you don't want to have that hardcoded path in there. So generate the .ini with a template. The template development.ini.in uses a ${partname:variable} expansion language. in your case you need${buildout:directory}:
[app:main]
use = egg:your_app
buildout_dir = ${buildout:dir}
# ^^^^^^^^^^^^^^^
pyramid.reload_templates = true
pyramid.debug_authorization = false
...
Add a buildout part in buildout.cfg to generate development.ini from development.ini.in:
[buildout]
...
parts =
...
inifile
...
[inifile]
recipe = collective.recipe.template
input = ${buildout:directory}/development.ini.in
output = ${buildout:directory}/development.ini
Note that you can do all sorts of cool stuff with collective.recipe.template. ${serverconfig:portnumber} to generate a matching port number in your production.ini and in your your_site_name.nginx.conf, for instance. Have fun!

If the path to the file relative to the buildout root or location of paster.ini is always the same, which it seems it is from your question, you could set it in paster.ini:
[app:main]
...
config_file = %(here)s/path/to/file.txt
Then access it from the registry as in Reinout's answer:
def your_view(request):
config_file = request.registry.settings['config_file']

Here's a rather clumsy solution I've devised:
In buildout.cfg I used extra-paths option of zc.recipe.egg to add the buildout directory to sys.path:
....
[webserver]
recipe = zc.recipe.egg:scripts
eggs = ${buildout:eggs}
extra-paths = ${buildout:directory}
then I put a file called app_config.py into the buildout directory:
# This remembers the root of the installation (similar to {buildout:directory}
# so we can import it and use where we need access to the filesystem.
# Note: we could use os.getcwd() for that but it feels kinda wonky
# This is not directly related to Celery, we may want to move it somewhere
import os.path
INSTALLATION_ROOT = os.path.dirname(__file__)
Now we can import it in our Python code:
from app_config import INSTALLATION_ROOT
filename = os.path.join(INSTALLATION_ROOT, "somefile.ext")
do_stuff_with_file(filename)
If anyone knows a nicer solution you're welcome :)

Reload Flask app when template file changes

By default, when running Flask application using the built-in server (Flask.run), it monitors its Python files and automatically reloads the app if its code changes:
* Detected change in '/home/xion/hello-world/app.py', reloading
* Restarting with reloader
Unfortunately, this seems to work for *.py files only, and I don't seem to find any way to extend this functionality to other files. Most notably, it would be extremely useful to have Flask restart the app when a template changes. I've lost count on how many times I was fiddling with markup in templates and getting confused by not seeing any changes, only to find out that the app was still using the old version of Jinja template.
So, is there a way to have Flask monitor files in templates directory, or does it require diving into the framework's source?
Edit: I'm using Ubuntu 10.10. Haven't tried that on any other platforms really.
After further inquiry, I have discovered that changes in templates indeed are updated in real time, without reloading the app itself. However, this seems to apply only to those templates that are passed to flask.render_template.
But it so happens that in my app, I have quite a lot of reusable, parametrized components which I use in Jinja templates. They are implemented as {% macro %}s, reside in dedicated "modules" and are {% import %}ed into actual pages. All nice and DRY... except that those imported templates are apparently never checked for modifications, as they don't pass through render_template at all.
(Curiously, this doesn't happen for templates invoked through {% extends %}. As for {% include %}, I have no idea as I don't really use them.)
So to wrap up, the roots of this phenomenon seems to lie somewhere between Jinja and Flask or Werkzeug. I guess it may warrant a trip to bug tracker for either of those projects :) Meanwhile, I've accepted the jd.'s answer because that's the solution I actually used - and it works like a charm.

you can use
TEMPLATES_AUTO_RELOAD = True
From http://flask.pocoo.org/docs/1.0/config/
Whether to check for modifications of the template source and reload it automatically. By default the value is None which means that Flask checks original file only in debug mode.

In my experience, templates don't even need the application to restart to be refreshed, as they should be loaded from disk everytime render_template() is called. Maybe your templates are used differently though.
To reload your application when the templates change (or any other file), you can pass the extra_files argument to Flask().run(), a collection of filenames to watch: any change on those files will trigger the reloader.
Example:
from os import path, walk
extra_dirs = ['directory/to/watch',]
extra_files = extra_dirs[:]
for extra_dir in extra_dirs:
for dirname, dirs, files in walk(extra_dir):
for filename in files:
filename = path.join(dirname, filename)
if path.isfile(filename):
extra_files.append(filename)
app.run(extra_files=extra_files)
See here: http://werkzeug.pocoo.org/docs/0.10/serving/?highlight=run_simple#werkzeug.serving.run_simple

When you are working with jinja templates, you need to set some parameters. In my case with python3, I solved it with the following code:
if __name__ == '__main__':
app.jinja_env.auto_reload = True
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.run(debug=True, host='0.0.0.0')

You need to set a TEMPLATES_AUTO_RELOAD property as True in your app config:
from flask import Flask
app = Flask(__name__)
app.config["TEMPLATES_AUTO_RELOAD"] = True
See more on http://flask.pocoo.org/docs/1.0/config/

Actually for me TEMPLATES_AUTO_RELOAD = True does not work (0.12 version). I use jinja2 and what i have done:
Create function before_request
def before_request():
app.jinja_env.cache = {}
Register it in application
app.before_request(before_request)
That's it.

Updated as of March 2021:
The flask CLI is recommended over app.run() for running a dev server, so if we want to use the CLI then the accepted solution can't be used.
In Flask 1.1 or later, the environment variable FLASK_RUN_EXTRA_FILES or the option --extra-files effectively do the same thing as the accepted answer. See also this github issue.
Example usage:
flask run --extra-files "app/templates/index.html"
# or
export FLASK_RUN_EXTRA_FILES="app/templates/index.html"
flask run
in Linux. To specify multiple extra files, separate file paths with colons., e.g.
export FLASK_RUN_EXTRA_FILES="app/templates/index.html:app/templates/other.html"
Whole directories are also supported:
flask run --extra-files app/templates/

What worked for me is just adding this:
#app.before_request
def before_request():
# When you import jinja2 macros, they get cached which is annoying for local
# development, so wipe the cache every request.
if 'localhost' in request.host_url or '0.0.0.0' in request.host_url:
app.jinja_env.cache = {}
(taken from #dikkini's answer)

To reload the application on the server AND in the browser I used the livereload package. Installed through the CLI with
$ pip install livereload
and running the code
from flask import Flask, render_template
app = Flask(__name__)
#app.route("/")
def hello():
return render_template("index.html")
if __name__ == '__main__':
from livereload import Server
server = Server(app.wsgi_app)
server.serve(host = '0.0.0.0',port=5000)
all answers here using the extra_files argument or TEMPLATES_AUTO_RELOAD config work to reload it on the server but for a smooth development experience without damaging your keyboard's F5 key I'd go with livereload

Using the latest version of Flask on Windows, using the run command and debug set to true; Flask doesn't need to be reset for changes to templates to be brought in to effect. Try Shift+F5 (or Shift plus the reload button) to make sure nothing it being cached.

See http://flask.pocoo.org/docs/1.0/quickstart/
and use FLASK_ENV=development

I had the same trouble. The solution is really simple though. Instead of this:
if __name__ == '__main__':
app.jinja_env.auto_reload = True
app.config["TEMPLATES_AUTO_RELOAD"] = True
app.run(debug=True)
Put
app.jinja_env.auto_reload = True
app.config["TEMPLATES_AUTO_RELOAD"] = True
above the main function. So final output for example:
from flask import Flask, app,render_template
app= Flask(__name__)
app.jinja_env.auto_reload = True
app.config["TEMPLATES_AUTO_RELOAD"] = True
#app.route('/')
def index():
return render_template('index.html')
if __name__ == '__main__':
app.run(debug=True)

Templates are reloaded automatically, why not doing ctrl+f5 to refresh the webpage,
cause web-browsers usually save cache.

Adding app.config['TEMPLATES_AUTO_RELOAD'] = True after if __name__ == '__main__': doesn't work for me!
What works is adding app.config['TEMPLATES_AUTO_RELOAD'] = True after app = Flask(__name__)
Notice that I am using app.run(debug=True)

Bottle.py caching templates despite being in debug mode

I just built my first Bottle.py app on GAE. It's working except that when I change the templates I have to restart the dev server to see the changes. The docs say that template caching is supposed to be disabled when bottle.debug(True), and that you can call bottle.TEMPLTE.clear() as well, but neither of those work. I also tried setting run(reloader=True) but that causes an error. What am I doing wrong? Does bottle.debug() work for anyone else on GAE?
import bottle
bottle.debug(True)
bottle.TEMPLATES.clear()
#bottle.route('/')
def index(name='World'):
return bottle.template('main')
bottle.run(server='gae')
Update:
Instead of using bottle.run(server='gae'), I included the standard main() function myself and now it works.
def main():
app = bottle.default_app()
util.run_wsgi_app(app)
if __name__ == "__main__":
main()

The standard method introduced by Bottle/GAE doc is:
app = bottle.app()
then invoke dev_appserver.py, it reads app.yaml and import your app from the script you defined, and handle everything else for a GAE environment.
You shouldn't run your .py directly. Running from the bottle way will skip those handles from dev_appserver, including the template cached mechanism. Of course, using the util from Google does trick way and works, but according to uwsgi or other wsgi related projects' documents, the app variable in the script module is the object should be offered for the upper handling.

From the documentation:
Templates are cached in memory after compilation. Modifications made to the template files will have no affect until you clear the template cache. Call bottle.TEMPLATES.clear() to do so. Caching is disabled in debug mode.
The method run:
bottle.run( debug = True )
will enable debuggmode.
The default template is SimpleTemplate in stable version 0.11.6.
You can write your own adapter for your favourite template engine or
use one of the predefined adapters. Currently there are four fully
supported template engines:
Class,URL,Decorator,Render,function
SimpleTemplate, SimpleTemplate, Engine, view(), template()
MakoTemplate, http://www.makotemplates.org, mako_view(), mako_template()
CheetahTemplate, http://www.cheetahtemplate.org/, cheetah_view(), cheetah_template()
Jinja2Template, http://jinja.pocoo.org/, jinja2_view(), jinja2_template()
>>> Try using some other template engine, than the default. <<<
To use MakoTemplate as your default template engine, just import its
specialised decorator and render function:
from bottle import mako_view as view, mako_template as template
>>> Check that you dont have duplicated files in the view paths <<<
TEMPLATE_PATH = ['./', './views/']
>>> Print out templates dictionary <<<
print bottle.TEMPLATES