I made a change on one of the .js files that I use and no matter what I do, flask insists on picking up, from memory cache, the last version of the file, without the change.
To clarify, I have the following structure. It all starts with foo.html
return render_template foo.html
foo.html has a form inside that calls flask with some data and then returns a second template bar.html:
return render_template bar.html
This second template calls some .js file, placed in the static folder, but it doesn't update when the code changes.
I mention the structure above because if the .js file was placed on foo.html instead of bar.html then Flask would pick up the new changes on the file. But in bar.html Flask completely ignores them.
What is happening?
The only thing that worked was to click on "disable cache" on the browser and reload again.
Ultimately this is a frustrating browser cache issue, which can be solved by forcing the browser to do a "hard refresh", which is going to be a browser/OS dependent keystroke, but generally this works:
Windows: Ctrl+F5
Mac: Cmd+Shift+R
Linux: Ctrl+Shift+R
There are other filename tricks one can use to avoid this issue (mentioned in comments of the OP). These are especially important in production where you have no control over browser behavior.
For non-Static Flask responses you can set the cache_control.max_age property, which should tell the browser when to expire the response if it is cached. For instance if you have a Flask XHR endpoint that returns JSON data you could do this:
#app.route('/_get_ajax_data/')
def get_ajax_data():
data = {"hello": "world"}
response = jsonify(data)
response.cache_control.max_age = 60 * 60 * 24 # 1 day (in seconds)
return response
You typically can also set default values in your production web server configuration for specific resource types (e.g. CSS/JS/HTML/JSON/etc)
Edit 4/1/2019 (unrelated to April Fools day)
Mac / Safari keystroke now appears to be: Cmd+Opt+R (via comments, thanks!).
See the new answer from #MarredCheese for a very elegant "filename trick" to force the browser to ignore cached copies for updated files.
Caching is normally good, so it's not advisable to eliminate it entirely. And using control + F5 or whatever to do a hard refresh is obviously not a scalable solution since you have to do it in every browser on every computer.
A better idea is to let browsers cache the files most of the time, but not right after they've been updated. You can achieve this by appending the time that the source file was last updated as an argument to its path URL. For simplicity, you can use the most recent modification time for any file in the static folder (or any subfolders), as opposed to looking at each file individually.
Python
def dir_last_updated(folder):
return str(max(os.path.getmtime(os.path.join(root_path, f))
for root_path, dirs, files in os.walk(folder)
for f in files))
#app.route('/my-site')
def my_site():
return render_template('my-site.html',
last_updated=dir_last_updated('mydir/static'))
Jinja Template
<script type="text/javascript" src="/static/my-script.js?u={{ last_updated }}"></script>
HTML Result
<script type="text/javascript" src="/static/my-script.js?u=1547330602.31"></script>
If you are serving your static assets with Flask (this is typically the case in a development environment), then you might need to set the SEND_FILE_MAX_AGE_DEFAULT configuration value:
Default cache control max age to use with send_static_file() (the default static file handler) and send_file(), as datetime.timedelta or as seconds. Override this value on a per-file basis using the get_send_file_max_age() hook on Flask or Blueprint, respectively. Defaults to 43200 (12 hours).
Solving this can be as simple as updating the app.config dictionary, like so:
app = Flask(__name__)
...
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0
If you do that, your browser will not cache static assets that are served by Flask.
Use:
if __name__ == '__main__':
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0
app.run(debug=True)
You can update using F5 or CTRL + R.
USE CTRL+ f5 to refresh the browser
Related
I want to get some data from the browser's cache. Chrome's cache filename is like f_00001, which is meaningless. ChromeCacheView can obtain the request link corresponding to the cache file name.
ChromeCacheView is a small utility that reads the cache folder of Google Chrome Web browser, and displays the list of all files currently stored in the cache. For each cache file, the following information is displayed: URL, Content type, File size, Last accessed time, Expiration time, Server name, Server response, and more.
You can easily select one or more items from the cache list, and then extract the files to another folder, or copy the URLs list to the clipboard.
But this is a GUI program that can only run on Windows. I want to know how it works.
In other words, how can I get more information about cached files, especially request links etc.
After my long search, I found the answer.
Instructions for Chrome disk cache format can be found on the following pages:
Disk cache
Chrome Disk Cache Format
By reading these documents, we can implement parsers in arbitrary programming languages.
Fortunately, I found two python libraries to do this.
dtformats
pyhindsight
The first one doesn't seem to work correctly under Python3. The first one doesn't seem to work properly under Python3. And the second one is fantastic and does the job perfectly. About how to use pyhindsight, there are detailed instructions on the home page, I will introduce how to integrate it into our project.
import pyhindsight
from pyhindsight.analysis import AnalysisSession
import logging
import os
analysis_session = AnalysisSession()
cache_dir = '~\AppData\Local\Microsoft\Edge\User Data\Default'
analysis_session.input_path = cache_dir
analysis_session.cache_path = os.path.join(cache_dir, 'Cache\Cache_Data')
analysis_session.browser_type = 'Chrome'
analysis_session.no_copy = True
analysis_session.timezone = None
logging.basicConfig(filename=analysis_session.log_path, level=logging.FATAL,
format='%(asctime)s.%(msecs).03d | %(levelname).01s | %(message)s',
datefmt='%Y-%m-%d %H:%M:%S')
run_status = analysis_session.run()
for p in analysis_session.parsed_artifacts:
if isinstance(p, pyhindsight.browsers.chrome.CacheEntry):
print('Url: {}, Location: {}'.format(p.url, p.location))
That's all, please join it. Thanks for pyhindsight.
The changes used to reflect on it when I'd refresh the page, but then it just stopped and stayed stuck on that same style. Changes I make to the html and the python code continue to reflect on it though. I read a similar question here about it, that made me notice that the terminal now says
Restarting with stat
instead of
Restarting with reloader
I tried installing the watchdog package like that thread had suggested, but that didn't work either.
You "cache bust" the CSS by appending the modification stamp of the file. Something like
css_path = os.path.join(os.dirname(__file__), 'static', 'style.css')
css_time = int(os.stat(css_path).st_mtime)
...
#app.context_processor
def inject_css_mtime()
return {'css_mtime', css_mtime}
This makes css_mtime available to templates.
<link rel=stylesheet href="{{ url_for('static', filename='style.css') }}?v={{ css_mtime }}">
When the .css file changes, its mtime will change, and browsers will know to request it again. Adjust the css_path calculation to match your app structure.
You'll need to do this for any JavaScript files, too.
I am running a server in django and I want to serve a file. I am storing the file under '/upload/directory/filename' and I return it using
from django.shortcuts import redirect
file_path = '/upload/directory/filename'
return redirect(file_path)
However, the file appears to have been cached to the first version that had been placed locally and is never updated. Even if I remove the file, the file is still served. I checked that if I change the path to 'upload/directory_2/filename then I correctly get my new file. What is going on and how can I fight this ?
This is happening locally and I am making a direct server request (hence there is no possibility of browser caching or anything else).
Additional information:
I understand that maybe I should be using static files, although for instance this answer suggests that it is quite debatable for files that I am uploading myself.
When I say "I want to serve files with django" I just mean that I have associated a file path to a particular entity in my database (using models.FileField) and based on what the user requests I want to return this file. I kind of doubt this is a clear cut for using static files in that case.
There are many workarounds to my issue, like generating unique filenames every time I want to "clear my cache" or explicitly opening the file:
with open(absolute_file_path) as file:
response = HttpResponse(file.read(), content_type='application/octet-stream')
My question was about understanding why the particular piece of code above does what it does, i.e. leads to data caching, and how to prevent this.
If you must do this using Django itself, I would suggest skipping the redirect and setting up your app according to these directions:
https://docs.djangoproject.com/en/1.11/howto/static-files/#serving-files-uploaded-by-a-user-during-development
Make your MEDIA_URL something like /media/ or if you want it to match your current case /upload/ or something. MEDIA_ROOT could point to os.path.join(BASE_DIR, 'upload') and your FileField(upload_to='directory').
You must use django static facility to serve static files. On your local machine this is setup properly in settings to point a folder static in your app then you can point to the file using the static function.
All is explained here:
https://docs.djangoproject.com/en/1.11/howto/static-files/
In an url you access the code:
from django.templatetags.static import static
url_to_file = static('some_app/path/to_file')
On a production machine the static files are served by a web server or a specific service like aws S3 or similar but not from django! For this reason the static facility is a must.
To avoid the cache, in case you have this problem, have a look at the never_cache decorator: https://docs.djangoproject.com/en/1.11/topics/http/decorators/#caching
There is a special HttpResponse in Django that can help you to serve a file easily.
def file_dowload(request):
file_name = request.GET.get('file_name', '')
try:
temp = open(file_name, 'rb')
except IOError:
temp = None
if not temp:
return HttpResponse("Not found")
response = FileResponse(temp, content_type='text/plain')
return response
I have 2 static directories in Flask.
static/
css/
js/
results/
1/
index.html
details.json
2/
index.html
details.json
I followed along a few other answers and glanced through the documentation for serving static files.
app = Flask(__name__)
app.config['RESULT_STATIC_PATH'] = "results/"
#app.route('/results/<path:file>')
def serve_results(file):
# Haven't used the secure way to send files yet
return send_from_directory(app.config['RESULT_STATIC_PATH'], file)
With the following code I can now request for files in the results directory.
I'm more familiar with PHP and Apache. Say if a directory has an index.php or index.html file, it is served automatically.
My requirement:-
Access any file in the results directory
If I send a request for localhost:3333/results/1 I should be served index.html
I can do the same with Flask by adding a route and checking if index.html exists withing the sub directory and then serving it. I found a few more similar pointers here
I currently use two routes to get the functionality. There certainly is a better way.
Why did I include details about the static directory?
It is highly possible that I may be doing something wrong. Please let me know. Thanks :)
Try this:
#app.route('/results/<path:file>', defaults={'file': 'index.html'})
def serve_results(file):
# Haven't used the secure way to send files yet
return send_from_directory(app.config['RESULT_STATIC_PATH'], file)
This is how you pass a default value for that argument. However, I don't agree to this, if you want to serve static content just set a rule in your web server (apache in your case).
I'm using the following single route solution, the "defaults" didn't work for me, as it stopped serving any other file.
Instead of raising the error when the path does not end with a slash, you can also generate a redirect to the path with a slash. Not having this slash but serving the index file with cause problems with relative paths.
#app.route('/<path:path>')
def staticHost(self, path):
try:
return flask.send_from_directory(app.config['RESULT_STATIC_PATH'], path)
except werkzeug.exceptions.NotFound as e:
if path.endswith("/"):
return flask.send_from_directory(app.config['RESULT_STATIC_PATH'], path + "index.html")
raise e
I couldn't find a solution which only used one route. I ended up using this.
#app.route('/results/<path:filename>')
def serve_static(filename):
return send_from_directory("results/", filename)
#app.route('/results/<guid>')
def index(guid):
return send_from_directory("results/" + guid, 'index.html')
Other suggestions
#ipinak suggested using defaults={'file': 'index.html'} as another parameter. That doesn't solve this problem as it results in some security issue which #ipinak has been kind enough to research about.
defaults is a neat option which I'll try using in other places of my application as well.
#dirn suggested in the comments to use Apache/Nginx to serve static files. It would save time as I wouldn't have to implement stuff in Flask. Since static files don't have to passed to the the Python interpreter, we can serve them via another server with hardly any extra work.
I finally managed to get my web-server up and running python.
I am now able to enter the direct url pointing to the python script and it is being executed.
I wrote a script which generated a html page and returns it - working; and if string is saved as html and opened, it is displayed exactly how I want it.
However, I want this page to be displayed if a user navigates to the "Database" link which is displayed in the navigation menu
<nav>
<ul class="main-nav">
**<li>Database</li>**
<li>About</li>
<li>Admin</li>
</ul>
</nav>
I would like the script to be executed when the user clicks on the menu link but I do not want the user to see the actual link to the python script.
Ie the link should display localhost/database.html but the script should be executed and whatever is returned should be displayed.
I hope this makes sense - any help is appreciated
The easiest way to achieve what you want (the end user not seeing that its not a HTML file) is to move the file into a folder and configure the server to execute and server the results of that file when the user navigates to that folder, i.e. move the file into a folder called:
/database
and then change your link to point there, then configure the webserver to accept the python filename as a default file, in your case running apache it should be something like this:
Find the DirectoryIndex directive in your Apache configuration file (httpd.conf) or add it to a .htaccess file and change it to look like this if you want to limit your default index file to just index.html:
DirectoryIndex index.html
You can also include more resources and they will be used in the order given, e.g
DirectoryIndex index.html index.php
would display the index.html file first if both index.html and index.php existed.
Don't forget to restart Apache if you made the change to the httpd.conf file.
this way the url that you would see when testing locally would be:
localhost/database
and avoids putting any extension on it, this is simpler that trying to do some kind of redirection.
personally when using pages that do something like perform a query i tend to call them process to indicate that they actually do something when i'm working on it, and static files i tent to call index (unless there are multiple files in that location) this isn't necessary, just my personal preference, but it helps keep the configuration simple as there are then only so many names you have to add to the default list.