I am running a server in django and I want to serve a file. I am storing the file under '/upload/directory/filename' and I return it using
from django.shortcuts import redirect
file_path = '/upload/directory/filename'
return redirect(file_path)
However, the file appears to have been cached to the first version that had been placed locally and is never updated. Even if I remove the file, the file is still served. I checked that if I change the path to 'upload/directory_2/filename then I correctly get my new file. What is going on and how can I fight this ?
This is happening locally and I am making a direct server request (hence there is no possibility of browser caching or anything else).
Additional information:
I understand that maybe I should be using static files, although for instance this answer suggests that it is quite debatable for files that I am uploading myself.
When I say "I want to serve files with django" I just mean that I have associated a file path to a particular entity in my database (using models.FileField) and based on what the user requests I want to return this file. I kind of doubt this is a clear cut for using static files in that case.
There are many workarounds to my issue, like generating unique filenames every time I want to "clear my cache" or explicitly opening the file:
with open(absolute_file_path) as file:
response = HttpResponse(file.read(), content_type='application/octet-stream')
My question was about understanding why the particular piece of code above does what it does, i.e. leads to data caching, and how to prevent this.
If you must do this using Django itself, I would suggest skipping the redirect and setting up your app according to these directions:
https://docs.djangoproject.com/en/1.11/howto/static-files/#serving-files-uploaded-by-a-user-during-development
Make your MEDIA_URL something like /media/ or if you want it to match your current case /upload/ or something. MEDIA_ROOT could point to os.path.join(BASE_DIR, 'upload') and your FileField(upload_to='directory').
You must use django static facility to serve static files. On your local machine this is setup properly in settings to point a folder static in your app then you can point to the file using the static function.
All is explained here:
https://docs.djangoproject.com/en/1.11/howto/static-files/
In an url you access the code:
from django.templatetags.static import static
url_to_file = static('some_app/path/to_file')
On a production machine the static files are served by a web server or a specific service like aws S3 or similar but not from django! For this reason the static facility is a must.
To avoid the cache, in case you have this problem, have a look at the never_cache decorator: https://docs.djangoproject.com/en/1.11/topics/http/decorators/#caching
There is a special HttpResponse in Django that can help you to serve a file easily.
def file_dowload(request):
file_name = request.GET.get('file_name', '')
try:
temp = open(file_name, 'rb')
except IOError:
temp = None
if not temp:
return HttpResponse("Not found")
response = FileResponse(temp, content_type='text/plain')
return response
Related
I have a django app (my_app) that based on the user query:
creates a file from the db
runs a program on the file from step-1 and gets an output file
generates a json file from the step-2 output file
renders a D3 visualization from a django template using the data from the json file from step-3
I need the program to run on the server side and the json file to be generated server-side as well.
Because the json files are query-specific, I thought it's not a good idea to keep these files in the /static/ folder and thought of keeping the files (even if temporarily) in e.g. /myapp/output_files/ folder.
The problem is that there is no url pattern corresponding to /myapp/output_files/my_file.json and I get a "Page not found (404)" error if I try to open the generated file and it obviously doesn't load in the javascript code in the template.
Is there a better way to design the system?
If the design is ok, how can I access a json file in the app's folder from the django template? Do I need something in the urls.py?
P.S. Everything works fine if I change the json location to /static/ or its subfolder.
Just add the location to your STATICFILES_DIRS setting as shown here
However, you probably need to build a view function that can somehow return the json based on some parameter in the url. Static files are meant to stay static...
I made a change on one of the .js files that I use and no matter what I do, flask insists on picking up, from memory cache, the last version of the file, without the change.
To clarify, I have the following structure. It all starts with foo.html
return render_template foo.html
foo.html has a form inside that calls flask with some data and then returns a second template bar.html:
return render_template bar.html
This second template calls some .js file, placed in the static folder, but it doesn't update when the code changes.
I mention the structure above because if the .js file was placed on foo.html instead of bar.html then Flask would pick up the new changes on the file. But in bar.html Flask completely ignores them.
What is happening?
The only thing that worked was to click on "disable cache" on the browser and reload again.
Ultimately this is a frustrating browser cache issue, which can be solved by forcing the browser to do a "hard refresh", which is going to be a browser/OS dependent keystroke, but generally this works:
Windows: Ctrl+F5
Mac: Cmd+Shift+R
Linux: Ctrl+Shift+R
There are other filename tricks one can use to avoid this issue (mentioned in comments of the OP). These are especially important in production where you have no control over browser behavior.
For non-Static Flask responses you can set the cache_control.max_age property, which should tell the browser when to expire the response if it is cached. For instance if you have a Flask XHR endpoint that returns JSON data you could do this:
#app.route('/_get_ajax_data/')
def get_ajax_data():
data = {"hello": "world"}
response = jsonify(data)
response.cache_control.max_age = 60 * 60 * 24 # 1 day (in seconds)
return response
You typically can also set default values in your production web server configuration for specific resource types (e.g. CSS/JS/HTML/JSON/etc)
Edit 4/1/2019 (unrelated to April Fools day)
Mac / Safari keystroke now appears to be: Cmd+Opt+R (via comments, thanks!).
See the new answer from #MarredCheese for a very elegant "filename trick" to force the browser to ignore cached copies for updated files.
Caching is normally good, so it's not advisable to eliminate it entirely. And using control + F5 or whatever to do a hard refresh is obviously not a scalable solution since you have to do it in every browser on every computer.
A better idea is to let browsers cache the files most of the time, but not right after they've been updated. You can achieve this by appending the time that the source file was last updated as an argument to its path URL. For simplicity, you can use the most recent modification time for any file in the static folder (or any subfolders), as opposed to looking at each file individually.
Python
def dir_last_updated(folder):
return str(max(os.path.getmtime(os.path.join(root_path, f))
for root_path, dirs, files in os.walk(folder)
for f in files))
#app.route('/my-site')
def my_site():
return render_template('my-site.html',
last_updated=dir_last_updated('mydir/static'))
Jinja Template
<script type="text/javascript" src="/static/my-script.js?u={{ last_updated }}"></script>
HTML Result
<script type="text/javascript" src="/static/my-script.js?u=1547330602.31"></script>
If you are serving your static assets with Flask (this is typically the case in a development environment), then you might need to set the SEND_FILE_MAX_AGE_DEFAULT configuration value:
Default cache control max age to use with send_static_file() (the default static file handler) and send_file(), as datetime.timedelta or as seconds. Override this value on a per-file basis using the get_send_file_max_age() hook on Flask or Blueprint, respectively. Defaults to 43200 (12 hours).
Solving this can be as simple as updating the app.config dictionary, like so:
app = Flask(__name__)
...
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0
If you do that, your browser will not cache static assets that are served by Flask.
Use:
if __name__ == '__main__':
app.config['TEMPLATES_AUTO_RELOAD'] = True
app.config['SEND_FILE_MAX_AGE_DEFAULT'] = 0
app.run(debug=True)
You can update using F5 or CTRL + R.
USE CTRL+ f5 to refresh the browser
I have 2 static directories in Flask.
static/
css/
js/
results/
1/
index.html
details.json
2/
index.html
details.json
I followed along a few other answers and glanced through the documentation for serving static files.
app = Flask(__name__)
app.config['RESULT_STATIC_PATH'] = "results/"
#app.route('/results/<path:file>')
def serve_results(file):
# Haven't used the secure way to send files yet
return send_from_directory(app.config['RESULT_STATIC_PATH'], file)
With the following code I can now request for files in the results directory.
I'm more familiar with PHP and Apache. Say if a directory has an index.php or index.html file, it is served automatically.
My requirement:-
Access any file in the results directory
If I send a request for localhost:3333/results/1 I should be served index.html
I can do the same with Flask by adding a route and checking if index.html exists withing the sub directory and then serving it. I found a few more similar pointers here
I currently use two routes to get the functionality. There certainly is a better way.
Why did I include details about the static directory?
It is highly possible that I may be doing something wrong. Please let me know. Thanks :)
Try this:
#app.route('/results/<path:file>', defaults={'file': 'index.html'})
def serve_results(file):
# Haven't used the secure way to send files yet
return send_from_directory(app.config['RESULT_STATIC_PATH'], file)
This is how you pass a default value for that argument. However, I don't agree to this, if you want to serve static content just set a rule in your web server (apache in your case).
I'm using the following single route solution, the "defaults" didn't work for me, as it stopped serving any other file.
Instead of raising the error when the path does not end with a slash, you can also generate a redirect to the path with a slash. Not having this slash but serving the index file with cause problems with relative paths.
#app.route('/<path:path>')
def staticHost(self, path):
try:
return flask.send_from_directory(app.config['RESULT_STATIC_PATH'], path)
except werkzeug.exceptions.NotFound as e:
if path.endswith("/"):
return flask.send_from_directory(app.config['RESULT_STATIC_PATH'], path + "index.html")
raise e
I couldn't find a solution which only used one route. I ended up using this.
#app.route('/results/<path:filename>')
def serve_static(filename):
return send_from_directory("results/", filename)
#app.route('/results/<guid>')
def index(guid):
return send_from_directory("results/" + guid, 'index.html')
Other suggestions
#ipinak suggested using defaults={'file': 'index.html'} as another parameter. That doesn't solve this problem as it results in some security issue which #ipinak has been kind enough to research about.
defaults is a neat option which I'll try using in other places of my application as well.
#dirn suggested in the comments to use Apache/Nginx to serve static files. It would save time as I wouldn't have to implement stuff in Flask. Since static files don't have to passed to the the Python interpreter, we can serve them via another server with hardly any extra work.
I am able to access the static file in question via direct url (localhost:8000/static/maps/foo.txt), so I guess I have it all working well. But I can't do the following: I want to open that text file in views.py. It's because I'm working on a simple web browser adventure game and I wanted to store maps in static/maps and load those maps using f=open('/static/maps/' + mapname + '.txt', 'r'). I get the IOError: no such file or directory. I really don't understand it, because there is such directory when I search for it in address.
Can it be done somehow?
You need to use the place they are stored on disk, which is probably in settings.STATIC_ROOT or settings.STATICFILES_DIRS, not the place they are being served by the web app.
Note however that if you are modifying these files programmatically, they aren't (by definition) static files. You'd be better off using the MEDIA_ROOT location. Also note that Django has helpers to do this sort of thing - see the documentation on Managing files.
I have a form where I upload a file, and I generate a report out of it. The thing is, I would also like to make the report available for download, as an archive. I would like to somehow include the CSS and the JS ( that I inherit from my layout ) inside the report, but I don't really know how to go about this.
So far, I am not storing the file ( the report's being generated from ) on server side, I delete it after I'm done with it.
The only solution I could think of so far, was: from my archive generating view, use urllib to post to the form generating the report, save the response, and just rewrite the links to the stylesheet/JS files.
Is there a simpler way to go about this? Is there a way to keep some files on server side as long as the client's session lives?
Use the HttpResponse in the view you use to show the generated report instead of posting with urllib. If you have something along the lines of
def report_view(request):
...
return render_to_response(request,....)
Then use the response object to create an archive
def report_view(request):
...
archive_link = "/some/nice/url/to/the/archive"
response = render_to_response(request, ... { "archive-link" : archive_link})
store_archive(response)
return response
def store_archive(response):
# here you will need to find css/js files etc
# and bundle them in whatever type of archive you like
# then temporarily store that archive so it can be accessed by the archive_link
# you previously used in your view to allow for downloading
def report_archive_view(request):
# serve the temporarily stored archive, then delete it if you like
You can find all you need to know about HttpResponse in the Django docs.
Although this might work for you I doubt that is what you are really after, maybe what you are really looking for is to generate a pdf report using ReportLab?
you could always keep the files, and have a cron job that deletes files whose session has expired