I created web application in Python, Django framework. Web application gets data from MongoDB database, it gets about 10000 documents from MongoDB database and works very slow. Now looking for ways to speed up my web app. So, is it possible to store result from method render_to_response as HTML temporarily on server? It looks like this:
I have a HTML form; when the user types data in the form and clicks on submit button, web app executes a view that gets data from the Mongo database, and sends that data via variable mongo_data to home.html:
return render_to_response('home.html', {'mongo_data': mongo_data, 'request': request},
context_instance=RequestContext(request))
home.html show data that are stored in variable mongo_data. In the web application, I have a lot of identical queries, for identical queries I get identical result for home.html. So I want to store home.html to folder for example temp on server, and when user types data in the HTML form and clicks the submit button, first to check is if home.html for his data is in temp folder; if it is, then load that home.html, if not then go to view which will generate new home.html with specific mongo_data.
If this is possible , it will speed up my web app a lot ..
The Django caching framework was made exactly for this purpose; see https://docs.djangoproject.com/en/dev/topics/cache/.
In your case, you either cache the entire view for a period of time:
#cache_page(60 * 15)
def my_mongo_view(request):
return render_to_response('home.html', ...)
(From https://docs.djangoproject.com/en/dev/topics/cache/#the-per-view-cache)
or you use the Low-level Cache API (https://docs.djangoproject.com/en/dev/topics/cache/#the-low-level-cache-api):
from django.core.cache import cache
def my_mongo_view(request):
ret = cache.get('home-rendered')
if ret is None:
ret = render_to_response('home.html', ...)
cache.set('home-rendered', ret)
return ret
You'll find more options for caching (e.g. inside your templates) if you just read the doc.
P.S. you can also parametrize your cache by a variable or user ID or something else.
Related
So I'm designing a website with Django that does some heavy scraping based on user input.
This can take up to 5-6 secs and while I am working on cutting that down I would like for some kind of a loader to show up while the backend is scraping.
I have put a loader as you normally would using CSS and JavaScript inside the template but that only pops up when the template is actually loading and not when the view is scraping to gather data for the template.
Tried this in Django:
def scrape(request):
render(request,'loader.html')
*do scraping*
return render(request,'results.html',scraped_data)
The method I would use to solve this is to fire an async function within the scrape view.
#shared_task
def do_the_needful():
return "hello I am doing the needful"
def scrape_result(request, scrape_id):
result = AsyncResult(scrape_id).get()
# convert result to json or some other web format
return result_as_json
def scrape(request):
scrape_request_id = do_the_needful.submit()
return render(request, "scrape.html", context={"scrape_request_id": scrape_request_id}
Then within the HTML you'll need to create some Javascript which will perform Ajax requests to the scrape_result view using the scrape_request_id in the context.
I have a basic view that retrieves some data, renders my page and sends some data to this page:
def myview(request)
one = values.objects.get(user=request.user).address
two = values.objects.get(user=request.user).number
return render(request, "main/mytemplate.html",
context={'address': one, 'numbers': two})
So the values retrieved by those two queries are shown on my page.
Now, on the same page, called mytemplate.html, i'm using another view, which is supposed to handle a form and some other operations:
def secondview(request):
if request.method == 'POST':
if 'button1' in request.POST:
form = MyForm(request.POST)
# check whether it's valid:
if form.is_valid():
profile = form.save(commit=False)
profile.user = request.user
profile.save()
return HttpResponseRedirect(request.path_info)
else:
form = MyForm()
return HttpResponse('it works!')
How can i use the data retrieved by those two queries in the second view? The queries are executed when the page is loaded by the first view. Then, in the same page the second view is used. I want to use the two variables one and two in the second view. Is there a way to do this in Django?
Why don't you make the same queries in the second view? Because i would like the second form to be as fast as possible in terms of reload, without having to do a DB query each time that view is used. Also, since i already retrieved those values when the page is opened, it would be a waste to do that again.
I don't know if this question is clear enough, but the core of it is: can i pass variables/data between two views in django?
You have few options:
Simplest way: include this data in request to the second view (as part of the form data, see an example below). You might even use a single view: if POST was send - store data else do request and show it on a page.
Use cache for that (see an example below) - But I'd recommend to use Django built-in package. Here is a basic example how to use it
Use Django Sessions (see an example below) - it is working option despite of that they have another purpose. When customer is loaded Django will load full session record, so you'll have all data in request.session variable. But that is bad practice: you can get a lot of data duplication and increased database memory consumption.
Use API (e.g. using DjangoRestFramework) together with usual Django app. So you'll just get data you need, and when you need. These API requests can also be cached so it is fast solution.
Yes, you can use session to pass data across views. A session works like a temporary server storage and keeps the needed data in a dictionary form.
For instance, add the following lines to myview:
request.session['one'] = one
request.session['two'] = two
Then, retrieve the data in secondview by referring to the session:
one = request.session['one']
two = request.session['two']
you can use cookies. but if you want more secure your request i suggest to you using redis and the python client for redis
file settings.py
redis = redis.Redis(host='localhost', port=6379, db=0)
file views.py
def view1(request):
redis.set("foo", "boo")
def view2(request):
boo = redis.get("foo")
Why not just saving the results of the two queries as hidden fields in the form rendered by the first template ?
<form ...>
<input type="hidden" id="address" name="address" value="{{address}}">
<input type="hidden" id="numbers" name="numbers" value="{{numbers}}">
...
Then, you can either add 'address' and 'numbers' form fields to MyForm
address = forms.CharField(widget=forms.HiddenInput(), required=False)
...
or just retrieve the values from request.POST
After running the index.html file when we click on login button, the form needs to be displayed for which I have created a login.html file, now how do I link this file in flask? I have put all the html files in the templates folder. Obviously the info added by the user needs to be entered into the MySQL workbench database.
Correct me if I'm wrong, but it sounds like you'd like to have users directed from the index.html page to the login.html page. If so, you could simply add a link to the login page URL within the index page and have Flask handle the connection via a route.
A simple example would be to insert something like the following into your views.py/routes.py file, or your main app file if you are doing a very simple application (in which case you can leave out the import, obviously):
from app import app
#app.route('/path/to/login')
def login()
return render_template('login.html', title='Login')
where app is your Flask application object and /path/to/login is a relative URL appended to your main domain URI.
To make things a bit more maintainable and reusable, you might reference the link to the login page in index.html by using the Jinja2 template engine to automatically populate your anchor tag's href with the desired URL path:
Login
This should get you from index.html to the login.html page with the form.
For more info, check the docs.
I have a question regarding HTML files and Django. Can the django views file serve html files that are not stored locally and stored on amazon s3?
For example instead of:
def index(request):
return render(request, 'index.html')
have something like:
def index(request):
return render(request, 'http://bucket.s3.amazonaws.com/file.html')
Obviously its currently appending the url link to the folder it things html are served from.
Is this sort of thing even possible?
Thanks
I do not believe this is feasible. Web servers exists just for this reason: to serve HTML pages to clients. Amazon S3 exists for file storage, such as images, videos, js files, css files, fonts etc. Not HTML!
On the other hand, suppose this was feasible, your Web server should make a GET request to the HTML stored in S3 (to get the html), then fill it with context variables and then send it to the client (response).
The job of a Django view is to take a Web request and return a Web response. Not making another GET request to get the html which will send as a respose right away (!).
Hope that clarifies your question!
I'm developing a web-app using Flask and pyMongo, and I've recently started to integrate the Flask-Admin module (1.0.4), given the fresh mongodb support.
All is smooth and fine when using ModelViews, but when it comes to subclassing a BaseView I simply can't get it working.
Here is my setup:
user_view = Admin(app, name='User stuff', url="/user", endpoint="user")
class ProfileForm(wtf.Form):
username = wtf.TextField('Username', [wtf.Required()])
name = wtf.TextField('Name', [wtf.Required()])
class Profile(BaseView):
#expose('/', methods=('GET', 'POST'))
def profile(self):
user = User(uid) # gets the user's data from DB
form = ProfileForm(request.form, obj=user)
if form.validate_on_submit():
data = form.data
user.set(**data)
user.save()
flash("Your profile has been saved")
else:
flash("form did not validate on submit")
return self.render('user/profile.html', form=form, data=user)
user_view.add_view(Profile(name='Profile', url='profile'))
When submitting the form, wtforms does not report any error (unless there is any) but the validation does not return to my profile view (the else: branch is always executed)
There is no way I could find to make this work, inspite having thoroughly scanned flask-admin documentation, source code and examples.
Could anybody suggest how I could fix my code, or work around this problem ?
I have suspicion that form is getting submitted using GET method instead of POST or Flask-WTF CSRF check fails.
Here's small gist I made with your sample code. It works as expected: https://gist.github.com/4556210
Few comments:
Template uses some Flask-Admin library functions to render the form. You don't have to use them if you dont want to;
Uses mock user object
Put template under templates/ subdirectory if you want to run the sample.
In either case, Flask-Admin views behave exactly same way like "normal" Flask views, they're just organised differently.