Definitely a beginner question... basically, I want to have the same design from this website. I understand the HTML part, but the problem is that I am not completely sure how to actually render multiple html files in one view.
http://www.htmliseasy.com/frames_tutor/templates/template1.html
My webpage is in this linkk. http://everyology.appspot.com/biology
What I am using is Google app engine. Here is a partial code that is relevant to the question. My concern is with class BioPage... I wrote these and am seeing the frames but getting 404 errors in my webpage.
import os
import webapp2
from string import letters
import jinja2
template_dir = os.path.join(os.path.dirname(__file__), 'templates')
jinja_env = jinja2.Environment(loader = jinja2.FileSystemLoader(template_dir),
autoescape = True)
def render_str(template, **params):
t = jinja_env.get_template(template)
return t.render(params)
class MainHandler(webapp2.RequestHandler):
def write(self, *a, **kw):
self.response.out.write(*a, **kw)
def render_str(self, template, **params):
return render_str(template, **params)
def render(self, template, **kw):
self.write(self.render_str(template, **kw))
class MainPage(MainHandler):
def get(self):
self.render('index.html')
class BioPage(MainHandler):
def get(self):
self.render('biology.html')
self.render('doc1.html')
self.render('doc2.html')
The problem you're running in to is how <frame> tags operate. To fill in frames, browsers make an additional request per frame. Your code is trying to guess ahead, shipping all of the frame contents back in one clump. That's not how it works. You're going to need separate handlers for each of your frames, in addition to the main page. Alternatively, those pages that are completely static can be served static pages.
Related
Hello Stackoverflow Guru's!
I'm a complete newb, and I've got a question that I can't seem to find the answer to (hopefully because it's so simple nobody has bothered to ask).
I'm designing a website that has a bunch of recipes using google app engine. I'd like to be able to render a bunch of the recipe pages using one handler, because I plan of having lots of recipes later and I don't want to have to make a new handler for each one. My code is below:
import urllib2
import webapp2
import jinja2
import os
JINJA_ENVIRONMENT = jinja2.Environment(
loader=jinja2.FileSystemLoader(os.path.dirname(__file__)),
extensions=['jinja2.ext.autoescape'],
autoescape=True)
class Handler(webapp2.RequestHandler):
def write(self, *a, **kw):
self.response.out.write(*a, **kw)
def render_str(self,template,**params):
t = JINJA_ENVIRONMENT.get_template(template)
return t.render(params)
def render(self,template,**kw):
self.write(self.render_str(template,**kw))
class MainHandler(Handler):
def get(self):
template = JINJA_ENVIRONMENT.get_template('main.html')
self.response.write(template.render())
class RecipeHandler(Handler, recipe):
def get(self, recipe):
recipe_pages = {
'carbonara' : 'carbonara.html'
'burger' : 'burger.html'
}
if recipe in recipe_pages:
template = JINJA_ENVIRONMENT.get_template(recipe_pages[recipe])
self.response.write(template.render())
else:
self.abort(404)
app = webapp2.WSGIApplication([
('/', MainHandler),
('/carbonara', RecipeHandler(carbonara)),
('/burger',RecipeHandler(burger)),
], debug=True)
I basically want to avoid writing out a "CarbonaraHander" and "BurgerHandler", and just use "RecipeHandler" to render both pages. I know this should be possible, but I have no idea how to do it.
Any help is appreciated!
Edit: I think I should be using something called regular expressions? But I don't really understand how they need to be used in this case.
AFAIK you can't pass args to the handler, you need to extract them from the request. This is what I'd do (pushed it a bit further to directly use the template name in the URl routing):
class RecipeHandler(Handler):
def extract_template_name_from_request(self):
return self.request.path_info[9:] # strip leading '/recipes/' (or whatever else you need)
def get(self):
template_name = self.extract_template_name_from_request()
try:
template = JINJA_ENVIRONMENT.get_template(template_name)
except Exception:
# can't locate a template matching the requested path
self.abort(404)
return
# prepare the template values as needed
values = {'recipe': {'name': template_name[:-5]}} # just an example
try:
self.response.write(template.render(values))
except Exception:
# failure rendering the template
self.abort(500)
app = webapp2.WSGIApplication([
('/recipes/.*.html', RecipeHandler), # see extract_template_name_from_request()
('/.*', MainHandler),
], debug=True)
I am working on creating a Django-based scraper in which a user can enter a search term. I use that search term(s) to build a URL and query the site, then returning un-rendered HTML and JS. I am then able to take the post request, render the page by creating a Qwebpage, passing it the URL and grabbing the frame's rendered HTML. This works one time in my Django app, and the next POST request crashes the site.
My first concern is that in this current set up, I am forced to use the xvfb-run wrapper to run. Is this going to pose an issue when I deploy - better question is: can I use an xvfb wrapper in production somehow?
With that said I am able to make one post request and this returns the page that I am looking for. If I hit back, and send another request, I see the following errors in console, and this then shuts down the ./manage.py server:
WARNING: QApplication was not created in the main() thread.
QObject::connect: Cannot connect (null)::configurationAdded(QNetworkConfiguration) to QNetworkConfigurationManager::configurationAdded(QNetworkConfiguration)
QObject::connect: Cannot connect (null)::configurationRemoved(QNetworkConfiguration) to QNetworkConfigurationManager::configurationRemoved(QNetworkConfiguration)
QObject::connect: Cannot connect (null)::configurationChanged(QNetworkConfiguration) to QNetworkConfigurationManager::configurationChanged(QNetworkConfiguration)
QObject::connect: Cannot connect (null)::onlineStateChanged(bool) to QNetworkConfigurationManager::onlineStateChanged(bool)
QObject::connect: Cannot connect (null)::configurationUpdateComplete() to QNetworkConfigurationManager::updateCompleted()
Segmentation fault (core dumped)
I will admit that I do not understand what in particular the error is here since I'm rather new to threading concepts. I am uncertain if this error means that it can't reconnect to the xvfb wrapper thats already running, or if indeed it is a threading issue. The code that works once is here. This has been changed slightly since I don't want to show the site I'm actually scraping. Also, I am not hunting for data in this sample. This sample will simply bring rendered HTML to your browser as a test:
import sys
from django.shortcuts import render
# Create your views here.
from django.http import HttpResponse
from django.http import HttpResponseRedirect
from django.views.generic import View
from PyQt4.QtGui import *
from PyQt4.QtCore import *
from PyQt4.QtWebKit import *
from bs4 import BeautifulSoup
from .forms import QueryForm
def query(request):
results = google.search("Real Estate")
context = {'results': results}
return render(request, 'searchlistings/search.html', context)
class Render(QWebPage):
def __init__(self, url):
self.app = QApplication(sys.argv)
QWebPage.__init__(self)
self.loadFinished.connect(self._loadFinished)
self.mainFrame().load(QUrl(url))
self.app.exec_()
def _loadFinished(self, result):
self.frame = self.mainFrame()
self.app.quit()
class SearchView(View):
form_class = QueryForm
template_name = 'searchlistings/index.html'
def get(self, request, *args, **kwargs):
form = self.form_class()
return render(request, self.template_name, {'form': form})
def post(self, request, *args, **kwargs):
form = self.form_class(request.POST)
if form.is_valid():
query = form.cleaned_data['query']
context = self.isOnSite(query)
#return context
#return render(request, 'searchlistings/search.html', {'context': context})
return HttpResponse(context)
def isOnSite(self, query):
url = "http://google.com"
#This does the magic.Loads everything
r = Render(url)
#result is a QString.
result = r.frame.toHtml()
r.app.quit()
return result;
So my primary questions are this:
Is XVFB wrapper appropriate here and can I use this set up in production on a different host. Will this work not on my local vagrant box?
The main() thread issue - is this a threading issue or an issue not connecting back to the xvfb server? Can this issue be resolved with Celery or something similar?
Is this an appropriate way to do what I want? I've seen lots of other solutions including scrapyjs, spynner, selenium and so on but they seem either overtly complicated or based on QT. A better question is do any of these alternative packages solve the main() thread issue?
Thanks for your help!
OK the solution here was to use twill as documented here http://twill.idyll.org/python-api.html - I am able to run this without the xvfb wrapper and it is much faster than previous methods with much less overhead. I can recommend this.
I'm building an Web site using Python and Flask and I want the website to run the same way as GrooveShark and Twitter do : having a masterPage with a $.ajax call that append the content of another .html file into a of the masterPage.
For the moment, my masterPage contains a $.ajax that calls my WebServer passing the current window.location.href
I would like to know how to catch all Url in a Python/Flask WebServer and then, on certain condition, redirect without begin recatched by the method !
E.g.
Based on http://flask.pocoo.org/snippets/57/
#app.route('/', defaults={'p_path': ''})
#app.route('/<path:p_path>')
def CatchAll(p_path):
if(p_path.startswith("page/")):
#if (route exist...I don't know yet how to do this part...):
#Call the method routed to the /Page/ route
# Without being re-catched by the CatchAll method
elif (p_path.startswith("api/")):
#Run a method on my API that will call my controller > DAL > Database
# Without being re-catched by the CatchAll method
else:
#I render my masterpage
return render_template("masterPage.html")
So if the website is loaded for the first time in the browser, it will load the MasterPage, which contains the ajax call that will recall the WebServer passing a URL like "/Page/Contact" which will render the html which I will append in my <div>
Do you have any idea of how doing this ?
It's my first draft so feel free to give me comments (optimization, best pratice, performance) because I'm new to Python/Flask
You do not need catch all requests for your case.
You can set main page access by GET request and ajax by POST, for example with next decorator:
class AjaxApp(Flask or Blueprint):
def ajax_route(self, rule, **options):
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
if request.method == 'POST':
return f(*args, **kwargs)
return render_template("masterPage.html")
endpoint = options.pop('endpoint', None)
methods = options.pop('methods', ['GET', 'POST'])
self.add_url_rule(rule, endpoint, wrapper, methods=methods, **options)
return wrapper
return decorator
You also can just check X-Requested-With header if do not want restrict access for each methods:
class AjaxApp(Flask or Blueprint):
def ajax_route(self, rule, **options):
def decorator(f):
#functools.wraps(f)
def wrapper(*args, **kwargs):
if request.headers.get('X-Requested-With') == 'XMLHttpRequest':
return f(*args, **kwargs)
return render_template("masterPage.html")
endpoint = options.pop('endpoint', None)
self.add_url_rule(rule, endpoint, wrapper, **options)
return wrapper
return decorator
This methods very implicit and you should use just ajax_route for your special ajax pages, but it increase complexity and probably better use Pluggable Views http://flask.pocoo.org/docs/views/.
If you want group logic by api, pages and other you can look at blueprints http://flask.pocoo.org/docs/blueprints/:
class AjaxBlueprint(Blueprint):
def route(self, rule, **options):
# one of examples
ajax = AjaxBlueprint('ajax', __name__)
#ajax.route('/main.html')
def main():
return render_template('main.html')
app.register_blueprint(ajax)
and override route with one of examples method for blueprint. So ajax bluprint in this case will have custom logic, others blueprint will have independent.
I want a menu on my web site.
I want this menu to be editable via django admin. So, I need to create a model for it.
Once I've created model for my menu, I can use it in my views.py:
def main(request):
menu_items = MenuItem.objects.all()
return direct_to_template(request, 'main.html', {'menu_items': menu_items})
It is ok. But... wait a minute...
If I have several pages, it will look like:
def main(request):
menu_items = MenuItem.objects.all()
return direct_to_template(request, 'main.html', {'menu_items': menu_items})
def page1(request):
menu_items = MenuItem.objects.all()
return direct_to_template(request, 'page1.html', {'menu_items': menu_items})
def page2(request):
menu_items = MenuItem.objects.all()
return direct_to_template(request, 'page2.html', {'menu_items': menu_items})`
Is there any way not to write the same code everytime I create new function?
Yep, create a context processor and your menu will be available on every page
Inside one of your apps create a file menu_context_processor.py (or similar)
from myapp.models import MenuItem
def menu(request):
return {'menu': MenuItem.objects.all() }
and add it to your TEMPLATE_CONTEXT_PROCESSOR setting in settings.py
TEMPLATE_CONTEXT_PROCESSORS = (
...,
'myapp.menu_context_processor.menu'
)
and now, in every template, you will have access to {{ menu }}. These are used to make {{ STATIC_URL }} and `{{ MEDIA_URL }}' available to you throughout your application and are useful for variables you want to include everywhere.
Ok, Timmy suggested you a Django way and it works great. By the way, I know a pythonic way that you can use decorators like this:
def apply_menu(func):
def wrapper(*args, **kwargs):
request = args[0]
menu_items = MenuItem.objects.all()
vars = {'menu_items': menu_items}
if kwargs:
page, template_var = func(*args, **kwargs)
else: page, template_var = func(*args)
vars.update( template_var)
return return direct_to_template(request, page, vars)
return wrapper
And then use the decorator this way:
#apply_menu
def main(request):
your_extra_vars = {}
return 'main.html', your_extra_vars
On all my template rendering for a particular app, the output ends with None:
...</html>None
This must be a bug and probably in my code and I've spent days trying to track it down. There's nothing special about my app and this bug appears on every page I use template rendering, whether I use a seperate template engine or not. There is nothing special about my code:
class Objectives(NewBaseHandler):
#user_required
def get(self):
user = auth_models.User.get_by_id(long(self.auth.get_user_by_session()['user_id']))
if user:
self.render_template('objectives.html', {'user': user})
else:
self.render_template('/', {})
class NewBaseHandler(BaseHandler):
"""
........BaseHandler for all requests
........Holds the auth and session properties so they are reachable for all requests
...."""
def dispatch(self):
"""
............Save the sessions for preservation across requests
........"""
# self.session_store = sessions.get_store(request=self.request)
# if self.request.host.find('localhost') > 0: # for a Swedish domain that uses Swedish
# or lang = os.environ.get("HTTP_ACCEPT_LANGUAGE")
i18n.get_i18n().set_locale('sv')
lang_code_get = self.request.get('hl', None)
if lang_code_get:
#self.session['i18n_language'] = lang_code_get
i18n.get_i18n().set_locale(lang_code_get)
try:
response = super(NewBaseHandler, self).dispatch()
self.response.write(response)
finally:
self.session_store.save_sessions(self.response)
#webapp2.cached_property
def auth(self):
return auth.get_auth()
#webapp2.cached_property
def session_store(self):
return sessions.get_store(request=self.request)
#webapp2.cached_property
def auth_config(self):
"""
............Dict to hold urls for login/logout
........"""
return {'login_url': self.uri_for('login'),
'logout_url': self.uri_for('logout')}
class BaseHandler(webapp2.RequestHandler):
#webapp2.cached_property
def jinja2(self):
return jinja2.get_jinja2(app=self.app)
def render_template(self, file, template_args):
path = os.path.join(os.path.dirname(__file__), 'templates',
file)
self.response.out.write(template.render(path, template_args))
def render_jinja(self, filename, **template_args):
self.response.write(self.jinja2.render_template(filename,
**template_args))
How can I check where the output None is coming from? It's probably not coming from the template and it doesn't seem to be coming from the handlers and there is no other code.
Thank you
In Objectives.get() you must return a value. Since you don't do this Python assumes the result is None. This value you get in NewBaseHandler.dispatch() when calling to base dispatch implementation and then write it to response.
If I get your app correctly returning empty string in get method will solve the problem.