Related
I'm new in python and web-crawling. I'm doing a few exercises crawling some web-sites and seems great with beautifulsoup. But recently, while I was crawling a Persian site (https://video.varzesh3.com) with the code below, I am receiving weird characters. I have done the same procedure on other Persian websites an I believe the problem is not with the encoding. This is my code:
url = 'https://video.varzesh3.com'
source_code = requests.get(url)
plain_text = source_code.text
soup = BeautifulSoup(plain_text, "html.parser")
print(soup)
And this is a part of the result:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="IE=edge" http-equiv="X-UA-Compatible"/>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<link href="/favicon.ico" rel="shortcut icon" type="image/x-icon"/>
<title>ÙÛدÛÙ Ùرزش س٠| خاÙÙ</title>
<meta content="sport ,varzesh ,football, soccer,livescores, live score, livescore, iran,football3,Daily soccer news , broadcast ,ÙÙتبا٠س٠, ÙتاÛج زÙد٠, Ø®ÙÛج Ùارس , perian gulf , ÙÛÚ¯ آزادگا٠, ÙÙرÙاÙا٠آسÛا , ÙÙرÙاÙا٠ارÙپا , ÙÛÚ¯ برتر , جا٠ØØ°ÙÛ , شبک٠س٠, Ùرزش , ÙÙتبا٠برتر , اÛرا٠, جا٠جÙاÙÛ , جا٠جÙاÙÛ 2010,ÙÙتبا٠3 ," name="keywords">
<meta content="پاÙگا٠ÙÛدÛÙ ÙØ±Ø²Ø´Û Ø¨Ø±Ø§Ù Ùارس٠زباÙا٠ÙÙ ÙÛدÛÙ ØÙز٠Ùرزش (ÙÙتباÙØÙاÙÙبا٠ØبسÙتبا٠Ù...) را ارائ٠ÙÛ Ú©Ùد" name="description">
<link href="/Static/css/frontend.min.css?v=9" rel="stylesheet" type="text/css"/>
<link href="https://static2.farakav.com/v3/static/css/fonts.css?version=6" rel="stylesheet" type="text/css"/>
<link href="https://static2.farakav.com/varzesh3/assets/font/varzesh3-icon/varzesh3.min.css" rel="stylesheet" type="text/css"/>
<script src="https://static2.farakav.com/football3_jscripts/jquery-1.8.0.min.js" type="text/javascript"></script>
<script src="/Static/js/jquery.cookie.js" type="text/javascript"></script>
<script src="/Static/js/mustache.js" type="text/javascript"></script>
<script type="text/javascript">
now = new Date();
var head = document.getElementsByTagName('head')[0];
var script = document.createElement('script');
script.type = 'text/javascript';
var script_address = 'https://cdn.yektanet.com/js/varzesh3.com/article.v1.min.js';
script.src = script_address + '?v=' + now.getFullYear().toString() + '0' + now.getMonth() + '0' + now.getDate() + '0' + now.getHours();
head.appendChild(script);
</script>
Why do I get this weird like "پاÙگا٠ÙÛدÛÙ ÙØ±Ø²Ø´Û Ø¨Ø±Ø§Ù Ùارس" characters?
You get those wired characters because of it's encoding.
>>> source_code.encoding
'ISO-8859-1'
Try this, set encoding to UTF-8
>>> source_code.encoding = 'UTF-8'
>>> plain_text = source_code.text
>>> BeautifulSoup(plain_text, "html.parser")
Output:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8"/>
<meta content="IE=edge" http-equiv="X-UA-Compatible"/>
<meta content="width=device-width, initial-scale=1" name="viewport"/>
<link href="/favicon.ico" rel="shortcut icon" type="image/x-icon"/>
<title>ویدیو ورزش سه | خانه</title>
<meta content="sport ,varzesh ,football, soccer,livescores, live score, livescore, iran,football3,Daily soccer news , broadcast ,فوتبال سه , نتایج زنده , خلیج فارس , perian gulf , لیگ آزادگان , قهرمانان آسیا , قهرمانان اروپا , لیگ برتر , جام حذفی , شبکه سه , ورزش , فوتبال برتر , ایران , جام جهانی , جام جهانی 2010,فوتبال 3 ," name="keywords">
<meta content="پايگاه ویدیو ورزشی براي فارسي زبانان كه ویدیو حوزه ورزش (فوتبال،واليبال ،بسكتبال و...) را ارائه می کند" name="description">
<link href="/Static/css/frontend.min.css?v=9" rel="stylesheet" type="text/css"/>
....
...
..
I am trying to simply load my /plot/ webpage from the code below, however, the bokeh plot is not being embedded in the webpage. Note, in newer versions of bokeh, it does not require a css file (from the CDN_css).
When using the output_file and show methods, I can easily get a separate webpage to spawn a html of the bokeh plot. However, when attempting to embed it, I merely get the html code from index, without the embedded chart.
Any help would be greatly appreciated!
'''python code: app.py'''
app = Flask(__name__)
def create_plot(ticker):
global p
ts = TimeSeries(key="**********", output_format='pandas')
data, meta_data = ts.get_intraday(symbol=ticker,interval='1min', outputsize='full')
data = data.iloc[::-1]
#gets the #tsla data from the API and orders it in reverese (5 days from most recent entry to furthest)
data["datetime"] = data.index #adds a datetime column of the index strings (need to convert to datetime objects)
datetime_list = []
for i in data["datetime"]:
datetime_list.append(datetime.strptime(i, "%Y-%m-%d %H:%M:%S"))
data["datetime"] = datetime_list #converts the datetime column from a str to a datetime object column
my_data = pd.DataFrame()
my_data["datetime"] = data["datetime"]
my_data["stock_price"] = data["1. open"]
last_time = my_data["datetime"][0]
one_day_ago = last_time - relativedelta(days=1)
daily_data = my_data[my_data["datetime"] > one_day_ago]
#bokeh chart
p = figure(x_axis_type="datetime",plot_width=1000, plot_height=400)
p.sizing_mode="scale_width"
p.title.text="# " + ticker + " Stock Price"
p.title.text_color="navy"
p.title.text_font="times"
p.title.text_font_style="bold"
p.title.text_font_size="28pt"
p.title.vertical_align="top"
p.title.align="center"
p.xaxis.minor_tick_line_color=None
p.yaxis.minor_tick_line_color=None
p.xaxis.axis_label="Time"
p.yaxis.axis_label="Price USD"
p.xaxis.axis_label_text_color="navy"
p.yaxis.axis_label_text_color="navy"
p.xaxis.axis_label_text_font_style="bold"
p.yaxis.axis_label_text_font_style="bold"
p.xaxis.axis_label_text_font="times"
p.yaxis.axis_label_text_font="times"
p.xaxis.axis_label_text_font_size = "16pt"
p.yaxis.axis_label_text_font_size = "16pt"
p.ygrid.band_fill_alpha = 0.1
p.ygrid.band_fill_color = "navy"
p.xgrid.grid_line_color = None
p.line(daily_data["datetime"], daily_data["stock_price"], line_width=2, color="blue")
#app.route("/")
def home():
return render_template("home.html")
#app.route("/plot/")
def plot():
create_plot("TSLA")
script1,div1 = components(p) # script_1 is the js source code, div_1 is the html source code
cdn_js = CDN.js_files[0] #need first element of this list
return render_template("plot.html",script1=script1, div1=div1, cdn_js=cdn_js)
if __name__ == "__main__":
app.run(debug = True)
'''home.html code'''
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
<meta charset="utf-8">
<link rel="stylesheet" href="../static/css/main.css">
<title>Daniel's #tsla chart webapp</title>
</head>
<body>
<h1>Daniel's #TSLA chart webapp</h1>
<h2>Hi, welcome to my stock market webapp which gives an analysis of #TSLA</h2>
<p>To proceed, please press the button below</p>
<form class="input" action="plot.html" method="post">
Please enter your email address below to access the #TSLA market data! <br>
<input type="submit" name="name_submission" value="submit">
</form>
</body>
</html>
'''plot.html code'''
<!DOCTYPE html>
<html lang="en" dir="ltr">
<head>
<meta charset="utf-8">
<link rel="stylesheet" href="../static/css/main.css">
<script type="text/javascript" src={{CDN_js | safe}}></script>
<title>Daniel's #tsla chart webapp</title>
</head>
<body>
<h1>Daniel's #TSLA chart webapp</h1>
<h2>Hi, welcome to my stock market webapp which gives an analysis of #TSLA</h2>
{{ script1|safe }}
{{ div1|safe }}
</body>
</html>
I have written a Python program for translation with Transcrypt to Javascript.
I can not get the addEventListener function to work. Any ideas?
Here is the code as dom7.py:
class TestSystem:
def __init__ (self):
self.text = 'Hello, DOM!'
self.para = 'A new paragraph'
self.textblock = 'This is an expandable text block.'
self.button1 = document.getElementById("button1")
self.button1.addEventListener('mousedown', self.pressed)
def insert(self):,
document.querySelector('output').innerText = self.text
# document.querySelector('test').innerText = "Test"+self.button1+":"
def pressed(self):
container = document.getElementById('textblock')
newElm = document.createElement('p')
newElm.innerText = self.para
container.appendChild(newElm)
testSystem = TestSystem()
And here follows the corresponding dom7.html for it:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script src="__javascript__/dom7.js"></script>
<title>Titel</title>
</head>
<body onload=dom7.testSystem.insert()>
<button id="button1">Click me</button><br>
<main>
<h1>DOM examples</h1>
<p>Testing DOM</p>
<p>
<output></output>
</p>
<p>
<test>Test String:</test>
</p>
<div id="textblock">
<p>This is an expandable text block.</p>
</div>
</main>
</body>
</html>
The problem is that your TestSystem constructor is called before the DOM tree is ready. There are three ways to deal with this, the last of which is the best.
The first way is to include your script after you populated your body:
class TestSystem:
def __init__ (self):
self.text = 'Hello, DOM!'
self.para = 'A new paragraph'
self.textblock = 'This is an expandable text block.'
self.button1 = document.getElementById("button1")
self.button1.addEventListener('mousedown', self.pressed)
def insert(self):
document.querySelector('output').innerText = self.text
# document.querySelector('test').innerText = "Test"+self.button1+":"
def pressed(self):
container = document.getElementById('textblock')
newElm = document.createElement('p')
newElm.innerText = self.para
container.appendChild(newElm)
testSystem = TestSystem()
and:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<title>Titel</title>
</head>
<body onload=dom7.testSystem.insert()>
<button id="button1">Click me</button><br>
<main>
<h1>DOM examples</h1>
<p>
Testing DOM
</p>
<p>
<output></output>
</p>
<p>
<test>Test String:</test>
</p>
<div id="textblock">
<p>This is an expandable text block.</p>
</div>
<script src="__javascript__/dom7.js"></script>
</main>
</body>
</html>
Still your insert function may be called too early, so may not work.
The second way is to include the script at the beginning and call an initialization function to connect event handlers to the DOM:
class TestSystem:
def __init__ (self):
self.text = 'Hello, DOM!'
self.para = 'A new paragraph'
self.textblock = 'This is an expandable text block.'
self.button1 = document.getElementById("button1")
self.button1.addEventListener('mousedown', self.pressed)
def insert(self):
document.querySelector('output').innerText = self.text
# document.querySelector('test').innerText = "Test"+self.button1+":"
def pressed(self):
container = document.getElementById('textblock')
newElm = document.createElement('p')
newElm.innerText = self.para
container.appendChild(newElm)
def init ():
testSystem = TestSystem()
and:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script src="__javascript__/dom7.js"></script>
<title>Titel</title>
</head>
<body onload=dom7.testSystem.insert()>
<button id="button1">Click me</button><br>
<main>
<h1>DOM examples</h1>
<p>
Testing DOM
</p>
<p>
<output></output>
</p>
<p>
<test>Test String:</test>
</p>
<div id="textblock">
<p>This is an expandable text block.</p>
</div>
<script>dom7.init ();</script>
</main>
</body>
</html>
Still there is a possibility that some browsers call the initialization function before the page is loaded, although this is rare. In addition to this the insert method is again called too early.
Third and best way, to solve both problems, is to run your initialization after a page load event, and call insert after you create your testSystem, so e.g. in the initalization function:
class TestSystem:
def __init__ (self):
self.text = 'Hello, DOM!'
self.para = 'A new paragraph'
self.textblock = 'This is an expandable text block.'
self.button1 = document.getElementById("button1")
self.button1.addEventListener('mousedown', self.pressed)
def insert(self):
document.querySelector('output').innerHTML = self.text
# document.querySelector('test').innerText = "Test"+self.button1+":"
def pressed(self):
container = document.getElementById('textblock')
newElm = document.createElement('p')
newElm.innerText = self.para
container.appendChild(newElm)
def init ():
testSystem = TestSystem()
testSystem.insert ()
and:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script src="__javascript__/dom7.js"></script>
<title>Titel</title>
</head>
<body onload="dom7.init ();">
<button id="button1">Click me</button><br>
<main>
<h1>DOM examples</h1>
<p>
Testing DOM
</p>
<p>
<output></output>
</p>
<p>
<test>Test String:</test>
</p>
<div id="textblock">
<p>This is an expandable text block.</p>
</div>
</main>
</body>
</html>
I looked at your mondrian example in the tutorial section and I saw that there is also a very simple way to attach the addEventListener to a document after it has loaded. You can use the DOMContentLoaded attribute in the header of the html doc for doing so:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="utf-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script src="__javascript__/addEventListener_example1.js"></script>
<script>document.addEventListener("DOMContentLoaded", addEventListener_example1.init)</script>
<title>Titel</title>
</head>
<body>
<button id="button1">Click me</button><br>
<main>
<h1>DOM examples</h1>
<p>
Testing DOM
</p>
<p>
<output></output>
</p>
<p>
<test>Test String:</test>
</p>
<div id="textblock">
<p>This is an expandable text block.</p>
</div>
</main>
</body>
</html>
and the code for addEventListener_example1.py would be:
#!/usr/bin/env python
# -*- coding: utf-8 -*-
def init():
insert()
def insert():
document.querySelector('output').innerHTML = 'Hello, DOM!'
button1 = document.getElementById("button1")
button1.addEventListener('mousedown', pressed)
def pressed():
para = 'A new paragraph'
container = document.getElementById('textblock')
newElm = document.createElement('p')
newElm.innerText = para
container.appendChild(newElm)
So i got this html from a website:
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xmlns:fb="http://www.facebook.com/2008/fbml" dir="rtl" lang="he" id="vbulletin_html">
<head>
<meta http-equiv="Content-Type" content="text/html; charset=UTF-8" />
<meta id="e_vb_meta_bburl" name="vb_meta_bburl" content="http://www.fxp.co.il" />
<meta http-equiv="X-UA-Compatible" content="IE=9"/>
<base href="//www.fxp.co.il" /><!--[if IE]></base><![endif]-->
<link rel="canonical" href="http://www.fxp.co.il/login.php?do=login" />
<link rel="shortcut icon" href="//images.fxp.co.il/images3/fav.png">
<meta name="generator" content="vBulletin 4.2.2" />
<meta name="keywords" content="FXP,פורום,פורומים,fxp,משחקים,סרטים,כיף,רשת,מחשבים,הורדות,הורדה,סרגל כלים,בדיקת IP,העלאת תמונות" />
<meta name="description" content="מחפשים אתר פורומים ? אתר FXP מכיל קהילות פורומים, משחקים, תמונות גולשים ועוד. הכנסו עכשיו אל קהילות הפורומים של FXP" />
<meta property="fb:app_id" content="415294715208536" />
<meta property="og:site_name" content="FXP" />
<meta property="og:description" content="מחפשים אתר פורומים ? אתר FXP מכיל קהילות פורומים, משחקים, תמונות גולשים ועוד. הכנסו עכשיו אל קהילות הפורומים של FXP" />
<meta property="og:url" content="http://www.fxp.co.il" />
<meta property="og:type" content="website" />
<link rel="stylesheet" type="text/css" href="//images.fxp.co.il/css_static_main/main_fxp_20.2.14.css?v=7.11" />
<link href="//www.fxp.co.il/clientscript/awesome/css/font-awesome.min.css" rel="stylesheet">
<script type="text/javascript" src="//images.fxp.co.il/clientscript/yui-2.9.js"></script>
<script type="text/javascript">
<!--
var SESSIONURL = "";
var SECURITYTOKEN = "1456672267-7067c7f37055c9dd77a4fa83ba3b7b6f316c82b1";
var IMGDIR_MISC = "//images.fxp.co.il/images_new/misc";
var IMGDIR_BUTTON = "//images.fxp.co.il/images_new/buttons";
var vb_disable_ajax = parseInt("0", 10);
var SIMPLEVERSION = "4116";
var BBURL = "http://www.fxp.co.il";
var LOGGEDIN = 1152224 > 0 ? true : false;
var THIS_SCRIPT = "login";
var RELPATH = "login.php?do=login";
var PATHS = {
forum : "",
cms : "",
blog : ""
};
var AJAXBASEURL = "http://www.fxp.co.il/";
//var AJAXBASEURL = "//www.fxp.co.il/";
// -->
</script>
<script type="text/javascript" src="//images.fxp.co.il/clientscript/vbulletin-core.js"></script>
<script type="text/javascript" src="//ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js"></script>
<script type="text/javascript" src="//images.fxp.co.il/css_static_main/jquery.cookie_new.js"></script>
<script type="text/javascript">
$(document).ready(function(){
$("#ajax").load('notifc.php?userid=1152224');
$("#noti").click(function () {
$("#ajax").load('notifc.php?userid=1152224');
});
$("#ajax_likes").load('likesno.php?userid=1152224');
$("#noti_like").click(function () {
$("#ajax_likes").load('likesno.php?userid=1152224');
});
});
</script>
<script type="text/javascript" src="//images.fxp.co.il/clientscript/set.js?v=6.5"></script>
<script type="text/javascript" src="//images.fxp.co.il/clientscript/lazyload.js"></script>
<script type="text/javascript">
$(function() {
if (getCookie_bar('bbc_lazyload_fxp') != '1') {
$(".postbody img").lazyload({placeholder : "clear.gif", effect: "fadeIn"});
}
});
</script>
<script type="text/javascript" src="//images.fxp.co.il/advertising/ads.js"></script>
<script type="text/javascript" src="//images.fxp.co.il/skinfxp/s.php"></script>
<title>FXP</title>
<script type="text/javascript">
var forumname = "";
var fxpcategory = "none";
</script>
<script type='text/javascript'>DY = {scsec : 8765235 ,API: function(){(DY.API.actions = DY.API.actions || []).push(arguments)}};</script>
<script type='text/javascript' src='//dy2.ynet.co.il/scripts/8765235/api_static.js'></script>
<script type='text/javascript' src='//dy2.ynet.co.il/scripts/8765235/api_dynamic.js'></script>
<script type="text/javascript">
(function(i,s,o,g,r,a,m){i['GoogleAnalyticsObject']=r;i[r]=i[r]||function(){
(i[r].q=i[r].q||[]).push(arguments)},i[r].l=1*new Date();a=s.createElement(o),
m=s.getElementsByTagName(o)[0];a.async=1;a.src=g;m.parentNode.insertBefore(a,m)
})(window,document,'script','//www.google-analytics.com/analytics.js','ga');
ga('create', 'UA-598971-1', 'auto');
ga('require', 'displayfeatures');
ga('send', 'pageview');
</script>
<script type="text/javascript">
ga('set', 'dimension1', 'Registered');
</script>
</head>
<body>
<div class="standard_error">
<form class="block vbform" method="post" action="http://www.fxp.co.il/" name="postvarform">
<h2 class="blockhead">מעביר...</h2>
<div class="blockbody formcontrols">
<p class="blockrow restore">התחברת בהצלחה, Copy_Pasta.</p>
</div>
<div class="blockfoot actionbuttons redirect_button">
<div class="group" id="redirect_button">
לחץ כאן אם הדפדפן אינו מעביר אותך אוטומטית
</div>
</div>
</form>
</div>
<noscript>
<meta http-equiv="Refresh" content="2; URL=http://www.fxp.co.il/" />
</noscript>
<script type="text/javascript">
<!--
function exec_refresh()
{
window.status = "מעביר..." + myvar;
myvar = myvar + " .";
var timerID = setTimeout("exec_refresh();", 100);
if (timeout > 0)
{
timeout -= 1;
}
else
{
clearTimeout(timerID);
window.status = "";
window.location = "http://www.fxp.co.il/";
}
}
var myvar = "";
var timeout = 20;
exec_refresh();
//-->
</script>
</body>
</html>
The page above is supposed to redirect you to another page.
I'm looking to click on the button that redirects you immediately if you press it.
It's this button if anyone's wondering
This is the relevant code :
def login_and_redirect():
#login into the site
usrnm=raw_input("Please enter your username: ")
pswrd=raw_input("Please enter your password: ")
print("Logging in, please stand by...")
driver.find_element_by_id("navbar_username").send_keys(usrnm)
driver.find_element_by_id("navbar_password_hint").click()
driver.find_element_by_id("navbar_password").send_keys(pswrd)
driver.find_element_by_id("navbar_password").submit()
#redirect to another page in the site after logging in
driver.get("http://www.fxp.co.il/forumdisplay.php?f=236")
for some weird reason the function is stuck and phantomJS isn't beign redirected to the URL I wanted it to. My guess is it's because of the redirection page, so i'm trying to get around it by clicking the button. i'd appreciate any help, thanks :)
You can try below xpaths to click on that re-direct button
//div[#id='redirect_button']/a
or
//a[#href='http://www.fxp.co.il/']
Of-course you can also use css selector here..
Thanks,
Murali
I'm trying to develop an app for Google App Engine using Cherrypy.
I'm using a custom tool to apply templates with. Part of the output is a html css stylesheet declaration. I can get cherrypy to find and load the css file no problem, the problem is the template gets applied to the css output.
Here's my code I'm using
import cherrypy
from lib.tools.template import HamlTool
engine = cherrypy.engine
from lib.plugin.template import MakoTemplatePlugin
engine.mako = MakoTemplatePlugin(engine, os.path.join(os.path.dirname(__file__), 'templates'))
engine.mako.subscribe()
class Root(object):
exposed = True
#tools.encode()
#tools.template(template='index.haml')
def GET(self):
return 'Hello I like CherryPy!'
cherrypy.tools.template = HamlTool()
conf = {
'/': {
'request.dispatch': cherrypy.dispatch.MethodDispatcher(),
'tools.sessions.on': True,
},
'/assets/css/bootstrap-responsive.min.css' : {
'tools.staticfile.on': True,
'tools.staticfile.filename' : os.path.join(os.path.dirname(__file__), 'assets/css/bootstrap-responsive.min.css')
}
}
root = Root()
app = cherrypy.tree.mount(root, '/', config=conf)
run_wsgi_app(app)
This is the HamlTool code
import cherrypy
import haml
import mako.lookup
class HamlTool(cherrypy.Tool):
def __init__(self):
cherrypy.Tool.__init__(self, 'before_finalize',
self._render,
priority=0)
def _render(self, template=None):
if cherrypy.response.status > 399:
return
# retrieve the data returned by the handler
data = {'data':''.join(cherrypy.response.body), 'title': 'Hello there'} or {}
try:
template = cherrypy.engine.publish("lookup-template", template).pop()
except:
template = None
print
print 'template: ', template
if template:
# dump the template using the dictionary
cherrypy.response.body = str(template.render(**data))
And the MakoTemplatePlugin code
from cherrypy.process import plugins
import mako.lookup
import haml
class MakoTemplatePlugin(plugins.SimplePlugin):
def __init__(self, engine, template_dir='templates'):
super(MakoTemplatePlugin, self).__init__(engine)
self.template_dir = template_dir
print 'MakoTemplatePlugin: init', self.template_dir
self.lookup = mako.lookup.TemplateLookup(
directories=self.template_dir,
preprocessor=haml.preprocessor
)
self.bus.subscribe('lookup-template', self.get_template)
def start(self):
print 'MakoTemplatePlugin: start', self.template_dir
self.lookup = mako.lookup.TemplateLookup(
directories=self.template_dir,
preprocessor=haml.preprocessor
)
self.bus.subscribe('lookup-template', self.get_template)
def stop(self):
self.bus.unsubscribe('lookup-template', self.get_template)
self.lookup = None
def get_template(self, name):
print 'MakoTemplatePlugin: lookup template', name, self.template_dir
return self.lookup.get_template(name)
The base template
!!! 5
%html(lang="en")
%head
%title= title
%meta(name="viewport", content="width=device-width, initial-scale=1.0")
%link(href="/assets/css/bootstrap-responsive.min.css", rel="stylesheet")
%body
.container-fluid=self.body()
%script(src="http://code.jquery.com/jquery.js")
%script(src="/assets/js/bootstrap-responsive.min.js")
The actual template which inherits from base
<%inherit file="base.haml"/>
.row-fluid
.span12-fluid=data
The output from the root is fine
<!DOCTYPE html>
<html lang="en">
<head>
<title>Hello there</title>
<meta content="width=device-width, initial-scale=1.0" name="viewport" />
<link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet" />
</head>
<body>
<div class="container-fluid">
<div class="row-fluid">
<div class="span12-fluid">Hello I like CherryPy!</div>
</div>
</div>
<script src="http://code.jquery.com/jquery.js"></script>
<script src="/assets/js/bootstrap-responsive.min.js"></script>
</body>
</html>
but when you have a look at the output from the css you get
<!DOCTYPE html>
<html lang="en">
<head>
<title>Hello there</title>
<meta content="width=device-width, initial-scale=1.0" name="viewport" />
<link href="/assets/css/bootstrap-responsive.min.css" rel="stylesheet" />
</head>
<body>
<div class="container-fluid">
<div class="row-fluid">
<div class="span12-fluid">/*!
* Bootstrap Responsive v2.3.1
*
* Copyright 2012 Twitter, Inc
* Licensed under the Apache License v2.0
* http://www.apache.org/licenses/LICENSE-2.0
*
* Designed and built with all the love in the world #twitter by #mdo and #fat.
*/.clearfix{*zoom:1}.clearfix:before,.clearfix:after{display:table;line-height:0;content:""}.clearfix:after{clear:both}.hide-text{font:0/0 a;color:transparent;text-shadow:none;background-color:transparent;border:0}.input-block-level{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}#-ms-viewport{width:device-width}.hidden{display:none;visibility:hidden}.visible-phone{display:none!important}.visible-tablet{display:none!important}.hidden-desktop{display:none!important}.visible-desktop{display:inherit!important}#media(min-width:768px) and (max-width:979px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-tablet{display:inherit!important}.hidden-tablet{display:none!important}}#media(max-width:767px){.hidden-desktop{display:inherit!important}.visible-desktop{display:none!important}.visible-phone{display:inherit!important}.hidden-phone{display:none!important}}.visible-print{display:none!important}#media print{.visible-print{display:inherit!important}.hidden-print{display:none!important}}#media(min-width:1200px){.row{margin-left:-30px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;min-height:1px;margin-left:30px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:1170px}.span12{width:1170px}.span11{width:1070px}.span10{width:970px}.span9{width:870px}.span8{width:770px}.span7{width:670px}.span6{width:570px}.span5{width:470px}.span4{width:370px}.span3{width:270px}.span2{width:170px}.span1{width:70px}.offset12{margin-left:1230px}.offset11{margin-left:1130px}.offset10{margin-left:1030px}.offset9{margin-left:930px}.offset8{margin-left:830px}.offset7{margin-left:730px}.offset6{margin-left:630px}.offset5{margin-left:530px}.offset4{margin-left:430px}.offset3{margin-left:330px}.offset2{margin-left:230px}.offset1{margin-left:130px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.564102564102564%;*margin-left:2.5109110747408616%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .controls-row [class*="span"]+[class*="span"]{margin-left:2.564102564102564%}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.45299145299145%;*width:91.39979996362975%}.row-fluid .span10{width:82.90598290598291%;*width:82.8527914166212%}.row-fluid .span9{width:74.35897435897436%;*width:74.30578286961266%}.row-fluid .span8{width:65.81196581196582%;*width:65.75877432260411%}.row-fluid .span7{width:57.26495726495726%;*width:57.21176577559556%}.row-fluid .span6{width:48.717948717948715%;*width:48.664757228587014%}.row-fluid .span5{width:40.17094017094017%;*width:40.11774868157847%}.row-fluid .span4{width:31.623931623931625%;*width:31.570740134569924%}.row-fluid .span3{width:23.076923076923077%;*width:23.023731587561375%}.row-fluid .span2{width:14.52991452991453%;*width:14.476723040552828%}.row-fluid .span1{width:5.982905982905983%;*width:5.929714493544281%}.row-fluid .offset12{margin-left:105.12820512820512%;*margin-left:105.02182214948171%}.row-fluid .offset12:first-child{margin-left:102.56410256410257%;*margin-left:102.45771958537915%}.row-fluid .offset11{margin-left:96.58119658119658%;*margin-left:96.47481360247316%}.row-fluid .offset11:first-child{margin-left:94.01709401709402%;*margin-left:93.91071103837061%}.row-fluid .offset10{margin-left:88.03418803418803%;*margin-left:87.92780505546462%}.row-fluid .offset10:first-child{margin-left:85.47008547008548%;*margin-left:85.36370249136206%}.row-fluid .offset9{margin-left:79.48717948717949%;*margin-left:79.38079650845607%}.row-fluid .offset9:first-child{margin-left:76.92307692307693%;*margin-left:76.81669394435352%}.row-fluid .offset8{margin-left:70.94017094017094%;*margin-left:70.83378796144753%}.row-fluid .offset8:first-child{margin-left:68.37606837606839%;*margin-left:68.26968539734497%}.row-fluid .offset7{margin-left:62.393162393162385%;*margin-left:62.28677941443899%}.row-fluid .offset7:first-child{margin-left:59.82905982905982%;*margin-left:59.72267685033642%}.row-fluid .offset6{margin-left:53.84615384615384%;*margin-left:53.739770867430444%}.row-fluid .offset6:first-child{margin-left:51.28205128205128%;*margin-left:51.175668303327875%}.row-fluid .offset5{margin-left:45.299145299145295%;*margin-left:45.1927623204219%}.row-fluid .offset5:first-child{margin-left:42.73504273504273%;*margin-left:42.62865975631933%}.row-fluid .offset4{margin-left:36.75213675213675%;*margin-left:36.645753773413354%}.row-fluid .offset4:first-child{margin-left:34.18803418803419%;*margin-left:34.081651209310785%}.row-fluid .offset3{margin-left:28.205128205128204%;*margin-left:28.0987452264048%}.row-fluid .offset3:first-child{margin-left:25.641025641025642%;*margin-left:25.53464266230224%}.row-fluid .offset2{margin-left:19.65811965811966%;*margin-left:19.551736679396257%}.row-fluid .offset2:first-child{margin-left:17.094017094017094%;*margin-left:16.98763411529369%}.row-fluid .offset1{margin-left:11.11111111111111%;*margin-left:11.004728132387708%}.row-fluid .offset1:first-child{margin-left:8.547008547008547%;*margin-left:8.440625568285142%}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:30px}input.span12,textarea.span12,.uneditable-input.span12{width:1156px}input.span11,textarea.span11,.uneditable-input.span11{width:1056px}input.span10,textarea.span10,.uneditable-input.span10{width:956px}input.span9,textarea.span9,.uneditable-input.span9{width:856px}input.span8,textarea.span8,.uneditable-input.span8{width:756px}input.span7,textarea.span7,.uneditable-input.span7{width:656px}input.span6,textarea.span6,.uneditable-input.span6{width:556px}input.span5,textarea.span5,.uneditable-input.span5{width:456px}input.span4,textarea.span4,.uneditable-input.span4{width:356px}input.span3,textarea.span3,.uneditable-input.span3{width:256px}input.span2,textarea.span2,.uneditable-input.span2{width:156px}input.span1,textarea.span1,.uneditable-input.span1{width:56px}.thumbnails{margin-left:-30px}.thumbnails>li{margin-left:30px}.row-fluid .thumbnails{margin-left:0}}#media(min-width:768px) and (max-width:979px){.row{margin-left:-20px;*zoom:1}.row:before,.row:after{display:table;line-height:0;content:""}.row:after{clear:both}[class*="span"]{float:left;min-height:1px;margin-left:20px}.container,.navbar-static-top .container,.navbar-fixed-top .container,.navbar-fixed-bottom .container{width:724px}.span12{width:724px}.span11{width:662px}.span10{width:600px}.span9{width:538px}.span8{width:476px}.span7{width:414px}.span6{width:352px}.span5{width:290px}.span4{width:228px}.span3{width:166px}.span2{width:104px}.span1{width:42px}.offset12{margin-left:764px}.offset11{margin-left:702px}.offset10{margin-left:640px}.offset9{margin-left:578px}.offset8{margin-left:516px}.offset7{margin-left:454px}.offset6{margin-left:392px}.offset5{margin-left:330px}.offset4{margin-left:268px}.offset3{margin-left:206px}.offset2{margin-left:144px}.offset1{margin-left:82px}.row-fluid{width:100%;*zoom:1}.row-fluid:before,.row-fluid:after{display:table;line-height:0;content:""}.row-fluid:after{clear:both}.row-fluid [class*="span"]{display:block;float:left;width:100%;min-height:30px;margin-left:2.7624309392265194%;*margin-left:2.709239449864817%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="span"]:first-child{margin-left:0}.row-fluid .controls-row [class*="span"]+[class*="span"]{margin-left:2.7624309392265194%}.row-fluid .span12{width:100%;*width:99.94680851063829%}.row-fluid .span11{width:91.43646408839778%;*width:91.38327259903608%}.row-fluid .span10{width:82.87292817679558%;*width:82.81973668743387%}.row-fluid .span9{width:74.30939226519337%;*width:74.25620077583166%}.row-fluid .span8{width:65.74585635359117%;*width:65.69266486422946%}.row-fluid .span7{width:57.18232044198895%;*width:57.12912895262725%}.row-fluid .span6{width:48.61878453038674%;*width:48.56559304102504%}.row-fluid .span5{width:40.05524861878453%;*width:40.00205712942283%}.row-fluid .span4{width:31.491712707182323%;*width:31.43852121782062%}.row-fluid .span3{width:22.92817679558011%;*width:22.87498530621841%}.row-fluid .span2{width:14.3646408839779%;*width:14.311449394616199%}.row-fluid .span1{width:5.801104972375691%;*width:5.747913483013988%}.row-fluid .offset12{margin-left:105.52486187845304%;*margin-left:105.41847889972962%}.row-fluid .offset12:first-child{margin-left:102.76243093922652%;*margin-left:102.6560479605031%}.row-fluid .offset11{margin-left:96.96132596685082%;*margin-left:96.8549429881274%}.row-fluid .offset11:first-child{margin-left:94.1988950276243%;*margin-left:94.09251204890089%}.row-fluid .offset10{margin-left:88.39779005524862%;*margin-left:88.2914070765252%}.row-fluid .offset10:first-child{margin-left:85.6353591160221%;*margin-left:85.52897613729868%}.row-fluid .offset9{margin-left:79.8342541436464%;*margin-left:79.72787116492299%}.row-fluid .offset9:first-child{margin-left:77.07182320441989%;*margin-left:76.96544022569647%}.row-fluid .offset8{margin-left:71.2707182320442%;*margin-left:71.16433525332079%}.row-fluid .offset8:first-child{margin-left:68.50828729281768%;*margin-left:68.40190431409427%}.row-fluid .offset7{margin-left:62.70718232044199%;*margin-left:62.600799341718584%}.row-fluid .offset7:first-child{margin-left:59.94475138121547%;*margin-left:59.838368402492065%}.row-fluid .offset6{margin-left:54.14364640883978%;*margin-left:54.037263430116376%}.row-fluid .offset6:first-child{margin-left:51.38121546961326%;*margin-left:51.27483249088986%}.row-fluid .offset5{margin-left:45.58011049723757%;*margin-left:45.47372751851417%}.row-fluid .offset5:first-child{margin-left:42.81767955801105%;*margin-left:42.71129657928765%}.row-fluid .offset4{margin-left:37.01657458563536%;*margin-left:36.91019160691196%}.row-fluid .offset4:first-child{margin-left:34.25414364640884%;*margin-left:34.14776066768544%}.row-fluid .offset3{margin-left:28.45303867403315%;*margin-left:28.346655695309746%}.row-fluid .offset3:first-child{margin-left:25.69060773480663%;*margin-left:25.584224756083227%}.row-fluid .offset2{margin-left:19.88950276243094%;*margin-left:19.783119783707537%}.row-fluid .offset2:first-child{margin-left:17.12707182320442%;*margin-left:17.02068884448102%}.row-fluid .offset1{margin-left:11.32596685082873%;*margin-left:11.219583872105325%}.row-fluid .offset1:first-child{margin-left:8.56353591160221%;*margin-left:8.457152932878806%}input,textarea,.uneditable-input{margin-left:0}.controls-row [class*="span"]+[class*="span"]{margin-left:20px}input.span12,textarea.span12,.uneditable-input.span12{width:710px}input.span11,textarea.span11,.uneditable-input.span11{width:648px}input.span10,textarea.span10,.uneditable-input.span10{width:586px}input.span9,textarea.span9,.uneditable-input.span9{width:524px}input.span8,textarea.span8,.uneditable-input.span8{width:462px}input.span7,textarea.span7,.uneditable-input.span7{width:400px}input.span6,textarea.span6,.uneditable-input.span6{width:338px}input.span5,textarea.span5,.uneditable-input.span5{width:276px}input.span4,textarea.span4,.uneditable-input.span4{width:214px}input.span3,textarea.span3,.uneditable-input.span3{width:152px}input.span2,textarea.span2,.uneditable-input.span2{width:90px}input.span1,textarea.span1,.uneditable-input.span1{width:28px}}#media(max-width:767px){body{padding-right:20px;padding-left:20px}.navbar-fixed-top,.navbar-fixed-bottom,.navbar-static-top{margin-right:-20px;margin-left:-20px}.container-fluid{padding:0}.dl-horizontal dt{float:none;width:auto;clear:none;text-align:left}.dl-horizontal dd{margin-left:0}.container{width:auto}.row-fluid{width:100%}.row,.thumbnails{margin-left:0}.thumbnails>li{float:none;margin-left:0}[class*="span"],.uneditable-input[class*="span"],.row-fluid [class*="span"]{display:block;float:none;width:100%;margin-left:0;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.span12,.row-fluid .span12{width:100%;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.row-fluid [class*="offset"]:first-child{margin-left:0}.input-large,.input-xlarge,.input-xxlarge,input[class*="span"],select[class*="span"],textarea[class*="span"],.uneditable-input{display:block;width:100%;min-height:30px;-webkit-box-sizing:border-box;-moz-box-sizing:border-box;box-sizing:border-box}.input-prepend input,.input-append input,.input-prepend input[class*="span"],.input-append input[class*="span"]{display:inline-block;width:auto}.controls-row [class*="span"]+[class*="span"]{margin-left:0}.modal{position:fixed;top:20px;right:20px;left:20px;width:auto;margin:0}.modal.fade{top:-100px}.modal.fade.in{top:20px}}#media(max-width:480px){.nav-collapse{-webkit-transform:translate3d(0,0,0)}.page-header h1 small{display:block;line-height:20px}input[type="checkbox"],input[type="radio"]{border:1px solid #ccc}.form-horizontal .control-label{float:none;width:auto;padding-top:0;text-align:left}.form-horizontal .controls{margin-left:0}.form-horizontal .control-list{padding-top:0}.form-horizontal .form-actions{padding-right:10px;padding-left:10px}.media .pull-left,.media .pull-right{display:block;float:none;margin-bottom:10px}.media-object{margin-right:0;margin-left:0}.modal{top:10px;right:10px;left:10px}.modal-header .close{padding:10px;margin:-10px}.carousel-caption{position:static}}#media(max-width:979px){body{padding-top:0}.navbar-fixed-top,.navbar-fixed-bottom{position:static}.navbar-fixed-top{margin-bottom:20px}.navbar-fixed-bottom{margin-top:20px}.navbar-fixed-top .navbar-inner,.navbar-fixed-bottom .navbar-inner{padding:5px}.navbar .container{width:auto;padding:0}.navbar .brand{padding-right:10px;padding-left:10px;margin:0 0 0 -5px}.nav-collapse{clear:both}.nav-collapse .nav{float:none;margin:0 0 10px}.nav-collapse .nav>li{float:none}.nav-collapse .nav>li>a{margin-bottom:2px}.nav-collapse .nav>.divider-vertical{display:none}.nav-collapse .nav .nav-header{color:#777;text-shadow:none}.nav-collapse .nav>li>a,.nav-collapse .dropdown-menu a{padding:9px 15px;font-weight:bold;color:#777;-webkit-border-radius:3px;-moz-border-radius:3px;border-radius:3px}.nav-collapse .btn{padding:4px 10px 4px;font-weight:normal;-webkit-border-radius:4px;-moz-border-radius:4px;border-radius:4px}.nav-collapse .dropdown-menu li+li a{margin-bottom:2px}.nav-collapse .nav>li>a:hover,.nav-collapse .nav>li>a:focus,.nav-collapse .dropdown-menu a:hover,.nav-collapse .dropdown-menu a:focus{background-color:#f2f2f2}.navbar-inverse .nav-collapse .nav>li>a,.navbar-inverse .nav-collapse .dropdown-menu a{color:#999}.navbar-inverse .nav-collapse .nav>li>a:hover,.navbar-inverse .nav-collapse .nav>li>a:focus,.navbar-inverse .nav-collapse .dropdown-menu a:hover,.navbar-inverse .nav-collapse .dropdown-menu a:focus{background-color:#111}.nav-collapse.in .btn-group{padding:0;margin-top:5px}.nav-collapse .dropdown-menu{position:static;top:auto;left:auto;display:none;float:none;max-width:none;padding:0;margin:0 15px;background-color:transparent;border:0;-webkit-border-radius:0;-moz-border-radius:0;border-radius:0;-webkit-box-shadow:none;-moz-box-shadow:none;box-shadow:none}.nav-collapse .open>.dropdown-menu{display:block}.nav-collapse .dropdown-menu:before,.nav-collapse .dropdown-menu:after{display:none}.nav-collapse .dropdown-menu .divider{display:none}.nav-collapse .nav>li>.dropdown-menu:before,.nav-collapse .nav>li>.dropdown-menu:after{display:none}.nav-collapse .navbar-form,.nav-collapse .navbar-search{float:none;padding:10px 15px;margin:10px 0;border-top:1px solid #f2f2f2;border-bottom:1px solid #f2f2f2;-webkit-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.1);-moz-box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.1);box-shadow:inset 0 1px 0 rgba(255,255,255,0.1),0 1px 0 rgba(255,255,255,0.1)}.navbar-inverse .nav-collapse .navbar-form,.navbar-inverse .nav-collapse .navbar-search{border-top-color:#111;border-bottom-color:#111}.navbar .nav-collapse .nav.pull-right{float:none;margin-left:0}.nav-collapse,.nav-collapse.collapse{height:0;overflow:hidden}.navbar .btn-navbar{display:block}.navbar-static .navbar-inner{padding-right:10px;padding-left:10px}}#media(min-width:980px){.nav-collapse.collapse{height:auto!important;overflow:visible!important}}
</div>
</div>
</div>
<script src="http://code.jquery.com/jquery.js"></script>
<script src="/assets/js/bootstrap-responsive.min.js"></script>
</body>
</html>
What have I done wrong and how do I prevent the tenmplate from being applied to css and javascript files referenced by this page?
UPDATE:
I forgot to mention that I already tried disabling the template plugin on the static files by putting
'tools.template.on':False
in the static files part of the config. The only way I seem to be able to stop this is to remove the templates decorator all together. The only problem with that is the template will not be applied and the output will not include the html which includes the desired css import.
In your static sections, simply disable the tool:
tools.template.on: False