Develop Reference

loadhook function in web.py not working

I was trying to experiment with the loadhook function in web.py, however I am not quite able to make it work. Here is my code:
import web
render = web.template.render('templates/')
urls = (
'/(.*)', 'index'
)
class index:
def GET(self, name):
return render.base(name)
def test():
print "damn"
render.base("test")
if __name__ == "__main__":
app = web.application(urls, globals())
app.run()
app.add_processor(web.loadhook(test))
The base.html template is pretty simple which echoes back the "name" parameter.
What I understood from the documentation was that the loadhook function will be called before every request. But it doesn't seem to work. I have tried going to the homepage, another page etc. Neither do I see a print statement on my CMD, nor does the base template with the name test gets executed.
I tried running the same code with just the add_processor as well, but no luck.
Can anyone help me figure out how to run a function before a request happens on a page?
Also, I am assuming request only encompasses browser level requests. Is there any way to capture more via web.py? (such as call a function on keypress, mouse click etc.)
Any help is much appreciated!

loadhooks are called early in the processing and are used to either set configuration or intercept. For example, I implement a black list similar to the following:
def my_hook():
# If requester's IP is in my blacklist, redirect his browser.
if blacklist.in_blacklist(web.ctx.ip) and web.ctx.path != '/blacklist':
raise web.seeother('/blacklist')
....
app.add_processor(web.loadhook(my_hook))
In your example, your test hook calls render (I'm guessing you're trying to render the test page?) Problem is loadhooks don't return data to the browser, so calling render here doesn't do what you want.
Couple other issues: you need to call app.add_processor(web.loadhook(my_hook)) prior to calling app.run(), because the latter sets your polling loop & never returns.
As for your final question: to capture keypresses, etc. you need your javascript to send something to the server.... Everytime there's a keypress, do an ajax call to the server to log the action.
Python's powerful, but still can't read minds.

JMeter - Run a python script before calling each HTTP request sampler

I am new to Jmeter. My HTTP request sampler call looks like this
Path= /image/**image_id**/list/
Header = "Key" : "Key_Value"
Key value is generated by calling a python script which uses the image_id to generate a unique key.
Before each sampler I wanted to generate the key using python script which will be passed as a header to the next HTTP Request sampler.
I know I have to used some kind of preprocessor to do that. Can anyone help me do it using a preprocessor in jmeter.

I believe that Beanshell PreProcessor is what you're looking for.
Example Beanshell code will look as follows:
import java.io.BufferedReader;
import java.io.InputStreamReader;
Runtime r = Runtime.getRuntime();
Process p = r.exec("/usr/bin/python /path/to/your/script.py");
p.waitFor();
BufferedReader b = new BufferedReader(new InputStreamReader(p.getInputStream()));
String line = "";
StringBuilder response = new StringBuilder();
while ((line = b.readLine()) != null) {
response.append(line);
}
b.close();
vars.put("ID",response.toString());
The code above will execute Python script and put it's response into ID variable.
You will be able to refer it in your HTTP Request as
/image/${ID}/list/
See How to use BeanShell: JMeter's favorite built-in component guide for more information on Beanshell scripting in Apache JMeter and a kind of Beanshell cookbook.
You can also put your request under Transaction Controller to exclude PreProcessor execution time from load report.

A possible solution posted by Eugene Kazakov here:
JSR223 sampler has good possibility to write and execute some code,
just put jython.jar into /lib directory, choose in "Language" pop-up
menu jython and write your code in this sampler.
Sadly there is a bug in Jython, but there are some suggestion on the page.
More here.

You can use a BSF PreProcessor.
First download the Jython Library and save to your jmeter's lib directory.
On your HTTP sampler add a BSF PreProcessor, choose as language Jython and perform your needed magic to obtain the id, as an example I used this one:
import random
randImageString = ""
for i in range(16):
randImageString = randImageString + chr(random.randint(ord('A'),ord('Z')))
vars.put("randimage", randImageString)
Note the vars.put("randimage",randImageString") which will insert the variable available later to jmeter.
Now on your test you can use ${randimage} when you need it:
Now every Request will be different changing with the value put to randimage on the Python Script.

how to get current request's url in webpy

I am using webpy framework. I want get current request's url in webpy.
please help me, thanks.

Just try printing web.ctx in your controller method and you will see a bunch of environmental variables.
from pprint import pprint
pprint(web.ctx)
So your url probably should be ctx.home + ctx.path + ctx.query or ctx.home + ctx.fullpath.
UPD: You may also take a look at web.url and web.changequery, find them in api docs: http://webpy.org/docs/0.3/api

Is ctx path what you are after?
Data Found in ctx
Request
path – the path requested by the user, relative to the current
application. If you are using subapplications, any part of the url
matched by the outer application will be trimmed off. E.g. you have a
main app in code.py, and a subapplication called admin.py. In code.py,
you point /admin to admin.app. In admin.py, you point /stories to a
class called stories. Within stories, web.ctx.path will be /stories,
not /admin/stories. E.g. /articles/845

Global variable in Python server

Background: I am a complete beginner when it comes to servers, but I know my way around programming in Python.
I am trying to setup a simple server using the basic Python 2.7 modules (SimpleHTTPServer, CGIHTTPServer, etc). This server needs to load a global, read-only variable with several GB of data from a file when it starts; then, when each user accesses the page, the server uses the big data to generate some output which is then given to the user.
For the sake of example, let's suppose I have a 4 GB file names.txt which contains all possible proper nouns of English:
Jack
John
Allison
Richard
...
Let's suppose that my goal is to read the whole list of names into memory, and then choose 1 name at random from this big list of proper nouns. I am currently able to use Python's native CGIHTTPServer module to accomplish this. To start, I just run the CGIHTTPServer module directly, by executing from a terminal:
python -m CGIHTTPServer
Then, someone accesses www.example-server.net:8000/foo.py and they are given one of these names at random. I have the following code in foo.py:
#!/usr/bin/env python
import random
name_list = list()
FILE = open('names.txt','r')
for line in FILE:
name = line[:-1]
name_list.append(name)
FILE.close()
name_to_return = random.choice(name_list)
print "Content-type: text/html"
print
print "<title>Here is your name</title>"
print "<p>" + name_to_return + "</p>"
This does what I want; however, it is extremely inefficient, because every access forces the server to re-read a 4 GB file.
How can I make this into an efficient process, where the variable name_list is created as global immediately when the server starts, and each access only reads from that variable?

Just for future reference, if anyone ever faces the same problem: I ended up sub-classing CGIHTTPServer's request handler and implementing a new do_POST() function. If you had a working CGI script without global variables, something like this should get you started:
import CGIHTTPServer
import random
import sys
import cgi
class MyRequestHandler(CGIHTTPServer.CGIHTTPRequestHandler):
global super_important_list
super_important_list = range(10)
random.shuffle(super_important_list)
def do_POST(s):
"""Respond to a POST request."""
form = cgi.FieldStorage(fp=s.rfile,headers=s.headers,environ={'REQUEST_METHOD':'POST','CONTENT_TYPE':s.headers['Content-Type'],})
s.wfile.write("<html><head><title>Title goes here.</title></head>")
s.wfile.write("<body><p>This is a test.</p>")
s.wfile.write("<p>You accessed path: %s</p>" % s.path)
s.wfile.write("<p>Also, super_important_list is:</p>")
s.wfile.write(str(super_important_list))
s.wfile.write("<p>Furthermore, you POSTed the following info: ")
for item in form.keys():
s.wfile.write("<p>Item: " + item)
s.wfile.write("<p>Value: " + form[item].value)
s.wfile.write("</body></html>")
if __name__ == '__main__':
server_address = ('', 8000)
httpd = CGIHTTPServer.BaseHTTPServer.HTTPServer(server_address, MyRequestHandler)
try:
httpd.serve_forever()
except KeyboardInterrupt:
sys.exit()
Whenever someone fills out your form and performs a POST, the variable form will be a dictionary-like object with key-value pairs which may differ for each user of your site, but the global variable super_important_list will be the same for every user.
Thanks to everyone who answered my question, especially Mike Steder, who pointed me in the right direction!

CGI works by spawning a process to handle each request. You need to run a server process that stays in memory handles HTTP requests.
You could use a modified BaseHTTPServer, just define your own Handler class. You'd load the dataset once in your code and then the do_GET method of your handler would just pick one randomly.
Personally, I'd look into something like CherryPy as a simple solution that is IMO a lot nicer than BaseHTTPServer. There are tons of options other than CherryPy like bottle, flask, twisted, django, etc. Of course if you need this server to be behind some other webserver you'll need to look into setting up a reverse proxy or running CherryPy as a WSGI app.

You may want to store the values of the names in a db and store the names according to the letter that they start with. Then you can do a random for a letter between a and z and from there randomize again to get a random name from your random beginning letter.

Build a prefix tree (a.k.a. trie) once and generate a random walk whenever you receive a query.
That should be pretty efficient.

Using response.out.write from within an included file in Google App Engine

I think this is quite an easy question to answer, I just haven't been able to find anywhere detailing how to do it.
I'm developing a GAE app.
In my main file I have a few request handlers, for example:
class Query(webapp.RequestHandler):
def post(self):
queryDOI = cgi.escape(self.request.get('doiortitle'))
import queryCosine
self.response.out.write(queryCosine.cosine(queryDOI))
In that handler there I'm importing from a queryCosine.py script which is doing all of the work. If something in the queryCosine script fails, I'd like to be able to print a message or do a redirect.
Inside queryCosine.py there is just a normal Python function, so obviously doing things like
self.response.out.write("Done")
doesn't work. What should I use instead of self or what do I need to include within my included file? I've tried using Query.self.response.out.write instead but that doesn't work.

A much better, more modular approach, is to have your queryCosine.cosine function throw an exception if something goes wrong. Then, your handler method can output the appropriate response depending on the return value or exception. This avoids unduly coupling the code that calculates whatever it is you're calculating to the webapp that hosts it.

Pass it to the function.
main file:
import second
...
second.somefunction(self.response.out.write)
second.py:
def somefunction(output):
output('Done')