Develop Reference

web2py: How to execute instructions before delete using SQLFORM.smartgrid

I use SQLFORM.smartgrid to show a list of records from a table (service_types). In each row of the smartgrid there is a delete link/button to delete the record. I want to executive some code before smartgrid/web2py actually deletes the record, for example I want to know if there are child records (services table) referencing this record, and if any, flash a message telling user that record cannot be deleted. How is this done?
db.py
db.define_table('service_types',
Field('type_name', requires=[IS_NOT_EMPTY(), IS_ALPHANUMERIC()]),
format='%(type_name)s',
)
db.define_table('services',
Field('service_name',requires=[IS_NOT_EMPTY(),IS_NOT_IN_DB(db,'services.service_name')]),
Field('service_type','reference service_types',requires=IS_IN_DB(db,db.service_types.id,
'%(type_name)s',
error_message='not in table',
zero=None),
ondelete='RESTRICT',
),
Field('interest_rate','decimal(15,2)',requires=IS_DECIMAL_IN_RANGE(0,100)),
Field('max_term','integer'),
auth.signature,
format='%(service_name)s',
)
db.services._plural='Services'
db.services._singular='Service'
if db(db.service_types).count() < 1:
db.service_types.insert(type_name='Loan')
db.service_types.insert(type_name='Contribution')
db.service_types.insert(type_name='Other')
controller
def list_services():
grid = SQLFORM.smartgrid(db.services
, fields = [db.services.service_name,db.services.service_type]
)
return locals()
view
{{extend 'layout.html'}}
{{=grid}}

There are two options. First, the deletable argument can be a function that takes the Row object of a given record and returns True or False to indicate whether the record is deletable. If it returns False, the "Delete" button will not be shown for that record, nor the delete operation be allowed on the server.
def can_delete(row):
return True if [some condition involving row] else False
grid = SQLFORM.smartgrid(..., deletable=can_delete)
Second, there is an ondelete argument that takes the db Table object and the record ID. It is called right before the delete operation, so to prevent the delete, you can do a redirect within that function:
def ondelete(table, record_id):
record = table(record_id)
if [some condition]:
session.flash = 'Cannot delete this record'
redirect(URL())
grid = SQLFORM.smartgrid(..., ondelete=ondelete)
Note, if the grid is loaded via an Ajax component and its actions are therefore performed via Ajax, using redirect within the ondelete method as shown above will not work well, as the redirect will have no effect and the table row will still be deleted from the grid in the browser (even though the database record was not deleted). In that case, an alternative approach is to return a non-200 HTTP response to the browser, which will prevent the client-side Javascript from deleting the row from the table (the delete happens only on success of the Ajax request). We should also set response.flash instead of session.flash (because we are not redirecting/reloading the whole page):
def ondelete(table, record_id):
record = table(record_id)
if [some condition]:
response.flash = 'Cannot delete this record'
raise HTTP(403)
Note, both the deletable and ondelete arguments can be dictionaries with table names as keys, so you can specify different values for different tables that might be linked from the smartgrid.
Finally, notice the delete URLs look like /appname/list_services/services/delete/services/[record ID]. So, in the controller, you can determine if a delete is being requested by checking if 'delete' in request.args. In that case, request.args[-2:] represents the table name and record ID, which you can use to do any checks.

From Anthony's answer I chose the second option and came up with the following:
def ondelete_service_type(service_type_table, service_type_id):
count = db(db.services.service_type == service_type_id).count()
if count > 0:
session.flash = T("Cant delete")
#redirect(URL('default','list_service_types#'))
else:
pass
return locals()
def list_service_types():
grid = SQLFORM.smartgrid(db.service_types
, fields = [db.service_types.type_name, db.services.service_name]
, ondelete = ondelete_service_type
)
return locals()
But, if I do this...
if count > 0:
session.flash = T("Cant delete")
else:
pass
return locals()
I get this error:
And if I do this:
if count > 0:
session.flash = T("Cant delete")
redirect(URL('default','list_service_types#')) <== please take note
else:
pass
return locals()
I get the flash error message Cant delete but the record appears deleted from the list, and reappears after a page refresh with F5 (apparently because the delete was not allowed in the database, which is intended).
Which one should I fix and how?
Note
If any of these issue is resolved I can accept Anthony's answer.

BDD behave Python need to create a World map to hold values

I'm not too familiar with Python but I have setup a BDD framework using Python behave, I now want to create a World map class that holds data and is retrievable throughout all scenarios.
For instance I will have a world class where I can use:
World w
w.key.add('key', api.response)
In one scenario and in another I can then use:
World w
key = w.key.get('key').
Edit:
Or if there is a built in way of using context or similar in behave where the attributes are saved and retrievable throughout all scenarios that would be good.
Like lettuce where you can use world http://lettuce.it/tutorial/simple.html
I've tried this between scenarios but it doesn't seem to be picking it up
class World(dict):
def __setitem__(self, key, item):
self.__dict__[key] = item
print(item)
def __getitem__(self, key):
return self.__dict__[key]
Setting the item in one step in scenario A: w.setitem('key', response)
Getting the item in another step in scenario B: w.getitem('key',)
This shows me an error though:
Traceback (most recent call last):
File "C:\Program Files (x86)\Python\lib\site-packages\behave\model.py", line 1456, in run
match.run(runner.context)
File "C:\Program Files (x86)\Python\lib\site-packages\behave\model.py", line 1903, in run
self.func(context, *args, **kwargs)
File "steps\get_account.py", line 14, in step_impl
print(w.__getitem__('appToken'))
File "C:Project\steps\world.py", line 8, in __getitem__
return self.__dict__[key]
KeyError: 'key'
It appears that the World does not hold values here between steps that are run.
Edit:
I'm unsure how to use environment.py but can see it has a way of running code before the steps. How can I allow my call to a soap client within environment.py to be called and then pass this to a particular step?
Edit:
I have made the request in environment.py and hardcoded the values, how can I pass variables to environment.py and back?

It's called "context" in the python-behave jargon. The first argument of your step definition function is an instance of the behave.runner.Context class, in which you can store your world instance. Please see the appropriate part of the tutorial.

Have you tried the
simple approach, using global var, for instance:
def before_all(context):
global response
response = api.response
def before_scenario(context, scenario):
global response
w.key.add('key', response)
Guess feature can be accessed from context, for instance:
def before_feature(context, feature):
feature.response = api.response
def before_scenario(context, scenario):
w.key.add('key', context.feature.response)
You are looking for:
Class variable: A variable that is shared by all instances of a class.
Your code in Q uses Class Instance variable.
Read about: python_classes_objects
For instance:
class World(dict):
__class_var = {}
def __setitem__(self, key, item):
World.__class_var[key] = item
def __getitem__(self, key):
return World.__class_var[key]
# Scenario A
A = World()
A['key'] = 'test'
print('A[\'key\']=%s' % A['key'] )
del A
# Scenario B
B = World()
print('B[\'key\']=%s' % B['key'] )
Output:
A['key']=test
B['key']=test
Tested with Python:3.4.2
Come back and Flag your Question as answered if this is working for you or comment why not.

Defining global var in before_all hook did not work for me.
As mentioned by #stovfl
But defining global var within one of my steps worked out.
Instead, as Szabo Peter mentioned use the context.
context.your_variable_name = api.response
and just use
context.your_variable_name anywhere the value is to be used.

For this I actually used a config file [config.py] I then added the variables in there and retrieved them using getattr. See below:
WSDL_URL = 'wsdl'
USERNAME = 'name'
PASSWORD = 'PWD'
Then retrieved them like:
import config
getattr(config, 'USERNAME ', 'username not found')

Using python class with spark DataFrame to parse URL's

I'm trying to process URL's in a pyspark dataframe using a class that I've written and a udf. I'm aware of urllib and other url parsing libraries but for this case I need to use my own code.
In order to get the tld of a url I cross check it against the iana public suffix list.
Here's a simplification of my code
class Parser:
# list of available public suffixes for extracting top level domains
file = open("public_suffix_list.txt", 'r')
data = []
for line in file:
if line.startswith("//") or line == '\n':
pass
else:
data.append(line.strip('\n'))
def __init__(self, url):
self.url = url
#the code here extracts port,protocol,query etc.
#I think this bit below is causing the error
matches = [r for r in self.data if r in self.hostname]
#extra functionality in my actual class
i = matches.index(self.string)
try:
self.tld = matches[i]
# logic to find tld if no match
The class works in pure python so for example I can run
import Parser
x = Parser("www.google.com")
x.tld #returns ".com"
However when I try to do
import Parser
from pyspark.sql.functions import udf
parse = udf(lambda x: Parser(x).url)
df = sqlContext.table("tablename").select(parse("column"))
When I call an action I get
File "<stdin>", line 3, in <lambda>
File "<stdin>", line 27, in __init__
TypeError: 'in <string>' requires string as left operand
So my guess is that it's failing to interpret the data as a list of strings?
I've also tried to use
file = sc.textFile("my_file.txt")\
.filter(lambda x: not x.startswith("//") or != "")\
.collect()
data = sc.broadcast(file)
to open my file instead, but that causes
Exception: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transforamtion. SparkContext can only be used on the driver, not in code that it run on workers. For more information, see SPARK-5063.
Any ideas?
Thanks in advance
EDIT: Apologies, I didn't have my code to hand so my test code didn't explain very well the problems I was having. The error I initially reported was a result of the test data I was using.
I've updated my question to be more reflective of the challenge I'm facing.

Why do you need a class in this case (the code for defining your class is incorrect, you never declared self.data before using it in the init method) the only relevant line that affects the output you want is self.string=string, so you are basically passing the identity function as udf.
The UnicodeDecodeError is due to an encoding issue in your file, it has nothing to do with your definition of the class.
The second error is in the line sc.broadcast(file) , details of which can be found here : Spark: Broadcast variables: It appears that you are attempting to reference SparkContext from a broadcast variable, action, or transforamtion
EDIT 1
I would redefine your class structure as follows. You basically need to create the instance self.data by calling self.data = data before you can use it. Also anything that you write before the init method is executed irrespective of whether you call that class or not. So moving out the file parsing part will not have any effect.
# list of available public suffixes for extracting top level domains
file = open("public_suffix_list.txt", 'r')
data = []
for line in file:
if line.startswith("//") or line == '\n':
pass
else:
data.append(line.strip('\n'))
class Parser:
def __init__(self, url):
self.url = url
self.data = data
#the code here extracts port,protocol,query etc.
#I think this bit below is causing the error
matches = [r for r in self.data if r in self.hostname]
#extra functionality in my actual class
i = matches.index(self.string)
try:
self.tld = matches[i]
# logic to find tld if no match

Understanding flow of execution of Python code

I'm trying to do home assignment connected with python from Data Manipulation at Scale: Systems and Algorithms at Curesra. Generally I have problems with understanding base code which was presented as an example of MapReduce alogorythm. I would be grateful for helping me understand it in 2 places, details below.
I tired to go step by step through code flow of below two files after running command:
python wordcount.py 'data/books.json'
File wordcount.py is opened
mr = MapReduce.MapReduce() - me object is created
def __init__(self): part from MapReduce.py is
executed
We come back to wordcount.py
Functions def mapper(record): and def reducer(key,list_of_values): are created but for the time being without execution
Python go to if __name__ == '__main__':
` inputdata = open(sys.argv[1]) - json file is assigned to a
variable
mr.execute(inputdata, mapper, reducer) - A call to the function from MapReduce.py.
And here is my first question we haven't deffined mapper or reducer variable/object so far. Is it just null/no value passed to this function or we somehow defined this variable before but I missed this?
Later me move to def execute(self, data, mapper, reducer): in
MapReduce.py
And there we have mapper(record).
So this is reference to a function in wordcount.py, am I right? But if we have reference to a function in different file shouldn't we use import at the beginning of the file and define from which file this function came?
(...) further code execution
wordcount.py file:
import MapReduce
import sys
"""
Word Count Example in the Simple Python MapReduce Framework
"""
mr = MapReduce.MapReduce()
# =============================
# Do not modify above this line
def mapper(record):
# key: document identifier
# value: document contents
key = record[0]
value = record[1]
words = value.split()
for w in words:
mr.emit_intermediate(w, 1)
def reducer(key, list_of_values):
# key: word
# value: list of occurrence counts
total = 0
for v in list_of_values:
total += v
mr.emit((key, total))
# Do not modify below this line
# =============================
if __name__ == '__main__':
inputdata = open(sys.argv[1])
mr.execute(inputdata, mapper, reducer)
MapReduce.py file:
import json
class MapReduce:
def __init__(self):
self.intermediate = {}
self.result = []
def emit_intermediate(self, key, value):
self.intermediate.setdefault(key, [])
self.intermediate[key].append(value)
def emit(self, value):
self.result.append(value)
def execute(self, data, mapper, reducer):
for line in data:
record = json.loads(line)
mapper(record)
for key in self.intermediate:
reducer(key, self.intermediate[key])
#jenc = json.JSONEncoder(encoding='latin-1')
jenc = json.JSONEncoder()
for item in self.result:
print jenc.encode(item)
Thank you in advance for help with that.

In python everything is a object, that include functions, so you can pass a functionA as argument to another functionB (or class or whenever), and if functionB expect that you to do it, it will assume that you give it a functions with the right firm and a proceed as normal.
In yours case
mr.execute(inputdata, mapper, reducer)
here mapper, reducer are the functions previously defined that are passed as argument to the method execute of the instance mr of the class MapReduce and as you can see, said method use it as the functions that it expect.
Thank to this you can, as the that code show, make generic code that do some calculus that can be used in similar way by many applications by given the user the options of supplies his/her own functions.
A much more generic example of this is the function map, this function receive a function that do something, map don't care what it does or where it comefrom, only that receive as many argument as map himself receive (others that say functions) and return a value to build a new list with the results.

Loop using app.route on Python

I'm trying to create several URLs on my serv thanks to a loop . The issue is that each function I create in a app.route can't have the same name than the others . And I don't know how to create different function names ...
Here is the code :
json_tweets = []
for line in open('C:\Users\Benjamin\Desktop\DashboardProject\last_rated_set.json',"r"):
json_tweets.append(json.loads(line,"ISO-8859-1"))
cashtag_tab = []
for tweet in json_tweets:
if not(tweet['cashtag'] in cashtag_tab) :
cashtag_tab.append(tweet['cashtag'])
for i in range(0,(len(cashtag_tab)-1)) :
var=cashtag_tab[i]
#app.route("/"+var)
def company(var) :
finance=Share(var)
datas = finance.get_historical('2014-01-01', '2014-12-31')
datas = json.dumps(datas, default=json_util.default)
return datas
I'm getting the error AssertionError : View function mapping is overwritting an existing endpoint function : company

This fails because Flask derives the endpoint name from the function by default, but it would anyway fail later because the function company requires an argument var and the route is not parameterised. The simplest option would be just checking the value inside the handler:
#api.route('/<var>')
def company(var):
if var not in cashtag_tab:
abort(404)
If you want all the routes to be in the routing map for any reason, I once needed a similar thing and came up with something like this:
def url_family(source, methods=('GET',)):
def decorator(f):
for entry in source:
# create a handler that delegates to your function
def view_func(entry=entry, **kwargs):
return f(entry, **kwargs)
endpoint = '{0}_{1}'.format(f.__name__, entry)
url = '/{0}'.format(entry)
api.add_url_rule(url,
methods=methods,
endpoint=endpoint,
view_func=view_func)
return decorator
Then you register the handlers as:
#url_family(cashtag_tab)
def company(var):
...

Assuming that you are using flask now, you should consider Custom URL Converter. Check links below
http://flask.pocoo.org/docs/0.10/api/#flask.Flask.url_map - url_map UrlConverter API
https://exploreflask.com/views.html#url-converters - example url converter
https://stackoverflow.com/a/5872904/3451543 - RegexConverter by Philip Southam
Anyway, specifying more details on your question is always helpful to get accurate answer :)