Develop Reference

What is the reason, sometimes the parameters in class does not defined in __init__ but added as instance variables?

Why does these attributes (commented) does not included in the init ?
class RRT(object):
def __init__(self, start, stop, obstacle_list, rand_area):
self.start = Node(start[0], start[1])
self.stop = Node(stop[0], stop[1])
self.min_rand = rand_area[0]
self.max_rand = rand_area[1]
self.exapandDis = 1.0 # just like this one
self.goalSampleRate = 0.05 # just like this one
self.maxIter = 500 # just like this one
self.obstacle_list = obstacle_list

Have you ever heard of constructor?
The __init__ is a constructor in Python.
A constructor is used to initialized member variables of a class. It is not necessary for a constructor to initialize each and every variable.
Sometimes, we want a constructor to initialize variables, which aren't passed in as parameters, with some default values.
It is not necessary for a construtor to only intialize the variables which are passed to it!
Thus, your code will initialize: exapandDis, goalSampleRate, maxIter with default values 1.0, 0.05, 500

You can give parameters default values. These are called "keyword arguments".
class RRT(object):
def __init__(self, start, stop, obstacle_list, rand_area,
exapandDis=1.0, goalSampleRate=.05, maxIter=500):
self.start = Node(start[0], start[1])
self.stop = Node(stop[0], stop[1])
self.min_rand = rand_area[0]
self.max_rand = rand_area[1]
self.exapandDis = expandDis # just like this one
self.goalSampleRate = goalSampleRate # just like this one
self.maxIter = maxIter # just like this one
self.obstacle_list = obstacle_list
Now, when someone creates an instance of the class, they must supply the positional arguments start, stop, obstacle_list, rand_area but the others are optional. For instance, if you want to keep defaults expandDis and goalSapleRate but want a different maxIter, you would do
RRT(1, 100, [], 66, maxIter=1000)

Python AttributeError when running nosetest

I have a file called Model.py that contains the code
class ModelCalibrator():
def __init__(self):
self.file1 = 'Mortality_Population.txt'
self.file2 = 'Deaths_1x1_adj.txt'
self.MaxAge = 101
self.MinAge = 18
self.basisAges = np.array([18, 50, 100])[np.newaxis]
self.mortalityData = PopulationData()
self.deathRateData = DeathRateData()
(self.age, self.phis) = computeBasisFunctions(ModelCalibrator)
def computeBasisFunctions(mc):
MaxAge = mc.MaxAge
MinAge = mc.MinAge
age = np.arange(MinAge, MaxAge)[np.newaxis]
basisAges = mc.basisAges
#calculations
...
return (age, phis)
In a separate test.py file I am running nosetests using the code
def testMC():
data = ModelCalibrator()
Phi = data.phis()
assert_equal(Phi[0], 1)
This keeps telling me that I have an attributeerror: type object 'ModelCalibrator' has no attributes 'MaxAge'. Can anyone tell me where I am going wrong please?

On this line, you are passing in the class instead of the object. Try replacing ModelCalibrator with self. The class is only a template for the object. self represents the current object with all of the properties set.
(self.age, self.phis) = computeBasisFunctions(self)
Alternatively, if you want these to be accessible without an object, you could set MaxAge and MinAge as class variables by moving them outside of the __init__ function, but inside the class as shown here.

Proper handling of spark broadcast variables in a Python class

I've been implementing a model with spark via a python class. I had some headaches calling class methods on a RDD defined in the class (see this question for details), but finally have made some progress. Here is an example of a class method I'm working with:
#staticmethod
def alpha_sampler(model):
# all the variables in this block are numpy arrays or floats
var_alpha = model.params.var_alpha
var_rating = model.params.var_rating
b = model.params.b
beta = model.params.beta
S = model.params.S
Z = model.params.Z
x_user_g0_inner_over_var = model.x_user_g0_inner_over_var
def _alpha_sampler(row):
feature_arr = row[2]
var_alpha_given_rest = 1/((1/var_alpha) + feature_arr.shape[0]*(1/var_rating))
i = row[0]
items = row[1]
O = row[3] - np.inner(feature_arr,b) - beta[items] - np.inner(S[i],Z[items])
E_alpha_given_rest = var_alpha_given_rest * (x_user_g0_inner_over_var[i] + O.sum()/var_rating)
return np.random.normal(E_alpha_given_rest,np.sqrt(var_alpha_given_rest))
return _alpha_sampler
As you can see, to avoid serialization errors, I define a static method that returns a function that is in turn applied to each row of an RDD (model is the parent class here, and this is called from within another method of model):
# self.grp_user is the RDD
self.params.alpha = np.array(self.grp_user.map(model.alpha_sampler(self)).collect())
Now, this all works fine, but is not leveraging Spark's broadcast variables at all. Ideally, all the variables I'm passing in this function (var_alpha, beta, S, etc.) could first be broadcast to the workers, so that I wasn't redundantly passing them as part of the map. But I'm not sure how to do this.
My question, then, is the following: How/where should I make these into broadcast variables such that they are available to the alpha_sampler function that I map to grp_user? One thing I believe will work would be to make them globals, e.g.
global var_alpha
var_alpha = sc.broadcast(model.params.var_alpha)
# and similarly for the other variables...
Then the alpha_sampler could be much simplified:
#staticmethod
def _alpha_sampler(row):
feature_arr = row[2]
var_alpha_given_rest = 1/((1/var_alpha.value) + feature_arr.shape[0]*(1/var_rating.value))
i = row[0]
items = row[1]
O = row[3] - np.inner(feature_arr,b.value) - beta.value[items] - np.inner(S.value[i],Z.value[items])
E_alpha_given_rest = var_alpha_given_rest * (x_user_g0_inner_over_var.value[i] + O.sum()/var_rating.value)
return np.random.normal(E_alpha_given_rest,np.sqrt(var_alpha_given_rest))
But of course this is really dangerous use of globals that I would like to avoid. Is there a better way that lets me leverage broadcast variables?

Assuming that variables you use here are simply scalars there is probably nothing to gain here from a performance perspective and using broadcast variables will make you code less readable but you can either pass a broadcast variable as an argument to the static method:
class model(object):
#staticmethod
def foobar(a_model, mu):
y = a_model.y
def _foobar(x):
return x - mu.value + y
return _foobar
def __init__(self, sc):
self.sc = sc
self.y = -1
self.rdd = self.sc.parallelize([1, 2, 3])
def get_mean(self):
return self.rdd.mean()
def run_foobar(self):
mu = self.sc.broadcast(self.get_mean())
self.data = self.rdd.map(model.foobar(self, mu))
or initialize it there:
class model(object):
#staticmethod
def foobar(a_model):
mu = a_model.sc.broadcast(a_model.get_mean())
y = a_model.y
def _foobar(x):
return x - mu.value + y
return _foobar
def __init__(self, sc):
self.sc = sc
self.y = -1
self.rdd = self.sc.parallelize([1, 2, 3])
def get_mean(self):
return self.rdd.mean()
def run_foobar(self):
self.data = self.rdd.map(model.foobar(self))

Python Class inheritance Constructor fail: What am I doing wrong?

I have a small Python OOP program in which 2 class, Flan and Outil inherit from a superclass Part.
My problem is when I call Flan everything works perfectly, however when I call Outil the program fails silently.
The Outil instance is created, but it lacks all the attributes it doesn't share with Part.
The Outil instance isn't added to Outil.list_instance_outils, nor to Part.list_instances.
class Outil(Part):
list_instance_outils = []
def __init___(self, name, part_type, nodes, elems):
Part.__init__(self, name, part_type, nodes, elems)
self.vect_norm = vectnorm(self.nodes[self.elems[0,1:]-1, 1:])
self.elset = Elset(self)
self.nset = Nset(self, refpoint=True, generate=False)
self.SPOS = Ab_surface(self, self.elset)
self.SNEG = Ab_surface(self, self.elset, type_surf='SNEG')
Outil.list_instance_outils.append(self)
Part.list_instances.append(self)
class Flan(Part):
list_instances_flans = []
def __init__(self, name, part_type, nodes, elems):
Part.__init__(self, name, part_type, nodes, elems)
self.vect_norm = vectnorm(self.nodes[self.elems[0,1:4]-1, 1:])
self.elset = Elset(self)
self.nset = Nset(self)
self.SPOS = Ab_surface(self, self.elset)
self.SNEG = Ab_surface(self, self.elset, type_surf='SNEG')
Flan.list_instances_flans.append(self)
Part.list_instances.append(self)
Both this Classes inherit from Part :
class Part():
list_instances = []
def __init__(self, name, part_type, nodes, elems):
self.name = name
self.name_instance = self.name + '-1'
self.part_type = part_type
self.elems = elems
self.nodes = nodes
offset = np.min(self.elems[:, 1:])-1
self.nodes[:, 0] -= offset
self.elems[:, 1:] -= offset
I cannot stress enough that I have no error message whatsoever.
What am I doing wrong here ?

You wrote __init__ with three trailing underscores instead of two in Outil.
Because of this, it doesn't get called -- Part.__init__ gets called instead. That's why the class is created but it lacks the attributes beyond what are in Part.

To solve this sort of problem, the best thing to do is to run the code through the debugger.
Get your classes into the python interpreter (import, paste, whatever you like), then call pdb: import pdb; pdb.run('Outil()'). You can now step through the code to see what is happening.

error with appending to a file and using an array

I have written this code:
class component(object):
def __init__(self,
name = None,
height = None,
width = None):
self.name = name
self.height = height
self.width = width
class system(object):
def __init__(self,
name = None,
lines = None,
*component):
self.name = name
self.component = component
if lines is None:
self.lines = []
else:
self.lines = lines
def writeTOFile(self,
*component):
self.component = component
line =" "
self.lines.append(line)
line= "#----------------------------------------- SYSTEM ---------------------------------------#"
self.lines.append(line)
Component1 = component ( name = 'C1',
height = 500,
width = 400)
Component2 = component ( name = 'C2',
height = 600,
width = 700)
system1 = system(Component1, Component2)
system1.writeTOFile(Component1, Component2)
and I get the error :
Traceback (most recent call last):
File "C:\Python27\Work\trial2.py", line 46, in <module>
system1.writeTOFile(Component1, Component2)
File "C:\Python27\Work\trial2.py", line 32, in writeTOFile
self.lines.append(line)
AttributeError: 'component' object has no attribute 'append'
And I don't really know how to fix it.
Also is there a way for defining my system1 as system(Component) where component = [Component1, Component2, ...Componentn] ?
Thanks in adavance

You've got things out of order in your __init__:
def __init__(self, *component, **kwargs):
self.name = kwargs.get('name')
self.component = component
self.lines = kwargs.get('lines', [])
Will work. You need lines and name to be after the * item that collects the component.
In Python 2, you can't then have named attributes after a *, so you need to instead use **kwargs and get('name') and get('lines') from the kwargs.
get just returns None if you don't supply a default, so you'll get self.name = None here. If you want to specify a default name, you can do
self.name = kwargs.get('name', 'defaultname')
like I did for lines.

in line 32 you use self.lines.append(line).
But lines is a member of the class system initialized with Component2, which type is the class component that does not have the method append.

The problem is in the fact that when defining system you pass Component1 as a line argument in constructor. Since python does all the operations he can and not checking for the argument types if the operation can be done legally, this passes.
Maybe it would be a nice idea in the system constructor to check if the given argument lines is really of type list, and maybe writing something like:
if lines is None or not isinstance(lines, list):
self.lines = []
else:
self.lines = lines
That way, you would know about the problem before you try appending to the non-list object.
And as for the second part of your question, you can do it exactly like you suggested:
system1 = system([Component1, Component2, MyComponent], [])
(if you, for example, want to make a system with 3 components, and an empty list as an "console" for lines)