I have a very expensive method that returns 2 values, and it is called by class A. Since it is expensive, I made the 2 values lazy evaluated, using properties. Since I don't want to call the very_expensive_function 2 times, the first time the user wants to access one of the 2 values, I save both.
So far I wrote this:
class A:
def __init__(self):
self._attr1 = None
self._attr2 = None
#property
def attr1(self):
self.calculate_metrics()
return self._attr1
#property
def attr2(self):
self.calculate_metrics()
return self._attr2
def calculate_metrics():
if self._attr1 is None:
attr1, attr2 = very_expensive_call()
self._attr1 = attr1
self._attr2 = attr2
As you can see, the first time the user access to attr1 or attr2, I save both. Is it correct or is it possible in another way? It seems very strange to have that calculate_metrics() copy-pasted every time.
Memoization is, simply put, remembering if you have already called a function with particular arguments. If you have it simply returns the already calculated return value rather than calculating it again.
import time
def long_calculation(x, y, memo={}):
try:
result = memo[x, y] # already calculated!
except KeyError:
# make long_calculation take a long time!
time.sleep(2)
result = x * y
memo[x, y] = result
return result
The dictionary memo is able to remember calls to the function because it is evaluated when the function is first loaded: every call to long_calculation shares the same memo dictionary.
To test this try:
# Note that (2,2) (7,8) and (10,10) are repeated here:
test_values = ((2,2),(4,5),(2,2),(7,8),(2,3),(7,8),(10,11),(4,5),(10,10),(10,10))
for values in test_values:
start = time.time()
res = long_calculation(*values)
end = time.time()
elapsed = end-start
print(values,' calculated in ',elapsed, "seconds")
It should be fairly easy to insert this kind of code into your class. If you always need the attributes calculated then you can put the call in __init__.
Related
i have 2 questions:
1-Below code in the forward(self,x) method, it gets an x when the class is called. Inside of this forward(self,x) method there is a property is returning (self._forward). But there is an x variable is given to the self._forward as if it is a function. How does it work?
2- After -somehow- x value is given to the self._forward property, it chooses a policy in the policies list. I am assuming that i understood the answer of the first question and again somehow our x value that it given to self._forward goes into the here -> policies[here] and it chose a self.polcy1 or 2 or 3. How does it run the self policy1 or 2 or 3 functions without taking the value?
Sorry for my complicated explanation but a lot of things doesn't make sense here.
class SpecAugment(nn.Module):
def __init__(self,rate,policy=3,freq_mask=2,time_mask=4):
super(SpecAugment, self).__init__()
self.rate = rate
self.specaug1 = nn.Sequential(
torchaudio.transforms.FrequencyMasking(freq_mask_param=freq_mask),
torchaudio.transforms.TimeMasking(time_mask_param=time_mask)
)
self.specaug2 = nn.Sequential(
torchaudio.transforms.FrequencyMasking(freq_mask_param=freq_mask),
torchaudio.transforms.TimeMasking(time_mask_param=time_mask),
torchaudio.transforms.FrequencyMasking(freq_mask_param=freq_mask),
torchaudio.transforms.TimeMasking(time_mask_param=time_mask)
)
policies = {1:self.policy1, 2:self.policy2, 3:self.policy3}
self._forward = policies[policy]
def forward(self,x):
return self._forward(x)
#this makes specaug1
def policy1(self,x):
probability = torch.rand(1,1).item()
if self.rate > probability:
return self.specaug1(x)
return x
#this makes specaug2
def policy2(self,x):
probability = torch.rand(1,1).item()
if self.rate > probability:
return self.specaug2(x)
return x
#this makes random choice because we did torch.rand
def policy3(self,x):
probability = torch.rand(1,1).item()
if probability > 0.5:
return self.policy1(x)
return self.policy2(x)
Let's try to see what's happening here.
When you create an instance of SpecAugment, call it sa, the __init__() assigns a few properties:
sa.rate contains the rate value
sa.specaug1 and sa.specaug2 contain whatever was returned by the calls to nn.Sequential()
sa._forward contains a callable: policy1() or 2 or 3 depending on the policy value you passed in
When you call sa.forward(x), the callable from sa._forward is called with the x parameter. This in turn returns x or the result of a call to sa.specaug1 or 2, depending on the values of probabibility, sa.rate, and sa._forward.
Note that if sa.specaug1 (or sa.specaug2) isn't a callable you will get an error
Let's say I have a simple class, with an attribute, x:
class A:
def __init__(self):
self.x = random.randint(-5, 5) # not the most efficient, but it serves purposes well
I'll also have a list, with hundreds of instances of this class:
Az = []
for i in range(150):
Az.append(A())
Now, let's say I want to loop through all the As in Az, and run a function on the classes who's x attribute is equivalent to less than one. This is one way, but alas, is very inefficient:
for cls in Az:
if cls.x<1:
func1(cls) # A random function that accepts a class as a parameter, and does something to it
So, to wrap it up, my question: How to optimize the speed of the checking?
Optimizing only the third step is tricky. Why not start at the second step by saving list ids of classes where attribute x is <1?
Az = []
ids = []
for i, id in enumerate(range(150)):
cls = A()
if cls.x < 1:
ids.append(id)
Az.append(cls)
And then modify the third step:
for id in ids:
func1(Az[id])
I have a module called preparation.py which verifies the arguments passed to a function, if it isn't present, instead of using a pre-stabilished value as a keyword argument, it searches the argument as an attribute of an object. The code is the following:
def prep(argslist, argsprovided, attributes):
argsout = []
for name in argslist:
if name in argsprovided:
argsout.append(argsprovided[name])
else:
argsout.append(getattr(attributes,name))
return argsout
A simple use of this would be:
import preparation as prep
class example(object):
def __init__(self,x,y):
self.x = x
self.y = y
E = example(1,1)
def foo(**kwargs):
[x,y] = prep.prep(['x','y'],kwargs,E)
return x + y
print( foo())
print( foo(x = 2))
Since almost every function in my code does this check everytime it's called, I want to know if the time spent on it is considerable. The time spent on a single time when this module is called can't be measured using time.time() method, so I just can't sum a bunch of smaller intervals. Is there a way of doing this?
Sorry for the title, I hope it reflects correctly my problem :
In the following code, I was expecting the result to be result 0 1 2 but instead I have 2 2 2. The code inside my_function seems to be interpreted with the last instance of obj. What is wrong ?
class Example:
def __init__(self, x):
self.x = x
def get(self):
return self.x
a_list = []
for index in range(3):
obj = Example(index)
def my_function(x):
#some stuff with x like obj.another_function(x)
return obj.get()
a_list.append(my_function)
for c in a_list:
print(c())
When you define this
def my_function():
return obj.get()
Python will understand that my_function should run the get() method of an object called obj and return the value. It won't know the value of obj and what the get() method does until you attempt to call it.
So, you are actually defining three different functions that will eventually do the same thing. And, in the end, running the same code thrice.
But why is the return 2 2 2?
Because after the last iteration, the value of obj is Example(2)* because you redefine its value at every iteration, and the last one remains.
*
because of this line obj = Example(index)
Understanding a few things about how python works will help you understand what's happening here. Here obj is a closure, closures are evaluated at call time, not when the function is defined so if I do this:
x = "hello"
def printX():
print x
x = "goodbye"
printX() # goodbye
I get "goodbye" because printX is referencing a global variable in my module, which changes after I create printX.
What you want to do is create a function with a closure that references a specific object. The functional way to do this is to create a function that returns another function:
x = "hello"
def makePrintX(a):
def printX():
# We print a, the object passed to `makePrintX`
print a
return printX
# x is evaluated here when it is still "hello"
myPrintX = makePrintX(x)
x = "goodbye"
myPrintX() # "hello"
If you're having trouble understanding the above example I would recommend reading up on python's scoping rules. For your example, you could do something like this:
class Example:
def __init__(self, x):
self.x = x
def get(self):
return self.x
def makeObjFunction(obj):
def objFunction(x):
return obj.get()
return objFunction
a_list = []
for index in range(3):
obj = Example(index)
my_function = makeObjFunction(obj)
a_list.append(my_function)
for c in a_list:
print(c("some value"))
You are appending three my_functions to the a_list which are all closures over the same Example object. Try:
def my_function():
return obj
<__main__.Example object at 0x0054EDF0>
<__main__.Example object at 0x0054EDF0>
<__main__.Example object at 0x0054EDF0>
You can see they have the same id so calling get() on each should give the same answer.
If you just append the obj.get function (and drop the my_function) it'll work fine.
a_list.append(obj.get)
....
0
1
2
Edit: You've updated your question so to let you do more stuff in my_function(). It's still basically a scoping problem.
def my_func_factory(p_obj):
def my_function(x):
#some stuff with x like obj.another_function(x)
return p_obj.get()
return my_function
for index in range(3):
obj = Example(index)
a_list.append(my_func_factory(obj))
Since my_function can't see obj being reassigned, each instance doesn't pick up the change.
I think append() during the for just append the function address in a_list[]. After for iteration, the a_list is really given the number. Then it discovers the address of my_function, and they get the number in my_function, this is, 2. That's why you get [2,2,2].
Or maybe, in my_function, function give the method of "obj". But for iteration change the "obj" memory address each time, so the symbol "obj" always aim to the newest object Example. Due to my_function always get "obj", you get the same number from the last object.
in my Python code I have the following issue: i have to copy the same object many times and then pass each copy to a function that modifies it. I tried with copy.deepcopy, but it's really computationally expensive, then i tried with itertools.repeat(), but it was a bad idea because after that i've to modify the object. So i wrote a simple method that copy an object simply returning a new object with the same attributes:
def myCopy(myObj):
return MyClass(myObj.x, myObj.y)
The problem is that this is really unefficient too: i've to make it abaout 6000 times and it takes more than 10 seconds! So, does exist a better way to do that?
The object to copy and modify is table, that is created like that:
def initialState(self):
table = []
[table.append(Events()) for _ in xrange(self.numSlots)]
for ei in xrange(self.numEvents - 1):
ei += 1
enr = self.exams[ei]
k = random.randint(0, self.numSlots - 1)
table[k].Insert(ei, enr)
x = EtState(table)
return x
class Event:
def __init__(self, i, enrollment, contribution = None):
self.ei = i
self.enrollment = enrollment
self.contribution = contribution
class Events:
def __init__(self):
self.count = 0
self.EventList = []
def getEvent(self, i):
return self.EventList[i].ei
def getEnrollment(self, i):
return self.EventList[i].enrollment
def Insert(self, ei, enroll = 1, contribution = None):
self.EventList.append(Event(ei, enroll, contribution))
self.count += 1
def eventIn(self, ei):
for x in xrange(self.count):
if(self.EventList[x].ei == ei):
self.EventList[x].enrollment += 1
return True
return False
More Pythonic way would be to create function(s) that modify the object, but don't modify the original object, just return its modified form. But from this code you posted, it is not clear what are you acutally trying to do, you should make a more simple (generic) example of what are you trying to do.
Since Object in Python means anything, class, instance, dict, list, tuple, 'a', etc..
to copy object is kind of not clear...
You mean copy instance of a Class if I understood it correctly
So write a function that takes one instance of that class, in that function create another instance and copy all atributes you need..