I need to create/declare 50 functions inside my Class. But, I need to do it dynamically with custom function names and cutom urls in the function body. Something as the following:
for item_nr in range(1, 51):
#task(1)
def view_item_with_id_{item_nr}(self, item_nr=item_nr):
self.client.get(
url=f"/my-url/id=SEPT24_00{item_nr}",
verify=False,
auth=(os.environ['USERNAME'], os.environ['PASSWORD'])
)
P.S since it's inside a class- I cannot really use another function to generate it as suggested in some other threads, because the 'self' parameter will not be visible then. Example (this will not work):
def function_builder(args):
def function(more_args):
#do stuff based on the values of args
return function
Any help is appreciated
This is an interesting question, but is there some reason you cant just use a single #task with a loop? (this solution is of course specific to Locust)
#task
def view_item_with_id(self):
for item_nr in range(1, 51):
self.client.get(
url=f"/my-url/id=SEPT24_00{item_nr}",
verify=False,
auth=(os.environ['USERNAME'], os.environ['PASSWORD'])
)
The other suggested answers might be closer to your original question, but this is so much simpler (it can of course easily be adjusted to pick items randomly, if that is your preference)
You could use setattr to define new class methods:
class Test:
def __init__(self):
for item_nr in range(1, 51):
setattr(self, f"view_item_with_id_{item_nr}", self.itemViewer)
#task(1)
def itemViewer(self, item_nr=None):
"""
self.client.get(
url=f"/my-url/id=SEPT24_00{item_nr}",
verify=False,
auth=(os.environ['USERNAME'], os.environ['PASSWORD'])
)
"""
print(f"/my-url/id=SEPT24_00{item_nr}")
t = Test()
t.view_item_with_id_14(14)
t.view_item_with_id_44(44)
Out:
/my-url/id=SEPT24_0014
/my-url/id=SEPT24_0044
I have to mention: Fundamentally, wanting to do this in the first place is almost certainly a mistake. You should surely, instead, modify the part of the code that calls these functions to just call one function and pass in a parameter, or otherwise work around the problem.
There are two parts to this question: how to generate the functions, and how to wrap them in decorators.
Create methods programmatically
To create methods programmatically, you need to use setattr(). This allows you to add attributes to an object specified by string rather than explicitly in your code. However, for a function added in this way to be treated as a method (i.e. your object is automatically passed as the self parameter), it must set on the class, not directly on the object.
class Foo:
pass
def f(self, val):
self.x = val
setattr(Foo, "f", f)
def g(self):
print(self.x)
setattr(Foo, "g", g)
obj = Foo()
obj.f(3)
obj.g() # prints 3
Call decorator manually
The second part is the easy bit. A decorator is just some syntactic sugar on a function that receives and returns another function. So these are equivalent:
#task(1)
def foo():
pass
def bar():
pass
bar = task(1)(bar)
In your case, you can simply call the decorator directly rather than having to use anything like the #task(1) notation.
Putting it together
Putting the two ideas together, here's what you want. (By the way, your default parameter technique is fine for creating the function, but functools.partialmethod is a bit cleaner because the result doesn't allow the caller to override the parameter, so I've used that below.)
def MyClass
pass # Whatever else you need
def fn(self, item_nr):
self.client.get(
url=f"/my-url/id=SEPT24_00{item_nr}",
verify=False,
auth=(os.environ['USERNAME'], os.environ['PASSWORD'])
)
for item_nr in range(1, 51):
this_fn = functools.partialmethod(fn, item_nr)
decorated_fn = task(1)(this_fn)
setattr(MyClass, f"view_item_with_id_{item_nr}", decorated_fn)
Related
I'm creating a class and I'm hoping to call a user-defined function within a method for that class. I'd also like to define the function within the class definition. However, when I call the class, I get the error message name *whatever function* is not defined.
For instance, something like this:
class ExampleClass():
def __init__(self, number):
self.number = number
def plus_2_times_4(x):
return(4*(x + 2))
def arithmetic(self):
return(plus_2_times_4(self.number))
But when I call:
instance = ExampleClass(number = 4)
instance.arithmetic()
I get the error message.
So basically I want to define the function in one step (def plus_2_times_4) and use the function when defining a method in another step (def arithmetic...). Is this possible?
Thanks so much in advance!
Define and call plus_2_times_4 with self, namely:
class ExampleClass():
def __init__(self, number):
self.number = number
def plus_2_times_4(self,x):
return(4*(x + 2))
def arithmetic(self):
return(self.plus_2_times_4(self.number))
This will work.
Call the method using ExampleClass.plus_2_times_4:
class ExampleClass():
def __init__(self, number):
self.number = number
def plus_2_times_4(x):
return(4*(x + 2))
def arithmetic(self):
return(ExampleClass.plus_2_times_4(self.number))
Alternatively, use the #staticmethod decorator and call the method using the normal method calling syntax:
class ExampleClass():
def __init__(self, number):
self.number = number
#staticmethod
def plus_2_times_4(x):
return(4*(x + 2))
def arithmetic(self):
return(self.plus_2_times_4(self.number))
The #staticmethod decorator ensures that self will never be implicitly passed in, like it normally is for methods.
Look at your plus_2_times_4 and arithmetic definitions. There’s no way for Python to tell that you wanted one of them to be a local function and the other one to be a method. They’re both defined exactly the same way.
And really, they’re both. In Python, anything you put in a class statement body is local while that class definition is happening, and it becomes a class attribute later.
If you want to be able to call the function as plus_2_times_4 later, you don’t want this. You just want to declare a global function, outside the class definition. And that really does seem like what you want here. The function doesn’t have any inherent connection to the class; it just takes a number and does stuff to that number without any thought of anything about your class.
Or, if you don’t want to “pollute the global namespace”, you can just define it as a local function within arithmetic. Then arithmetic can just call it—and nobody else can.
If, on the other hand, you want it to be a method, you have to make it usable as a method. A normal instance method has to take self as an extra first parameter, even if it’s not going to do anything with self. (Although not doing anything with self is usually a sign that you wanted a global function, not a method, it’s not illegal or anything.) And it has to be called on an instance, like self.plus_2_times_4(…).
You could declare it as a static method by adding the #staticmethod decorator. Then you don’t need to add the useless self parameter. But you still need to call it on an instance or on the class, because it’s still an attribute of the class, not a global name. (You could also use #classmethod if you have some idea of wanting subclasses to override it, but that doesn’t seem likely here.)
What if you really want to just capture the function value so you can call it without going through the class? Well, you could make it the default value of a parameter, like this:
def arithmetic(self, *, _func=plus_2_times_4):
return func(self.value)
Default values are captured at function definition time—that is, while the class is still being defined—so the function is still local there and can be captured there. But if this seems weird and ugly, there’s a good reason for that—this is not something you usually want to do. To a reader, the function still looks like an incorrect method rather than a disposable function needed by arithmetic. It even ends up as a member of the class, but it can’t be called normally. This is all pretty misleading. In the rare cases you need this, you probably want to give it a _private name, and del it once you’ve used it.
I have a function foo that takes a parameter stuff
Stuff can be something in a database and I'd like to create a function that takes a stuff_id, get the stuff from the db, execute foo.
Here's my attempt to solve it:
1/ Create a second function with suffix from_stuff_id
def foo(stuff):
do something
def foo_from_stuff_id(stuff_id):
stuff = get_stuff(stuff_id)
foo(stuff)
2/ Modify the first function
def foo(stuff=None, stuff_id=None):
if stuff_id:
stuff = get_stuff(stuff_id)
do something
I don't like both ways.
What's the most pythonic way to do it ?
Assuming foo is the main component of your application, your first way. Each function should have a different purpose. The moment you combine multiple purposes into a single function, you can easily get lost in long streams of code.
If, however, some other function can also provide stuff, then go with the second.
The only thing I would add is make sure you add docstrings (PEP-257) to each function to explain in words the role of the function. If necessary, you can also add comments to your code.
I'm not a big fan of type overloading in Python, but this is one of the cases where I might go for it if there's really a need:
def foo(stuff):
if isinstance(stuff, int):
stuff = get_stuff(stuff)
...
With type annotations it would look like this:
def foo(stuff: Union[int, Stuff]):
if isinstance(stuff, int):
stuff = get_stuff(stuff)
...
It basically depends on how you've defined all these functions. If you're importing get_stuff from another module the second approach is more Pythonic, because from an OOP perspective you create functions for doing one particular purpose and in this case when you've already defined the get_stuff you don't need to call it within another function.
If get_stuff it's not defined in another module, then it depends on whether you are using classes or not. If you're using a class and you want to use all these modules together you can use a method for either accessing or connecting to the data base and use that method within other methods like foo.
Example:
from some module import get_stuff
MyClass:
def __init__(self, *args, **kwargs):
# ...
self.stuff_id = kwargs['stuff_id']
def foo(self):
stuff = get_stuff(self.stuff_id)
# do stuff
Or if the functionality of foo depends on the existence of stuff you can have a global stuff and simply check for its validation :
MyClass:
def __init__(self, *args, **kwargs):
# ...
_stuff_id = kwargs['stuff_id']
self.stuff = get_stuff(_stuff_id) # can return None
def foo(self):
if self.stuff:
# do stuff
else:
# do other stuff
Or another neat design pattern for such situations might be using a dispatcher function (or method in class) that delegates the execution to different functions based on the state of stuff.
def delegator(stff, stuff_id):
if stuff: # or other condition
foo(stuff)
else:
get_stuff(stuff_id)
I am trying to make a class that will make api requests, retrying based on configuration options passed in to the retrying.retry decorator, and handle different error codes in the correct way for each job.
Here is my code:
from retrying import retry
class APIRequester:
def __init__(self, url, **kwargs):
self.url = url
self.retry_kwargs = kwargs
#retry(**self.retry_kwargs) # Obviously doesn't know what self is
def make_request(self):
pass
How can I pass in parameters to this method decorator? I tried making them a class attribute, which didn't work either.
A couple of notes/questions:
The #retry decorator will be applied to the make_request method at the time the class is created, while retry_kwargs will only become available when an instance of the class is created, and thus the former must precede the latter.
In which case, the former cannot depend on information that becomes available in the latter, ... as long as you use the decorator syntax ...
The decorator syntax
#decorator
def xxx(...):
...
is just syntax sugar for
def xxx(...):
...
xxx = decorate(xxx)
which means that, along with the fact that Python is very dynamic, you could force the issue by doing something like
class APIRequester:
def __init__(self, url, **kwargs):
self.url = url
self.retry_kwargs = kwargs
APIRequester.make_request = retry(**kwargs)(APIRequester.make_request)
def make_request(self):
pass
Whether this particular decorator chokes on the self parameter or not, I cannot tell you.
Will you have more than one instance of APIRequester? If so, note that the method will be re-decorated each time a new instance is created: can this work sensibly? (I doubt it.) But see the edit below ...
If you do not have more that one instance, then you probably don't need to rely on information that becomes availale at the singleton's construction time.
The above were some general Python principles. I doubt that you really want to force the issue in this case. It seems to me that you are trying to use the decorator in a way that it was not designed to be used.
Edit: instancemethods
If you replace the line that does the decorating in the constructor with
self.make_request = retry(**kwargs)(self.make_request)
then each instance will get its own decorated version of the function. This should avoid any problems with re-decoration of the same function. There may will still be problems with self getting in the way. In that case, you could remove the self parameter from the definition and wrap it with staticmethod:
self.make_request = retry(**kwargs)(staticmethod(self.make_request))
Or better still, use decorator syntax to apply staticmethod to make_request at the place where you define it, the way Guido inteded it.
Like this, it even stands a chance of working! :-)
Decorator is just a syntax sugar for func=decorator(func). You can do the assignment yourself:
class APIRequester:
def __init__(self, url, **kwargs):
self.url = url
self.make_request = retry(**kwargs)(self.make_request)
def make_request(self):
pass
This will internally replace a method (descriptor) by a function, but it will work as expected.
Of course self is available in the decorator at the call time. See answers to How to decorate a method inside a class? , on which I based my answer here:
def my_retry(fn):
from functools import wraps
#wraps(fn)
def wrapped(self):
print(self.retry_kwargs)
for i in range(self.retry_kwargs["times"]):
# you have total control
fn(self)
# around your method. Can even call it multiple times,
# call with original retry:
retry(**self.retry_kwargs)(fn)(self)
# check exceptions, return some value (here None), etc
#
return wrapped
class APIRequester(object):
def __init__(self, url, **kwargs):
self.url = url
self.retry_kwargs = kwargs
#my_retry
def make_request(self):
print("method")
a = APIRequester('http://something', times=3)
a.make_request()
That is, original decorator is wrapped with a new, configuration-aware decorator. No need to change the constructor, syntax remains simple.
Retry decorator doesn't support class method, because instance of the class is implicitly passed to function.
Please decorate normal function.
If you want to wrap function into class, please decorate static method.
I have a decorator which simply caches return values (called #cached in my example) and I wish to use it in conjunction with #property. This works just fine normally. The problem I am facing occurs when I try and use an expire attribute added by #cached.
def cached(f):
cache = [None]
def inner(*args, **kwargs):
if cache[0]:
cache[0] = f(*args, **kwargs)
return cache[0]
def expire():
cache[0] = None
inner.expire = expire
return inner
class Example(object):
#property
#cached
def something_expensive(self):
print("expensive")
return "hello"
e = Example()
e.something_expensive
e.something_expensive.expire()
How am I able to get access to the expire function? added to the function after its replaced by #property. I understand why this doesn't work I am interested in a way of working around that problem.
Some restrictions:
I cannot change the #cached decorator its in a library I don't control
I would really rather not remove #property because I want to expire in my unit tests and they make my code much nicer to use.
One solution that I think is rather bad is (because In reality I have a lot of properties that I want to do this for):
class Example(object):
#cached
def _something_expensive(self):
return "hello"
#property
def something_expensive(self):
return self._something_expensive()
You can access it using the class dictionary:
type(e).__dict__['something_expensive'].fget.expire()
In general e.something_expensive is equivalent to:
type(e).__dict__['something_expensive'].__get__(e, type(e))
For more details read up: Descriptor HowTo Guide
Note that inside the expiry function you're not setting cache from the outer function cached function as None, you're simply creating a new local variable. You may want to do something like this:
def expire():
del cache[:]
cache.append(None)
In Python 3 it's even easier to update cache using the nonlocal keyword.
I'm working on a class that basically allows for method chaining, for setting some attrbutes for different dictionaries stored.
The syntax is as follows:
d = Test()
d.connect().setAttrbutes(Message=Blah, Circle=True, Key=True)
But there can also be other instances, so, for example:
d = Test()
d.initialise().setAttrbutes(Message=Blah)
Now I believe that I can overwrite the "setattrbutes" function; I just don't want to create a function for each of the dictionary. Instead I want to capture the name of the previous chained function. So in the example above I would then be given "connect" and "initialise" so I know which dictionary to store these inside.
I hope this makes sense. Any ideas would be greatly appreciated :)
EDIT:
Would this work / Be a good work-around the above problem:
Using method overloading, I can have the following methods:
def setAttrbutes(self, Name="Foo", Message="", Circle=False):
print "Attrbutes method called for 'Foo'"
def setAttrbutes(self, Name="Boo", Message=""):
print "Attrbutes method called for 'Boo'"
So therefore, I can say which method to call depends on the name that is used. For example, in main, if I have the following:
d.setAttrbutes(Name="Foo", Message="Hello world", Circle=True) # this will call the first
d.setAttrbutes(Name="Boo", Message="Hello world") # this will call the second
Would this work, and, if not, why?
This is almost certainly a bad idea… but it is doable, in a few different ways.
Most simply, you can just have each function save its name in the object, e.g.:
def stash_name(func):
#functools.wraps(func)
def wrapper(self, *args, **kwargs):
self._stashed_name = func.__name__
return func(self, *args, **kwargs)
return wrapper
class Test(object):
#stash_name
def foo(self, x):
print x
#stash_name
def bar(self):
print
Now, after calling d.connect(), d._stashed_name will be "connect".
At the opposite extreme, if you want to get really hacky, you can do this without any cooperation from the preceding method. Just use sys._getframe(1) to find your calling context, then you can examine the frame's f_code to see how you were called.
You can use the dis module to see real bytecode. But basically, it will looks like this pseudo-bytecode:
LOAD_NAME d
LOAD_ATTR connect
<possibly other ops to prepare arguments>
CALL_FUNCTION 1 (or any other CALL_FUNCTION_* variant)
LOAD_ATTR setAttributes
<various other ops to prepare arguments>
CALL_FUNCTION 0
In this case, you can either get the attribute name from the LOAD_ATTR, or get the value that was pushed and look at its im_func.__name__, depending which one you want.
Of course there will be other cases that don't look like this. For example, let's say I called it as getattr(d, ''.join('con', 'next'))() instead of d.connect(). Or I looked up the unbound method and built a bound method on the fly. Or… What would you want to do in each such case? If you have the answers to all such cases, then you can work out the rule that generates those answers, then figure out how to get that from the bytecode.
Since you tacked on a second, completely different, question, here's a second answer.
Would this work / Be a good work-around the above problem:
Using method overloading, I can have the following methods:
No, you can't. Python does not have method overloading. If you def a method with the same name as a previous method, it just replaces the first one entirely.
There are ways to simulate method overloading by dispatching on the argument values manually within the method body. For example:
def _setAttrbutes_impl1(self, Name, Message, Circle):
pass
def _setAttrbutes_impl2(self, Name, Message):
pass
def setAttrbutes(self, Name=None, Message="", Circle=None):
if Circle is None:
return _setAttrbutes_impl2("Boo" if Name is None else Name, Message)
else:
return _setAttrbutes_impl1("Foo" if Name is None else Name, Message, Circle)
But this is rarely useful.