I'm trying to add flexibility to a python class, so that it notices when one of the init arguments is already an instance of that class. Skip "Initial situation" if you don't mind, how I got here.
Initial situation
I have this class:
class Pet:
def __init__(self, animal):
self._animal = animal
#property
def present(self):
return "This pet is a " + self._animal
...
and there are many functions which accept an instance of this class as an argument (def f(pet, ...)). Everything worked as expected.
I then wanted to add some flexibility to the usage of these functions: if the caller passes a Pet instance, everything keeps on working as before. In all other cases, a Pet instance is created. One way to achieve that, is like this:
def f(pet_or_animal, ...):
if isinstance(pet_or_animal, Pet): #Pet instance was passed
pet = pet_or_animal
else: #animal string was passed
pet = Pet(pet_or_animal)
...
This also works as expected, but these lines are repeated in every function. Not DRY, not good.
Goal
So, I'd like to extract the if/else from each of the functions, and integrate it into the Pet class itself. I tried changing its __init__ method to
class PetA: #I've changed the name to facilitate discussion here.
def __init__(self, pet_or_animal):
if isinstance(pet_or_animal, PetA):
self = pet_or_animal
else:
self._animal = pet_or_animal
...
and start each function with
def f(pet_or_animal, ...):
pet = PetA(pet_or_animal)
...
However, that is not working. If a Pet instance is passed, everything is good, but if a string is called, a Pet instance is not correctly created.
Current (ugly) solution
What is working, is to add a class method to the class, like so:
class PetB: #I've changed the name to facilitate discussion here.
#classmethod
def init(cls, pet_or_animal):
if isinstance(pet_or_animal, PetB):
return pet_or_animal
else:
return cls(pet_or_animal)
def __init__(self, animal):
self._animal = animal
...
and also change the functions to
def f(pet_or_animal, ...):
pet = PetB.init(pet_or_animal) #ugly
...
Questions
Does anyone know, how to change class PetA so, that it has the intended behavior? To be sure, here is the quick test:
pb1 = PetB.init('dog')
pb2 = PetB.init(pb1) #correctly initialized; points to same instance as pb1 (as desired)
pa1 = PetA('cat')
pa2 = PetA(pa1) #incorrectly initialized; pa1 != pa2
More generally, is this the right way to go about adding this flexibility? Another option I considered was writing a separate function to just do the checking, but this too is rather ugly and yet another thing to keep track of. I'd rather keep everything neat and wrapped in the class itself.
And one final remark: I realize that some people might find the added class method (petB) a more elegant solution. The reason I prefer to add to the __init__ method (petA) is that, in my real-world use, I already allow for many different types of initialization arguments. So, there is already a list of if/elif/elif/... statements that check, just which of the possibilities is used by the creator. I'd like to extend that by one more case, namely, if an initialized instance is passed.
Many thanks
I believe your current "ugly" solution is actually the correct approach.
This pushes the flexibility up as far as possible, since it is messy. Even though python allows for arbitrary types and values to float around, your users and yourself will thank you for keeping that constrained to the outermost levels.
I would think of it as (don't need to implement it this way)
class Pet:
#classmethod
def from_animal(cls, ...):
...
#classmethod
def from_pet(cls, ...):
...
#classmethod
def auto(cls, ...):
if is_pet(...):
return cls.from_pet(...)
def __init__(cls, internal_rep):
...
etc.
It is a code smell if you don't know whether your function is taking an object or an initializer. See if you can do processing as up-front as possible with user input and standardize everything beyond there.
You could use a function instead to get the same behaviour you want:
def make_pet_if_required(pet_or_animal):
if isinstance(pet_or_animal, PetA):
return pet_or_animal
else:
return Pet(pet_or_animal)
And then:
def f(pet_or_animal, ...):
pet = make_pet_if_required(pet_or_animal)
...
For more "beauty" you can try turning that function call into a decorator.
Related
Description & What I've tried:
I have seen many posts in stackoverflow about binding methods to class instances (I'm aware there are bunch of duplicates already).
However I havent found a discussion referring to binding a method to the class itself. I can think of workarounds but I'm curious if there is a simple way to achieve following:
import types
def quacks(some_class):
def quack(self, number_of_quacks):
self.number_of_quacks = number_of_quacks
setattr(some_class, "quack", types.MethodType(quack, some_class))
return some_class
#quacks
class Duck:
pass
but above would not work:
d1 = Duck()
d2 = Duck()
d1.quack(1)
d2.quack(2)
print(d2.number_of_quacks)
# 2
print(d1.number_of_quacks)
# 2
because quack is actually modifying the class itself rather than the instance.
There are two workarounds I can think of. Either something like below:
class Duck:
def __init__(self):
setattr(self, "quack", types.MethodType(quack, self))
or something like
class Quacks:
def quack(self, number_of_quacks):
self.number_of_quacks = number_of_quacks
class Duck(Quacks):
pass
Question:
So my question is, is there a simple way to achieve the simple #quacks class decorator I described above?
Why I'm asking:
I intend to create a set of functions to modularly add common methods I use to classes. If I dont quit this project, the list is likely to grow over time and I would prefer to have it look nice on code definition. And as a matter of taste, I think option 1 below looks nicer than option 2:
# option 1
#quacks
#walks
#has_wings
#is_white
#stuff
class Duck:
pass
# option 2
class Duck(
Quacks,
Walks,
HasWings,
IsWhite,
Stuff):
pass
If you don't mind changing your desired syntax completely to get the functionality you want, you can dynamically construct classes with type (see second signature).
The first argument is the name of the class, the second is a tuple of superclasses, and the third is a dictionary of attributes to add.
Duck = type("Duck", (), {
"quack", quack_function,
"walk", walk_function,
...
})
So, instead of decorators that inject the appropriate functionality after creation, you are simply adding the functionality directly at the time of creation. The nice thing about this method is that you can programatically build the attribute dictionary, whereas with decorators you cannot.
Found another workaround, I guess below would do it for me.
def quacks(some_class):
def quack(self, number_of_quacks):
self.number_of_quacks = number_of_quacks
old__init__ = some_class.__init__
def new__init__(self, *args, **kwargs):
setattr(self, "quack", types.MethodType(quack, self))
old__init__(self, *args, **kwargs)
setattr(some_class, "__init__", new__init__)
return some_class
Feel free to add any other alternatives, or if you see any drawbacks with this approach.
Edit: a less hacky way inspired from #SethMMorton's answer:
def quack(self, number_of_quacks):
self.number_of_quacks = number_of_quacks
def add_mixin(some_class, some_fn):
new_class = type(some_class.__name__, (some_class,), {
some_fn.__name__: some_fn
})
return new_class
def quacks(some_class):
return add_mixin(some_class, quack)
#quacks
class Duck:
pass
d1 = Duck()
d2 = Duck()
d1.quack(1)
d2.quack(2)
print(d1.number_of_quacks)
print(d2.number_of_quacks)
I wonder if there is a possibility to split jupyter classes into different cells? Lets say:
#first cell:
class foo(object):
def __init__(self, var):
self.var = var
#second cell
def print_var(self):
print(self.var)
For more complex classes its really annoying to write them into one cell.
I would like to put each method in a different cell.
Someone made this this last year but i wonder if there is something build in so i dont need external scripts/imports.
And if not, i would like to know if there is a reason to not give the opportunity to split your code and document / debug it way easier.
Thanks in advance
Two solutions were provided to this problem on Github issue "Define a Python class across multiple cells #1243" which can be found here: https://github.com/jupyter/notebook/issues/1243
One solution is using a magic function from a package developed for this specific case called jdc - or Jupyter dynamic classes. The documentation on how to install it and how to use can be found on package url at https://alexhagen.github.io/jdc/
The second solution was provided by Doug Blank and which just work in regular Python, without resorting to any extra magic as follows:
Cell 1:
class MyClass():
def method1(self):
print("method1")
Cell 2:
class MyClass(MyClass):
def method2(self):
print("method2")
Cell 3:
instance = MyClass()
instance.method1()
instance.method2()
I tested the second solution myself in both Jupyter Notebook and VS Code, and it worked fine in both environments, except that I got a pylint error [pylint] E0102:class already defined line 5 in VS Code, which is kind of expected but still runs fine. Moreover, VS Code was not meant to be the target environment anyway.
I don't feel like that whole stuff to be a issue or a good idea... But maybe the following will work for you:
# First cell
class Foo(object):
pass
# Other cell
def __init__(self, var):
self.var = var
Foo.__init__ = __init__
# Yet another cell
def print_var(self):
print(self.var)
Foo.print_var = print_var
I don't expect it to be extremely robust, but... it should work for regular classes.
EDIT: I believe that there are a couple of situations where this may break. I am not sure if that will resist code inspection, given that the method lives "far" from the class. But you are using a notebook, so code inspection should not be an issue (?), although keep that in mind if debugging.
Another possible issue can be related to use of metaclasses. If you try to use metaclasses (or derive from some class which uses a metaclass) that may broke it, because metaclasses typically expect to be able to know all the methods of the class, and by dynamically adding methods to a class, we are bending the rules on the flow of class creation.
Without metaclasses or some "quite-strange" use cases, the approach should be safe-ish.
For "simple" classes, it is a perfectly valid approach. But... it is not exactly an expected feature, so (ab)using it may give some additional problems which I may not
Here's a decorator which lets you add members to a class:
import functools
def update_class(
main_class=None, exclude=("__module__", "__name__", "__dict__", "__weakref__")
):
"""Class decorator. Adds all methods and members from the wrapped class to main_class
Args:
- main_class: class to which to append members. Defaults to the class with the same name as the wrapped class
- exclude: black-list of members which should not be copied
"""
def decorates(main_class, exclude, appended_class):
if main_class is None:
main_class = globals()[appended_class.__name__]
for k, v in appended_class.__dict__.items():
if k not in exclude:
setattr(main_class, k, v)
return main_class
return functools.partial(decorates, main_class, exclude)
Use it like this:
#%% Cell 1
class MyClass:
def method1(self):
print("method1")
me = MyClass()
#%% Cell 2
#update_class()
class MyClass:
def method2(self):
print("method2")
me.method1()
me.method2()
This solution has the following benefits:
pure python
Doesn't change the inheritance order
Effects existing instances
There is no way to split a single class,
You could however, add methods dynamically to an instance of it
CELL #1
import types
class A:
def __init__(self, var):
self.var = var
a = A()
And in a different cell:
CELL #2
def print_var(self):
print (self.var)
a.print_var = types.MethodType( print_var, a )
Now, this should work:
CELL #3
a.print_var()
Medhat Omr's answer provides some good options; another one I found that I thought someone might find useful is to dynamically assign methods to a class using a decorator function. For example, we can create a higher-order function like the one below, which takes some arbitrary function, gets its name as a string, and assigns it as a class method.
def classMethod(func):
setattr(MyClass, func.__name__, func)
return func
We can then use the syntactic sugar for a decorator above each method that should be bound to the class;
#classMethod
def get_numpy(self):
return np.array(self.data)
This way, each method can be stored in a different Jupyter notebook cell and the class will be updated with the new function each time the cell is run.
I should also note that since this initializes the methods as functions in the global scope, it might be a good idea to prefix them with an underscore or letter to avoid name conflicts (then replace func.__name__ with func.__name__[1:] or however characters at the beginning of each name you want to omit. The method will still have the "mangled" name since it is the same object, so be wary of this if you need to programmatically access the method name somewhere else in your program.
thanks#Medhat Omr, it works for me for the #classmethod as well.
Base class in the first cell
class Employee:
# define two class variables
num_empl = 0
raise_amt = 1.05
def __init__(self, first, last, pay):
self.first = first
self.last = last
self.pay = pay
...
...
#classmethod in an another cell:
class Employee(Employee):
#classmethod
def set_raise_amt(cls, amount):
cls.raise_amt = amount
empl = Employee("Jahn", "Smith", 65000)
Employee.set_raise_amt(1.04)
print(empl.full_name() + " is getting " + str(empl.apply_raise()))
I was just wondering if it's considered wildly inappropriate, just messy, or unconventional at all to use the init method to set variables by calling, one after another, the rest of the functions within a class. I have done things like, self.age = ch_age(), where ch_age is a function within the same class, and set more variables the same way, like self.name=ch_name() etc. Also, what about prompting for user input within init specifically to get the arguments with which to call ch_age? The latter feels a little wrong I must say. Any advice, suggestions, admonishments welcome!
I always favor being lazy: if you NEED to initialize everything in the constructor, you should--in a lot of cases, I put a general "reset" method in my class. Then you can call that method in init, and can re-initialize the class instance easily.
But if you don't need those variables initially, I feel it's better to wait to initialize things until you actually need them.
For your specific case
class Blah1(object):
def __init__(self):
self.name=self.ch_name()
def ch_name(self):
return 'Ozzy'
you might as well use the property decorator. The following will have the same effect:
class Blah2(object):
def __init__(self):
pass
#property
def name():
return 'Ozzy'
In both of the implementations above, the following code should not issue any exceptions:
>>> b1 = Blah1()
>>> b2 = Blah2()
>>> assert b1.name == 'Ozzy'
>>> assert b2.name == 'Ozzy'
If you wanted to provide a reset method, it might look something like this:
class Blah3(object):
def __init__(self, name):
self.reset(name)
def reset(self, name):
self.name = name
I have a set of related classes that all inherit from one base class. I would like to use a factory method to instantiate objects for these classes. I want to do this because then I can store the objects in a dictionary keyed by the class name before returning the object to the caller. Then if there is a request for an object of a particular class, I can check to see whether one already exists in my dictionary. If not, I'll instantiate it and add it to the dictionary. If so, then I'll return the existing object from the dictionary. This will essentially turn all the classes in my module into singletons.
I want to do this because the base class that they all inherit from does some automatic wrapping of the functions in the subclasses, and I don't want to the functions to get wrapped more than once, which is what happens currently if two objects of the same class are created.
The only way I can think of doing this is to check the stacktrace in the __init__() method of the base class, which will always be called, and to throw an exception if the stacktrace does not show that the request to make the object is coming from the factory function.
Is this a good idea?
Edit: Here is the source code for my base class. I've been told that I need to figure out metaclasses to accomplish this more elegantly, but this is what I have for now. All Page objects use the same Selenium Webdriver instance, which is in the driver module imported at the top. This driver is very expensive to initialize -- it is initialized the first time a LoginPage is created. After it is initialized the initialize() method will return the existing driver instead of creating a new one. The idea is that the user must begin by creating a LoginPage. There will eventually be dozens of Page classes defined and they will be used by unit testing code to verify that the behavior of a website is correct.
from driver import get_driver, urlpath, initialize
from settings import urlpaths
class DriverPageMismatchException(Exception):
pass
class URLVerifyingPage(object):
# we add logic in __init__() to check the expected urlpath for the page
# against the urlpath that the driver is showing - we only want the page's
# methods to be invokable if the driver is actualy at the appropriate page.
# If the driver shows a different urlpath than the page is supposed to
# have, the method should throw a DriverPageMismatchException
def __init__(self):
self.driver = get_driver()
self._adjust_methods(self.__class__)
def _adjust_methods(self, cls):
for attr, val in cls.__dict__.iteritems():
if callable(val) and not attr.startswith("_"):
print "adjusting:"+str(attr)+" - "+str(val)
setattr(
cls,
attr,
self._add_wrapper_to_confirm_page_matches_driver(val)
)
for base in cls.__bases__:
if base.__name__ == 'URLVerifyingPage': break
self._adjust_methods(base)
def _add_wrapper_to_confirm_page_matches_driver(self, page_method):
def _wrapper(self, *args, **kwargs):
if urlpath() != urlpaths[self.__class__.__name__]:
raise DriverPageMismatchException(
"path is '"+urlpath()+
"' but '"+urlpaths[self.__class.__name__]+"' expected "+
"for "+self.__class.__name__
)
return page_method(self, *args, **kwargs)
return _wrapper
class LoginPage(URLVerifyingPage):
def __init__(self, username=username, password=password, baseurl="http://example.com/"):
self.username = username
self.password = password
self.driver = initialize(baseurl)
super(LoginPage, self).__init__()
def login(self):
driver.find_element_by_id("username").clear()
driver.find_element_by_id("username").send_keys(self.username)
driver.find_element_by_id("password").clear()
driver.find_element_by_id("password").send_keys(self.password)
driver.find_element_by_id("login_button").click()
return HomePage()
class HomePage(URLVerifyingPage):
def some_method(self):
...
return SomePage()
def many_more_methods(self):
...
return ManyMorePages()
It's no big deal if a page gets instantiated a handful of times -- the methods will just get wrapped a handful of times and a handful of unnecessary checks will take place, but everything will still work. But it would be bad if a page was instantiated dozens or hundreds or tens of thousands of times. I could just put a flag in the class definition for each page and check to see if the methods have already been wrapped, but I like the idea of keeping the class definitions pure and clean and shoving all the hocus-pocus into a deep corner of my system where no one can see it and it just works.
In Python, it's almost never worth trying to "force" anything. Whatever you come up with, someone can get around it by monkeypatching your class, copying and editing the source, fooling around with bytecode, etc.
So, just write your factory, and document that as the right way to get an instance of your class, and expect anyone who writes code using your classes to understand TOOWTDI, and not violate it unless she really knows what she's doing and is willing to figure out and deal with the consequences.
If you're just trying to prevent accidents, rather than intentional "misuse", that's a different story. In fact, it's just standard design-by-contract: check the invariant. Of course at this point, SillyBaseClass is already screwed up, and it's too late to repair it, and all you can do is assert, raise, log, or whatever else is appropriate. But that's what you want: it's a logic error in the application, and the only thing to do is get the programmer to fix it, so assert is probably exactly what you want.
So:
class SillyBaseClass:
singletons = {}
class Foo(SillyBaseClass):
def __init__(self):
assert self.__class__ not in SillyBaseClass.singletons
def get_foo():
if Foo not in SillyBaseClass.singletons:
SillyBaseClass.singletons[Foo] = Foo()
return SillyBaseClass.singletons[Foo]
If you really do want to stop things from getting this far, you can check the invariant earlier, in the __new__ method, but unless "SillyBaseClass got screwed up" is equivalent to "launch the nukes", why bother?
it sounds like you want to provide a __new__ implementation: Something like:
class MySingledtonBase(object):
instance_cache = {}
def __new__(cls, arg1, arg2):
if cls in MySingletonBase.instance_cache:
return MySingletonBase.instance_cache[cls]
self = super(MySingletonBase, cls).__new__(arg1, arg2)
MySingletonBase.instance_cache[cls] = self
return self
Rather than adding complex code to catch mistakes at runtime, I'd first try to use convention to guide users of your module to do the right thing on their own.
Give your classes "private" names (prefixed by an underscore), give them names that suggest they shouldn't be instantiated (eg _Internal...) and make your factory function "public".
That is, something like this:
class _InternalSubClassOne(_BaseClass):
...
class _InternalSubClassTwo(_BaseClass):
...
# An example factory function.
def new_object(arg):
return _InternalSubClassOne() if arg == 'one' else _InternalSubClassTwo()
I'd also add docstrings or comments to each class, like "Don't instantiate this class by hand, use the factory method new_object."
You can also just nest classes in factory method, as described here:
https://python-3-patterns-idioms-test.readthedocs.io/en/latest/Factory.html#preventing-direct-creation
Working example from mentioned source:
# Factory/shapefact1/NestedShapeFactory.py
import random
class Shape(object):
types = []
def factory(type):
class Circle(Shape):
def draw(self): print("Circle.draw")
def erase(self): print("Circle.erase")
class Square(Shape):
def draw(self): print("Square.draw")
def erase(self): print("Square.erase")
if type == "Circle": return Circle()
if type == "Square": return Square()
assert 0, "Bad shape creation: " + type
def shapeNameGen(n):
for i in range(n):
yield factory(random.choice(["Circle", "Square"]))
# Circle() # Not defined
for shape in shapeNameGen(7):
shape.draw()
shape.erase()
I'm not fan of this solution, just want to add this as one more option.
Say I have a class and a function:
class AddressValidator(self):
def __init__(self):
pass
def validate(address):
# ...
def validate_address(addr):
validator = AddressValidator()
return validator.validate(addr)
The function is a shortcut for using the class, if you will. Now, what if this function has to be run thousands of times? If the validator class actually has to do something on instantiation, like connecting to a database, creating it over and over thousands of times is pretty wasteful. I was wondering if I could perhaps do something like this:
def validate_address(addr):
if not hasattr(validate_address, 'validator'):
validate_address.validator = AddressValidator()
validator = validate_address.validator
return validator.validate(addr)
Now the validator class is only instantiated once and saved "in the function", to put it that way. I've never seen this done though, so I'm guessing it's bad practice. If so, why?
Note: I know I can just cache the validator object in a module global. I'm just curious if this is a viable solution when I want to avoid littering my module.
Despite "everithing is an object", not everithing work as nice as instances of well controlled class.
This problem looks like typical case for "functor" or "callable object" as it called in python.
the code will be look something like
class AddressValidator(self):
def __init__(self):
pass
def __call__(self,address):
# ...
validate_address = AdressValidator()
or you could just define your function as shortcut to bound method
class AddressValidator(self):
def __init__(self):
pass
def validate(self,address):
# ...
validate_adress = AdressValidator().validate
I'd go with a default argument (evaluated once at function definition time and bound to the function):
def validate_address(addr, validator=AddressValidator())
return validator.validate(addr)
This is perfectly acceptable if instances of AddressValidator are considered immutable (i.e. they don't contain methods that modify their internal state), and it also allows you to later override the choice of validator should you find the need to (e.g. to provide a validator specialized for a particular country).