Develop Reference

Can I use dot operator in Python to put a class on another one?

my question may sound a bit weird. In JS we can put an object on another one, how does it work in Python? what is the name of this functionality?
Example:
class A:
def __init__(self):
print('I am A')
class B:
def __init__(self):
print('I am B')
def i_am_a_function(x):
print('I am function')
print(x)
# What if I create a method using dot:
A.holder_for_b = B
# OR:
A.some_method = i_am_a_function
########
print(A.holder_for_b)
print(A.some_method('foo'))
result:
<class '__main__.B'>
I am function
foo
None
and as a final question, what is that "None" ?

I think you mean:
class A:
def __init__(self):
print('I am A')
class B:
def __init__(self):
print('I am B')
def i_am_a_function(x):
print('I am function')
print(x)
a = A()
b = B()
# What if I create a method using dot:
a.holder_for_b = B
# OR:
a.some_method = i_am_a_function
########
print(a.holder_for_b)
print(a.some_method('foo'))
## outputs:
I am A
I am B
<class '__main__.B'>
I am function
foo
None
So it seems that you can assign a function to an object. This is called in Python "monkey patching".
But the disadvantage of this is - I think:
It applies only to the object a - so you can't inherit it to another object
(different than in javascript I guess - prototype).
You can't interact with any other property or method in the class,
because you don't have self (in js this) available in that monkey-patched method.
(If I am wrong, please tell me.)
And it is bad style, because you should have in the class definition everything what you need. It would be hard to understand for anyone who wants to maintain the code.
Also the a.holder_for_b = B is unnecessary.
For what you would need a.holder_for_b? You can't anyway not inherit such a monkey-patched method to another object in Python.
Python's class definitions are lexical.
None
The None is the return value of the function call.
x = a.some_method('foo')
## I am function
## foo
x
## Nothing returned
x is None
## returns: True
print(x)
## None
As you can see: x returns nothing.
The None would not be visible if you would not have used print() around the call a.some_method('foo').
So the function is called - which prints I am a function and foo.
But the return value (None) gets returned from this call - and the print() prints it.
Putting an object to another
Do you mean "attaching"? This is very well possible in Python - by monkey patching.
a = A()
a.b = B()
Now, b is an attribut/property of the object a, which holds the object generated by B(). You could also just assign any already generated object to a in a similar way.
But this applies only to a. And in Python, you cannot generate another object based on a. So it is a little bit pointless in my view.
Perhaps more pythonic ways would be:
# everytime you generate an instance of the A class, generate an instance of B
# and attache it to newly generated instance of the A class as `b`:
class A:
def __init__(self):
print('I am A')
self.b = B()
# generate an instance of A, but attach as `b` property an existing object:
class A:
def __init__(self, obj):
print('I am A')
self.b = obj
b = B()
# ...
a = A(b)
# a.b is then identical to the generated `b` - it is a reference to it.
# So everything to do to it will be done to `b` too!
# This can be very confusing and create bugs.
a.b.my_new_property = 1
a.b.my_new_property ## 1
b.my_new_property ## 1
But I think your problem is that you are trying to apply JavaScript's prototypical OOP system habits to Python. In Python, you should better define everything in your class definitions, give other objects or classes as arguments to the constructor methods (__init__ or __new__).
And if you want to modify class definitions in a re-usable manner, probably use decorators or decorator classes. Or design patterns.

A class defines a namespace. A def creates a function object and assigns it to a variable in the active namespace. If that def is in the class namespace (one indentation in from the class definition), the variable is assigned to the class namespace. In your example, both __init__ functions are assigned to variables called "__init__" in the classes. i_am_a_function is not in a class namespace so it is assigned to the module ("global") namespace.
Normally, accessing a variable in a class namespace is just a name lookup like any other variable.
print(A.holder_for_b)
simply looked up the object in A.holder_for_b, which is a class object. You could add
print(A.holder_for_b())
and get an instance of that class. Similarly A.some_method is just looking up the variable on A. When you call it, you are just calling a function. You saw the print in the function itself and its None return value.
But python does something different if you reference a variable off of a class instance object (as opposed to the class object itself). If you try calling a variable (and it is a function object), python will convert that function object into a method and will automatically add a reference to the the instance as a so-called "self" object.
a = A()
print(a)
print(a.some_method())
prints
I am A
<__main__.A object at 0x7fb6c0ab61c0>
I am function
<__main__.A object at 0x7fb6c0ab61c0>
Here, since you call a variable of an instance object, its first parameter that you called "x" is now the instance object (the "self" parameter). Python didn't really care what you called that variable, its just the first in the parameter list.

How to store functions as class variables in python?

I am writing a framework, and I want my base class to use different functions for renaming in the child classes. I figured the best way would be to use a class attribute, like in case of A, but I got TypeErrors when running it like in rename_columns(). However it worked with implementation like B
import pandas as pd
class A:
my_func_mask = str.lower
foo = 'bar'
def rename_columns(self, data):
return data.rename(columns=self.my_func_mask)
class B(A):
def rename_columns(self, data):
return data.rename(columns=self.__class__.my_func_mask)
So I experimented with the above a bit, and I get the following:
a = A()
a.foo # Works fine, gives back 'bar'
a.__class__.my_func_mask # Works as expected `a.__class__.my_func_mask is str.lower` is true
a.my_func_mask # throws TypeError: descriptor 'lower' for 'str' objects doesn't apply to 'A' object
My questions would be why can I use regular typed (int, str, etc.) values as class attributes and access them on the instance as well, while I cannot do that for functions?
What happens during the attribute lookup in these cases? What is the difference in the attribute resolution process?
Actually both foo and my_func_mask is in __class__.__dict__ so I am a bit puzzled. Thanks for the clarifications!

You are storing an unbound built-in method on your class, meaning it is a descriptor object. When you then try to access that on self, descriptor binding applies but the __get__ method called to complete the binding tells you that it can't be bound to your custom class instances, because the method would only work on str instances. That's a strict limitation of most methods of built-in types.
You need to store it in a different manner; putting it inside another container, such as a list or dictionary, would avoid binding. Or you could wrap it in a staticmethod descriptor to have it be bound and return the original. Another option is to not store this as a class attribute, and simply create an instance attribute in __init__.
But in this case, I'd not store str.lower as an attribute value, at all. I'd store None and fall back to str.lower when you still encounter None:
return data.rename(columns=self.my_func_mask or str.lower)
Setting my_func_mask to None is a better indicator that a default is going to be used, clearly distinguishable from explicitly setting str.lower as the mask.

You need to declare staticmethod.
class A:
my_func_mask = staticmethod(str.lower)
foo = 'bar'
>>> A().my_func_mask is str.lower
>>> True

Everything that is placed in the class definition is bound to the class, but you can't bind a built-in to your own class.
Essentially, all code that you place in a class is executed when the class is created. All items in locals() are then bound to your class at the end of the class. That's why this also works to bind a method to your class:
def abc(self):
print('{} from outside the class'.format(self))
class A:
f1 = abc
f2 = lambda self: print('{} from lambda'.format(self))
def f3(self):
print('{} from method'.format(self))
To not have the function bound to your class, you have to place it in the __init__ method of your class:
class A:
def __init__(self):
self.my_func_mask = str.lower

Python3 mock: assert_has_calls for production-code methods?

I've got this production class:
class MyClass:
def __init__(self):
self.value = None
def set_value(self, value):
self.value = value
def foo(self):
# work with self.value here
# raise RuntimeError("error!")
return "a"
Which is being used from another place, like this:
class Caller:
def bar(self, smth):
obj = MyClass()
obj.set_value(smth)
# ...
# try:
obj.foo()
# except MyError:
# pass
obj.set_value("str2")
# obj.foo()
and I got this:
class MyError(Exception):
pass
In my test I want to make sure that Caller.bar calls obj.set_value, first with smth="a", then with smth="b", but I want it to really set the value (i.e. call the real set_value method). Is there any way for me to tell the mock to use the actual method, so I can later on read what it was called with?
P.S. I know that I can just change "foo" to require the parameter "smth" so I could get rid of "set_value", but I want to know if there is another option than this.
Okay, so I have tried this in my test:
def test_caller(self):
with patch('fullpath.to.MyClass', autospec=MyClass) as mock:
mock.foo.side_effect = [MyError("msg"), "text"]
caller = Caller()
caller.bar("str1")
calls = [call("str1"), call("str2")]
mock.set_value.assert_has_calls(calls)
But I see that the mock was not successful since the real "foo" is called when I wanted it to first raise MyError, then return "text".
Also, the assertion fails:
AssertionError: Calls not found.
Expected: [call('str1'), call('str2')]
Actual: []

The problem here is that you have mocked out your Class, and are not properly using the instance of your class. This is why things are not behaving as expected.
So, lets take a look at what is going on.
Right here:
with patch('fullpath.to.MyClass', autospec=MyClass) as mock:
So, what you are doing right here is mocking out your class MyClass only. So, when you are doing this:
mock.set_value.assert_has_calls(calls)
And inspect what is going on when you execute your unittest, your mock calls will actually contain this:
[call().set_value('str1'), call().foo(), call().set_value('str2')]
Pay attention to call as it is written as call(). call is with reference to your mock here. So, with that in mind, you need to use the called (aka return_value within context of the mocking world) mock to properly reference your mock object that you are trying to test with. The quick way to fix this is simply use mock(). So you would just need to change to this:
mock().set_value.assert_has_calls(calls)
However, to be more explicit on what you are doing, you can state that you are actually using the result of calling mock. Furthermore, it would actually be good to note to use a more explicit name, other than mock. Try MyClassMock, which in turn you name your instance my_class_mock_obj:
my_class_mock_obj = MyClassMock.return_value
So in your unit test it is more explicit that you are using a mocked object of your class. Also, it is always best to set up all your mocking before you make your method call, and for your foo.side_effect ensure that you are also using the instance mock object. Based on your recent update with your exception handling, keep your try/except without comments. Putting this all together, you have:
def test_caller(self):
with patch('tests.test_dummy.MyClass', autospec=MyClass) as MyClassMock:
my_class_mock_obj = MyClassMock.return_value
my_class_mock_obj.foo.side_effect = [MyError("msg"), "text"]
caller = Caller()
caller.bar("str1")
calls = [call("str1"), call("str2")]
my_class_mock_obj.set_value.assert_has_calls(calls)

Test if some field has been initialized in python

I am trying to write a test in Python that checks if a method in a class that I am writing sets the attribute value for a dataset in some Hdf file. The logic is the following: An instance of the class is constructed by passing an instance of h5py.File, then one method creates a dataset inside this file. In the next step I have another method that sets certain attributes for this dataset.
What I am trying to test is if my class method create_attributes(self,attributes) sets the field hdf_file[dset_name].attrs[attr_name] to some value that is passed in the variable attributes. However, I would like to avoid to actually create a Hdf file. So far I have tried to mock an instance of a hdf file and work with that. The minimal working code example would be the following:
import h5py
class TestSomething:
#mock.patch('h5py.File')
def test_if_attr_is_initialized(self,mock_hdf):
# Here I would like to call a function that basically executes
# the following line:
mock_hdf['test_dset'].attrs['test_field']='value'
# Then I want to check if the attribute field has been assigned
assert mock_hdf['test_dset'].attrs['test_field']=='value'
Can anybody help me finding the correct thing to do to check whether or not the attribute in the hdf file is set correctly? Any help would be greatly appreciated, I am a complete newbie to all the mocking techniques.
Edit:
In the following I am providing a minimal code example for both the class, and the respective test as requeseted by wwii:
import h5py
class HdfWriter():
def __init__(self,hdf_file):
self.hdf_file=hdf_file
def create_attrs(self,attributes):
dset_name=attributes.keys()[0]
attrs=attributes[dset_name]
for key in attrs:
self.hdf_file[dset_name].attrs[key]=attrs[key]
Please note here that with a real hdf file I would first have to create a dataset but I would like to leave that for another test. The following test should just check, whether for a hypothetical hdf file, which has the dataset test_dset the attributes for this data set are written:
import h5py
import HdfWriter
class TestSomething:
#mock.patch('h5py.File')
def test_if_attr_is_initialized(self,mock_hdf):
writer=hw.HdfWriter(mock_hdf)
attr={'test_dset':{'test_field':'test_value'}}
writer.create_attrs(attr)
assert writer.hdf_file['test_dset'].attrs['test_field']=='value'

Mocking h5py.File
class HdfWriter():
def __init__(self,hdf_file):
self.hdf_file=hdf_file
def create_attrs(self,attributes):
dset_name=attributes.keys()[0]
attrs=attributes[dset_name]
for key in attrs:
self.hdf_file[dset_name].attrs[key]=attrs[key]
For the purpose of the create_attrs method, hdf_file behaves as a dictionary that returns an object that also behaves like a dictionary. The docs explain pretty clearly how to mock a dictionary.
You need a mock that has an attrs attribute that behaves like a dictionary:
import mock
attrs_d = {}
def setattrs(name, value):
## print 'setattrs', name, value
attrs_d[name] = value
def getattrs(name):
## print 'getattrs', name
return attrs_d[name]
mock = mock.MagicMock()
mock.attrs.__setitem__.side_effect = setattrs
mock.attrs.__getitem__.side_effect = getattrs
You need a mock for hdf_file that behaves like a dictionary and will return the mock object created above.
hdf_d = {'test_dset':mock}
def getitem(name):
## print 'getitem', name
return hdf_d[name]
def setitem(name, value):
hdf_d[name] = value
mock_hdf = mock.MagicMock()
mock_hdf.__getitem__.side_effect = getitem
mock_hdf.__setitem__.side_effect = setitem
hdf_d, as implemented, only works for the key 'test_dset'. Depending on your needs it may be better for getitems to just return mock regardless of the name argument.
def test_if_attr_is_initialized(mock_hdf):
writer=HdfWriter(mock_hdf)
attr={'test_dset':{'test_field':'test_value'}}
writer.create_attrs(attr)
print writer.hdf_file['test_dset'].attrs['test_field'], '==', attr['test_dset']['test_field']
assert writer.hdf_file['test_dset'].attrs['test_field']=='test_value'
test_if_attr_is_initialized(mock_hdf)
>>>
test_value == test_value
>>>
This should suffice to test create_attrs but it may not be optimal - maybe someone will chime in with some refinements.

Self in Class Demanding Argument

For some reason most instances of classes are returning Type errors saying that insufficient arguments were passed, the problem is with self.
This works fine:
class ExampleClass:
def __init__(self, some_message):
self.message = some_message
print ("New ExampleClass instance created, with message:")
print (self.message)
ex = ExampleClass("message")
However almost every other Class I define and call an instance of returns the same error. The almost identical function:
class Test(object):
def __init__(self):
self.defaultmsg = "message"
def say(self):
print(self.defaultmsg)
test = Test
test.say()
Returns a Type Error, saying that it needs an argument. I'm getting this problem not just with that class, but with pretty much every class I define, and I have no idea what the problem is. I just updated python, but was getting the error before. I'm fairly new to programming.

You have to instantiate the class:
test = Test() #test is an instance of class Test
instead of
test = Test #test is the class Test
test.say() #TypeError: unbound method say() must be called with Test instance as first argum ent (got nothing instead)
if you are curious you can try this:
test = Test
test.say(Test()) #same as Test.say(Test())
It works because I gave the class instance (self) to the unbound method !
Absolutely not recommended to code this way.

You should add parentheses to instantiate a class:
test = Test()

Your test refers to the class itself, rather than an instance of that class. To create an actual test instance, or to 'instantiate' it, add the parentheses. For example:
>>> class Foo(object):
... pass
...
>>> Foo
<class '__main__.Foo'>
>>> Foo()
<__main__.Foo object at 0xa2eb50>
The error message was trying to tell you that there was no such self object to pass in (implicitly) as the function argument, because in your case test.say would be an unbound method.