I have the following script:
from abc import ABC, abstractmethod
import dill
class A(ABC):
#abstractmethod
def test(self, a):
"""Abstract method"""
class B(A):
def test(self, a):
return a + 1
if __name__ == '__main__':
test_obj = B()
with open('test_save.pkl', 'wb') as f:
dill.dump(test_obj, f)
When I run this script in a python 3.6 environment, it runs successfully. When I run it in a python 3.7.1 environment, it errors with:
TypeError: can't pickle _abc_data objects
One work around I have found is to move the class definition of B into a separate file and import it - then saving works. HOWEVER, assuming I do NOT want to do that:
1) is there a way to serialize with dill a subclass of an abstractclass defined in the same file
2) what changed between python 3.6 and python 3.7 that caused this?
Related
I have some problem with mocking and testing my function/class.
I think the easiest way to explain it will be showing some code:
module.py:
import some_module
class MyClassToTest:
def __init__():
pass
def f(self, a):
if isinstance(a, some_module.someClass):
print("true")
I want to test my module.py:
test_module.py:
import sys
from unittest import mock
sys.modules['some_module'] = mock.MagicMock() # mocking whole module
def test_my_test():
to_test = MyClassToTest()
to_test.f(5)
Of course here I will have an error:
TypeError: isinstance() arg 2 must be a type or tuple of types
because some_module was mocked.
I really need to mock the "some_module" - it's needed for some other tests.
And yes, I probably should unmock the module just for some tests or maybe mock the isinstance when I'm using it in specific function?
Do you have any idea how to process it?
You are in the right way, you've just forgot to set the object spec. Here is a fully functional example based in your question.
some_module.py
class SomeModule(object):
def method(self):
return True
module.py
import some_module
class MyClass:
def __init__(self):
pass
def f(self, a):
if isinstance(a, some_module.SomeModule):
return True
return False
test_module.py
from unittest import mock
from unittest import main, TestCase
import function
class MyClassTestCase(TestCase):
def test_myclass(self):
m = mock.MagicMock(spec=function.some_module.SomeModule)
mc = function.MyClass()
self.assertEqual(mc.f(1), False)
I would like to persist an object that is created from a dynamically defined class that extends an abstract base class. I have tried to achieve this by pickling the object. I am using Python 3.6.8
The error message suggests that pickle is trying to look for the class I have created within the abc module and is unable to find it there. I have tried to pickle by creating the class within the main module (by defining class_factory inside test.py) but I get the same error message.
objmodel.py
from abc import ABC, abstractmethod
class mtype(ABC):
def __init__(self):
super().__init__()
#abstractmethod
def set_someattrib(self, attrval):
pass
#classmethod
def set_attrib1(cls, aval):
cls.attrib = aval
utils.py
from objmodel import *
def class_factory(mname, aval):
c1 = type(mname, (mtype,), {"set_someattrib": set_attrib1})
c1.set_someattrib(aval)
return c1
test.py
from utils import class_factory
import pickle
c1 = class_factory('C', 1)
print(c1.attrib)
m = c1()
fh = open('objtest', 'wb')
pickle.dump(m, fh)
I get the following error:
_pickle.PicklingError: Can't pickle : attribute lookup C on abc failed
python script test.py:
class Test:
def __init__(self):
pass
def addition(self, a, b):
return a + b
Below is the ansible playbook to execute Python script.
---
- name: execute install script
script: test.py
Instead of calling complete python script, Is it possible to call only addition method in test.py during ansible playbook execution
I do not think that this is possible. If test.py contains many classes, you can do the following, which will only define one class and never instanciate it.
from test import Test
print(Test.addition(None, 3, 4)) # prints 7
Why putting your function as a class method if you want to import it alone though ?
You could also do the following:
test.py
class Test:
def __init__(self):
pass
# Define function outside class so that it can be imported
def addition(self, a, b):
return a + b
# Add function to class
Test.addition = addition
And then do from test import addition
I would like to find all instances in the code where np.random.seed is called (without using grep). In order to set a breakpoint in ipdb, I tried to find the source file with
import inspect; inspect.getsourcefile(np.random.seed)
but it throws a TypeError because it is a built-in method (because it is coded in C).
Is it possible to watch any calls to np.random.seed by modifying something in the main source file?
Additionally, it would be suitable to patch this method, e.g. additionally logging it (or calling a debugger):
def new_random_seed(seed):
"""
This method should be called instead whenever np.random.seed
is called in any module that is invoked during the execution of
the main script
"""
print("Called with seed {}".format(seed))
#or: import ipdb; ipdb.set_trace()
return np.random.seed()
Maybe using a mock framework is the way to go?
The second question concerns the scenario in which a class B inherits from a class A in a library and I want to use the functionality of class B, but overwrite a function it uses from class A without modifying classes A and B. Probably, I should use mocking, but I am not sure about the overhead, so I wrote the following:
#in library
class A():
def __init__(self, name):
self.name = name
def work(self):
print("{} working".format(self.name))
class B():
def __init__(self):
self.A = A("Machine")
def run_task(self):
self.A.work()
# in main script
# Cannot change classes A and B, so make a subclass C
import types
class C(B):
def __init__(self, modified_work):
super().__init__()
self.A.work = types.MethodType(modified_work, self.A) #MethodType for self
b = B()
b.run_task()
modified_work = lambda self: print("{} working faster".format(self.name))
c = C(modified_work)
c.run_task()
The output is:
Machine working
Machine working faster
Is this good style?
This might be a simpler solution to your second question:
# lib.py
class A():
def work(self):
print('working')
class B():
def __init__(self):
self.a = A()
def run(self):
self.a.work()
Then in your code:
import lib
class A(lib.A):
def work(self):
print('hardly working')
lib.A = A
b = lib.B()
b.run()
Or:
import lib
class AA(lib.A):
def work(self):
print('hardly working')
class BB(lib.B):
def __init__(self):
self.a = AA()
b = lib.B()
b.run()
b = BB()
b.run()
I am new in Python and I am trying to create two classes with the same name in two different source files. Let’s call them "Main.py" and "Extension.py". The class is "MyClass". MyClass in Extesntion.py is derived from MyClass in file Main.py. If it works then when I create an object myclass and I import Extension in my code, then I would have more functions in comparison with file Main.py.
File Main.py
class MyClass:
def __init__(self):
Initialize something
def foo1(self, a, b):
Do something
Then extension would be like this:
File Extensions.py
import Main
class MyClass(MyClass):
def __init__(self):
Initialize something
def foo2(self, a, b):
Do something
def foo3(self, a, b):
Do something
And then if I have code like this. I expect that I can't use foo2 and foo3.
import Main
myclass = MyClass()
myclass.foo1(a, b)
And finally if I have code like this. I expect that I use all the functions.
import Extension
myclass = MyClass()
myclass.foo1(a, b)
myclass.foo2(a, b)
myclass.foo3(a, b)
If you do
import main
you'll need to use main.MyClass to create an object from main.py.
Instead you can do
from main import MyClass
to have it available directly.
If you need two different classes with the same name, you can instead do
from main import MyClass as MainClass
and you'll have the class available under the name MainClass
Unless you do from Extension import *, you'll need to specify the module in order to access the class.
import Main
import Extension
foo = Main.MyClass()
bar = Extension.MyClass()
If you don't want to have to specify the module, then the only way to avoid a name collision is to import the class under a different name like so:
from Main import MyClass as ClassA
It's quite easy when you explicitly import the given name using the from {module} import {name} syntax.
File main.py
class MyClass:
def __init__(self):
pass
def foo1(self, a, b):
pass
File extensions.py
from main import MyClass
class MyClass(MyClass):
def __init__(self):
pass
def foo2(self, a, b):
pass
def foo3(self, a, b):
pass
File client_main.py
from main import MyClass
myinstance = MyClass()
myinstance.foo1(a, b)
File client_extensions.py
from extensions import MyClass
myinstance = MyClass()
myinstance.foo1(a, b)
myinstance.foo2(a, b)
myinstance.foo3(a, b)
Generally in this case, you would do an import as. This allows you to alias your import as a new name. So in the file where your second class is, import the first class as:
from main import MyClass as MainMyClass
Then when doing your inheritance, refer to MainMyClass:
class MyClass(MainMyClass):