Field inheritance access in python - python

I have an issue with inheritance.
This is my main program:
def main(argv):
rfp = reqboxfileparserng() # Inherits from reqboxfileparser()
rfp.importsdir = './data/'
if rfp.parsingasutf8_win():
rfp.parsefile("./data/LRCv12.txt")
Here are the classes:
class reqboxfileparser():
def __init__(self):
... removed code ...
# Init mmap
self.file = None
self.f = None
def parsefile(self, filename):
# Public
self.filename = filename
# Init mmap
self.file = codecs.open(filename, encoding='utf-8', mode='r') # open(filename, 'r')
self.f = mmap.mmap(self.file.fileno(), 0, access=mmap.ACCESS_READ)
self.f.seek(0) # rewind
# Parsing stuff
self.getfunlist()
self.vlog(VERB_MED, "len(fun) = %d" % (len(self.funlist)))
self.getfundict()
self.vlog(VERB_MED, "fundict = %s" % (self.fundict))
... rest of reqboxfileparser() class code removed ...
class reqboxfileparserng(reqboxfileparser, object):
def __init__(self):
# Public
reqboxfileparser.__init__(self)
self.fundict = {}
self.importsdir = ''
def getfunlist(self):
"""
Overrided method to load from a CSV file
"""
self.funlist = []
fh = open(self.importsdir + 'in-uc-objects.csv', 'rb')
f = csv.reader(fh, delimiter=',')
... rest of the code removed, it works fine ...
def getfundict(self):
"""
Fills the fundict property with a dict where each element is indexed
by the fun name and each value is an object from the model
"""
self.__fundict = {}
beginloc = self.bodystartloc()
# PROBLEM!
finalloc = super(reqboxfileparser, self).f.size() - 1
... rest of the code removed ...
As you can see I have two classes, the first is reqboxfileparser() and the second one is reqboxfileparserng() which inherits from the first one.
On the main program I call the method: parsefile("./data/LRCv12.txt") [not overrided] which later calls getfundict() [overrided] on the second class, but when I try to access f.size() it always fails with TypeError: must be type, not classobj.
It's been a while since I don't develop classes with inheritance but if I'm not wrong the concepts are right. I'm newbie to Python.
Any help please?
Thanks a lot.

There are two issues at hand here:
Super and old-style classes:
class reqboxfileparser(): does not inherit from object, as a consequence, super(reqboxfileparser, self) will always yield the error:
TypeError: must be type, not classobj.
Improper super call in inheriting classes:
You're doing super(reqboxfileparser, self), but you're passing the inherited class (reqboxfileparser) as first argument, not the inheriting class.
As a consequence, Python would try to find a class that reqboxfileparser inherits from which implements what you're looking for you're looking for: f.
But that's not want you want: what you want an ancestor of reqboxfileparserng that implements f; that would be reqboxfileparser.
Please have a look at the documentation for the most common super call syntax.
Your solution
You probably guessed by now that you should be using super(reqboxfileparserng, self) instead.
Plus, you should always be using new-style classes (But that alone wouldn't solve your issue, you would get a different error complaining thatAttributeError: 'super' object has no attribute 'f', which would be True, as object does not provide f).
One last thing...
But here, you have one last issue!
You're trying to refer to f which is an attribute of the instance of the child class. This attribute is not present when you use the super call as it's not present in the class definition of the parent, which is the one the super call will use. (It's in the __init__ method)
I won't go into much more detail as to why this matters for super, but the idea is to basically use super only for stuff defined at class-level. Usually, methods are, so they're great candidates for super calls.
Here's an example describing what I mean:
class reqboxfileparser():
g = 'THIS IS G'
def __init__(self):
self.f = 'THIS IS F'
self.g = 'THIS IS NEW G'
def get_g(self):
return self.g
class reqboxfileparserng(reqboxfileparser, object):
def __init__(self):
reqboxfileparser.__init__(self)
def getfundict(self):
print super(reqboxfileparserng, self).g # Prints "THIS IS G"
print super(reqboxfileparserng, self).get_g() # Prints "THIS IS NEW G"
print super(reqboxfileparserng, self).f # This raises an AttributeError
if __name__ == '__main__':
o = reqboxfileparserng()
o.getfundict()
Overall, a super call is pretty similar to using ParentClass.stuff, only it deals with multiple inheritance better, and should be used for this reason.
You can see that here, reqboxfileparser.f would raise an AttributeError.
Footnote: a classobj is an old-style class, a type is a new-style class.

The point you're missing is that f is an attribute of the current class. It's inherited: when you call super(...).__init__(), the code sets f on self. So to access it from the subclass, you just do self.f, just like any other attribute.
The same is true of any attribute or method. You only need to use super when your subclass has actually overridden something, and you need to call the superclass's version. So there's no need to call super to get access to parsefile, for example: self.parsefile() would do fine.

Related

Initializing an attribute in a child class that is used in the parent class

I am using a 3rd party Python library (wxPython), which has a buggy class in one of its modules.
The problematic code section looks like this:
def OnText(self, event):
value = self.GetValue()
if value != self.__oldvalue:
pass # Here some more code follows ...
self.__oldvalue = value
The problem is the if statement, because at the first call to this method self.__oldvalue has not been initialized yet. So for a workaround until this bug has been fixed by the library devs I thought I could fix this with a little workaround. I simply wanted to derive a child class from that faulty class and initialize self.__oldvalue in this constructor:
class MyIntCtrl(wx.lib.intctrl.IntCtrl):
def __init__(self, *args, **kw):
self.__oldvalue = None
super().__init__(*args, **kw)
However, now when I use this new class MyIntCtrl instead of the original IntCtrl class, I do get exactly the same error as before:
Traceback (most recent call last):
File "/usr/local/lib/python3.6/dist-packages/wx/lib/intctrl.py", line 509, in OnText
if value != self.__oldvalue:
AttributeError: 'MyIntCtrl' object has no attribute '_IntCtrl__oldvalue'
Now I am wondering: What am I doing wrong, how else can I fix this issue in a child class?
Any member of class which starts with __ (double underscore) is private, you can use single underscore _ or not use underscores in naming for access them in derived classes.
class Parent:
def __init__(self):
self.__private_field = "private field"
self._protected_field = "protected field"
self.public_field = "public field"
class Child(Parent):
def __init__(self):
pass
def do(self):
print(self.__private_field) # It will throw exception
print(self._protected_field) # It will not throw exception
print(self.public_field) # It will not throw exception
Or you can bypass private/protected members by calling them like:
print(_Parent__private_field)

Python, main method not running (multiple classes in same file)

Context:
I developed a python script to be run on a remote linux server. Running using Python 3.6.1. The script worked but was very messy, and procedurally written as opposed to OO. So, I re-wrote this script into 2 different classes. One main class and a blueprint class for objects.
My script is a lot more complicated, i just simplified it for this question.
Desired Function:
Read values from CSV file. Create Objects from these values, 1 object per line. Do some calculations on the values on init'ing the object (in the objects class). Have these objects be accessible from the main class (Base class).
Problems:
I need some clarification on:
The main method is not running. Tried variants on the method call, like Base.main(), including the "if name" statement inside the Base class, and it complains about self not being defined
The "self" reference. Are my usages of this correct? For example: Adding the attribute "age" into the Person objects so you can access it with person.age for example. My method call "self.input_file_handling(Base.inputFilePath)" etc.
Script:
import csv
class Person:
def calculate_age(self):
self.age = 2017 - self.birthYear
def __init__(self, name, birthYear):
self.name = self.strip_characters(self, name)
self.birthYear = int(birthYear)
self.calculate_age()
class Base:
inputFilePath = "input.csv"
people = []
def main():
self.input_file_handling(Base.inputFilePath)
#More methods here
#staticmethod
def input_file_handling(input_file_path):
input_file_path = str(input_file_path)
with open(input_file_path, 'r') as csv_file:
csv_reader = csv.DictReader(csv_file)
for line in csv_reader:
name = line['Name']
age = line['age']
person = Person(name, age)
people.append(person)
if __name__ == '__main__':
main()
First the main method of Base class is not static because it use the self variable, so is necessary receive that.
If you want call the main method and use the self variable you need make something like that:
class Base:
def main(self):
pass
if __name__ == '__main__':
instance_of_base = Base()
instance_of_base.main()
You can call the input_file_handling method without using self, because it's static
Base.input_file_handling(Base.inputFilePath)
I think you need learn more about how python resolve static things and the class and object variables.
Python is not C. There is no main function that automagically executes.
The main method that you defined is inside Base class, but it doesn't accept an argument for the instance.
Either modify it so it accept it (ie self by the convention) or make it a static method.
Then in if __name__ == '__main__': either use Base().main() or Base.main(), depending on what approach you decided to take.
But you don't seem to need any of this, and only doing it for the sake of forcing Python to look/work as other languages (looking at you C++/Java). Python doesn't require you to have a class or a 'main' function/method to execute code.
Your code written in a Pythonic way would be: (Python3)
import csv
from time import time, gmtime
INPUT_FILE_PATH = "input.csv"
class Person:
def __init__(self, name, birth_year):
self.name = name.strip()
self.birth_year = birth_year
#property
def birth_year(self):
return self._birth_year
#setter.birth_year
def birth_year(self, value):
self._birth_year = value
self._age = gmtime(time()).tm_year - value
#property
def age(self):
return self._age
#setter.age
def age(self, value):
self._birth_year = gmtime(time()).tm_year - value
self._age = value
def input_file_handling(input_file_path):
people = []
with open(input_file_path, 'r') as csv_file:
csv_reader = csv.DictReader(csv_file)
for line in csv_reader:
people.append(Person(line['Name'], int(line['age'])))
return people
if __name__ == '__main__':
people = input_file_handling(INPUT_FILE_PATH)
You seem to come from a OOP-only language (C# maybe?).
Some tips:
Avoid globals when able for variables, use them for function definition, class definition and constants.
Do not use a new class to store functions that do not require it
Use lower case and '' for variable and function names, Use CamelCase for class names, use caps and '' for constants.
Use duck typing: do not check that a argument is of a given type, try to use it as if it was and handle throw exceptions if it isn't.
Properties ar your friends if you want to force a specific bahaviour at getting or setting a class attributes
If you do not understand somehting ask in the comments.

python name mangling in __init__ seems inconsistent

I'm trying to use a subclass that enhances instead of overriding the base class. I'm using the super method to call the base class. I find that I need to use the name mangling feature in __init__ (but only in init?) to make the code work. So for the heck of it I made the this print example. Since I didn't use name mangling I expected it to call subclass twice when I did the init, instead it calls the base class
It seems that __init__ sometimes sees the base class and sometimes sees the subclass. I'm sure it's just an incomplete understanding on my part, but was do I need name mangling for the real code, when in the print example it calls the base and subclass just fine?
the code
class base:
def __init__(self):
self.print()
def print(self):
print("base")
class subclass(base):
def __init__(self):
super(subclass, self).__init__()
self.print()
def print(self):
super(subclass, self).print()
print("subclass")
x = base()
x.print()
print("--")
y = subclass()
y.print()
the output - why doesn't y = subclass() print subclass instead of base since I didn't use name mangling?
> ./y.py
base
base
--
base
subclass
base
subclass
base
subclass
broken code when I don't use name mangling, works when I use self.__set and __set = set (the commented code). It gets the following error when I don't use __set:
File "./x.py", line 5, in __init__
self.set(arg)
TypeError: set() missing 1 required positional argument: 'arg2'
the code:
class base:
def __init__(self, arg):
self.set(arg)
# self.__set(arg)
# __set = set
def set(self, arg):
self.arg = arg
def print(self):
print("base",self.arg)
class subclass(base):
def __init__(self, arg1, arg2):
super(subclass, self).__init__(arg1)
self.set(arg1, arg2)
def set(self, arg1, arg2):
super(subclass, self).set(arg1)
self.arg2 = arg2
def print(self):
super(subclass, self).print()
print("subclass", self.arg2, self.arg)
x = base(1)
x.print()
x.set(11)
x.print()
y = subclass(2,3)
y.print()
y.set(4,5)
y.print()
======= update =======
I rewrote the code to look like this:
class base:
def __init__(self):
print("base init")
self.print()
def print(self):
print("base print")
class subclass(base):
def __init__(self):
print("sc init")
super(subclass, self).__init__()
print("sc after super")
self.print()
def print(self):
print("subclass print start")
super(subclass, self).print()
print("subclass print")
y = subclass()
print("--")
y.print()
when I run I get this output:
sc init
base init
subclass print start <<<< why is the subclass print called here
base print
subclass print
sc after super
subclass print start
base print
subclass print
--
subclass print start
base print
subclass print
why does the self.print in the base init call the subclass print when I'm initing the subclass? I was expecting that to call the base print. it does call the base print when I call it outside of the init.
Your subclass print explicitly calls the superclass one. So every time subclass.print is called, both "base" and "subclass" will be printed. This happens three times, because you call the print method three times: in subclass.__init__, in base.__init__ (which is called by subclass.__init__), and in subclass.print (which calls the superclass version).
In your "set" example, subclass.__init__ calls base.__init__, which tries to call self.set with just one argument. But since you are instantiating subclass, self.set is subclass.set, which takes two arguments.
It's unclear what you're trying to achieve with these examples. Your subclass doesn't really need to call base.__init__, because all that would do is call base.set, and you're already calling that from subclass.set. So even if you succeeded with all your calls, it would result in some methods getting called multiple times, just like with the print example.
My impression is that you're getting a bit carried away and trying to have every method call its superclass version. That's not always a good idea. If you write a subclass, and it calls a superclass method, you need to make sure that the subclass still provides an interface that's compatible with what the superclass expects. If it doesn't, you may need to not call the superclass method and instead have the subclass incorporate its functionality "inline" (although this may be more risky if other classes out in the world have made assumptions about how the base class works). The upshot is that you always need to think about what methods call which others; you can't just call every superclass method everywhere and expect that to work.

Object that raises exception when used in any way

I need to create an object that would raise a custom exception, UnusableObjectError, when it is used in any way (creating it should not create an exception though).
a = UnusableClass() # No error
b = UnusableClass() # No error
a == 4 # Raises UnusableObjectError
'x' in a # Raises UnusableObjectError
for i in a: # Raises UnusableObjectError
print(i)
# ..and so on
I came up with the code below which seems to behave as expected.
class UnusableObjectError(Exception):
pass
CLASSES_WITH_MAGIC_METHODS = (str(), object, float(), dict())
# Combines all magic methods I can think of.
MAGIC_METHODS_TO_CHANGE = set()
for i in CLASSES_WITH_MAGIC_METHODS:
MAGIC_METHODS_TO_CHANGE |= set(dir(i))
MAGIC_METHODS_TO_CHANGE.add('__call__')
# __init__ and __new__ must not raise an UnusableObjectError
# otherwise it would raise error even on creation of objects.
MAGIC_METHODS_TO_CHANGE -= {'__class__', '__init__', '__new__'}
def error_func(*args, **kwargs):
"""(nearly) all magic methods will be set to this function."""
raise UnusableObjectError
class UnusableClass(object):
pass
for i in MAGIC_METHODS_TO_CHANGE:
setattr(UnusableClass, i, error_func)
(some improvements made, as suggested by Duncan in comments)
Questions:
Is there an already existing class that behaves as described?
If not, is there any flaw in my UnusableClass() (e.g., situations when using the instances of the class wouldn't raise an error) and if so, how can I fix those flaws?
Turns out metaclasses and dunder (double underscore) methods don't go well together (which is unfortunate, since that would have been a more streamlined way to implement this).
I couldn't find any importable listing of magic method names, so I created one and put it on PyPi (https://pypi.python.org/pypi/magicmethods/0.1.1). With it, the implementation of UnusableClass can be written as a simple class decorator:
import magicmethods
class UnusableObjectError(Exception):
pass
def unusable(cls):
def _unusable(*args, **kwargs):
raise UnusableObjectError()
for name in set(magicmethods.all) - set(magicmethods.lifecycle):
setattr(cls, name, _unusable)
return cls
#unusable
class UnusableClass(object):
pass
magicmethods.lifecycle contains __new__, __init__, and __del__. You might want to adjust this..
This implementation also handles:
a = UnusableClass()
with a:
print 'oops'
You can use __getattribute__ to block all access to attributes, except special __ attributes like __contains__ or __eq__ which are not catched by __getattribute__, and use a whitelist to allow access to some methods:
class UnuseableClass(object):
whitelist = ('alpha', 'echo',)
def __init__(self):
self.alpha = 42
def echo(self, text):
print text
def not_callable(self):
return 113
def __getattribute__(self, name):
if name in type(self).whitelist:
return super(UnuseableClass, self).__getattribute__(name)
else:
raise Exception('Attribute is not useable: %s' % name)
unuseable_object = UnuseableClass()
print(unuseable_object.alpha)
unuseable_object.echo('calling echo')
try:
unuseable_object.not_callable()
except Exception as exc:
print(exc.message)
If you really need to catch even special method calls, you can use How to catch any method called on an object in python?.

Construct object via __init__ and ignore constructor exception

I have a Python class whose __init__ method raises a custom exception called WrongFileSpecified.
However, when I write a unit test, I want to assign the attributes of the instance object from a test fixture. So normally what I would be doing is reading data off a file and then working with the instance object.
But with the test, I cannot use any test files, so I basically need to hard code the data in the instance object in the setUp method of the unit test. Is there any way to get a instance created without __init__ complaining about the exception?
Sample code:
class A(object):
def __init__(self, folderPath):
#check folder path using os.isdir() otherwise raise exception
#...
self.folderPath = folderPath
#Call load record
self._load_records() #uses self.folderPath and raises exceptions as well
#Note i cannot avoid raising these exceptions, its required
class TestA(unittest.TestCase):
.......
obj = None
def setUp(self):
obj = A('fake folder path')
obj.val1 = "testparam1"
obj.param2 = "testparam2"
def test_1(self):
.....
You can create an empty object, bypassing __init__ by using __new__.
obj = obj_type.__new__(obj_type)
Note that obj_type is the appropriate type object. This is a little hacky but it works. You are reponsible for setting the object's members.
Edit: here is an example.
class Foo():
def __init__(self):
self.x = 1
self.y = 2
def say_hello(self):
print('Hello!')
r = Foo.__new__(Foo)
r.say_hello()
print(r.x)
Console output:
Hello!
Traceback (most recent call last):
File "C:\WinPython-64bit-3.3.5.7\python-
3.3.5.amd64\Scripts\projects\luc_utils\dev\test\
unit_test_serialization.py", line 29, in <module>
print(r.x)
AttributeError: 'Foo' object has no attribute 'x'
Here are two options:
Refactor the file loading out to a class method, which is the Pythonic method of providing an alternate constructor (see below); or
Provide an additional parameter to __init__ to suppress the exceptions when necessary (e.g. def __init__(self, folderPath, suppress=False), or validate=True, whichever makes more sense for your usage).
The latter is a bit awkward, in my opinion, but would mean that you don't have to refactor existing code creating A instances. The former would look like:
class A(object):
def __init__(self, ...):
"""Pass whatever is loaded from the file to __init__."""
...
#classmethod
def from_file(cls, folderPath):
"""Load the data from the file, or raise an exception."""
...
and you would replace e.g. a = A(whatever) with a = A.from_file(whatever).
There is a very useful module called mock, you can check it out later, I feel that in this case it will be too much. Instead, you should consider redesigning your class, like this, for example:
class A(object):
def __init__(self, folderPath):
self.folderPath = folderPath
def _load_records(self)
#check folder path using os.isdir() otherwise raise exception
...
#uses self.folderPath and raises exceptions as well
...
#classmethod
def load_records(cls, folderpath):
obj = cls(folderpath)
obj._load_records()
return obj
# Usage
records = A.load_records('/path/to/records')
Then you can do:
class TestA(unittest.TestCase):
.......
obj = None
def setUp(self):
self.obj = A('fake folder path')
self.obj.val1 = "testparam1"
self.obj.param2 = "testparam2"
def test_1(self):
self.assertRaises(self.obj._load_records, HorribleFailureError)
Also i highly recommend to check out pytest, it has wonderful facilities for test fixtures, including fixtures for files and folders.

Categories

Resources