Develop Reference

Python cannot instantiate an imported class

I have been developing a full-stack application that checks for files and uploads them to a cloud. However, I have come across an interesting problem that I was not able to solve.
I have a problem with instantiating a class, as you will see below:
class UploadFastq:
def __int__(self,
some_list, some_str, some_obj, **kwargs):
self.some_list = some_list
self.some_obj = some_obj
self.some_str = some_str
def process(self):
self.some_methods_calling_processes()
...
As you can imagine, I have trimmed the original code for privacy concerns (company dictates, sorry). This class is to handle some-backend related processes, and arguments only contain back related variables. Also, this class is on the different py script, which imports again back-related functions.
Now, the problem is, when I import to another script and try to call and instantiate the class, something funny happens...
from lib.some_back_related_script import UploadFastq
uploads = UploadFastq(some_list=the_list,some_str=the_str,some_obj=the_obj)
uploads.process
OUTPUT:
TypeError: UploadFastq() takes no arguments
I have looked if there are indentation problems, I could not find any. (I am using PyCharm as IDE, and reformatting the file also did not solve)
I have also tried this on an another script(the gui script) and could partially solve it as:
from lib.some_back_related_script import UploadFastq
uploader = UploadFastq()
uploader.__int__( ##TODO how is this possible???)
some_list=the_list,some_str=the_str,some_obj=the_obj
)
However, on the script the class suppose to be called, "__init__" method did not solve the case, and produced this error:
TypeError: UploadFastq.__init__() takes exactly one argument (the instance to initialize)
At this point I am clueless about what is going on and how to solve it. I have experiencing something like this for first time. I also could not find this kind of problem on the internet. soo, I would be much grateful if you could explain how to approach the problem.
P.S.: I work as a bioinformatician/python developer for a quite time and I found many many solutions on this platform. But, this is actually my first question on the stackoverflow!!!
Cheers!

You mispelled the constructor name __init__ as __int__ :
def __int__(self, some_list, some_str, some_obj, **kwargs):
Thus the default constructor (which takes as arguments only the "instance to initialize") was called, and the interpreter is complaining about the given arguments.
TypeError: UploadFastq.__init__() takes exactly one argument (the instance to initialize)

Parameterized generics cannot be used with class or instance checks

I wrote the code, but I get the following message in pycharm(2019.1):
"Parameterized generics cannot be used with class or instance checks"
def data_is_valid(data):
keys_and_types = {
'comment': (str, type(None)),
'from_budget': (bool, type(None)),
'to_member': (int, type(None)),
'survey_request': (int, type(None)),
}
def type_is_valid(test_key, test_value):
return isinstance(test_value, keys_and_types[test_key])
type_is_valid('comment', 3)
I really do not understand this message well. Did I do something wrong or is it a bug in pycharm?
The error disappears if I explicitly typecast to tuple.
def type_is_valid(test_key, test_value):
return isinstance(test_value, tuple(keys_and_types[test_key]))

That looks like a bug in pycharm where it's a bit overeager in assuming that you're using the typing module in an unintended way. See this example here where that assumption would have been correct:
The classes in the typing module are only useful in a type annotation context, not to inspect or compare to actual classes, which is what isinstance tries to do. Since pycharm sees a simple object with square brackets that do not contain a literal, it jumps to the wrong conclusion you are seeing.
Your code is fine, you can use it exactly as it is.

I will not repeat after others that this is a pycharm bug. Just if you are a perfectionist and the error hurts your eyes, add the comment
# noqa
to the line where the "error" is

This was a known bug in PyCharm 2018, reported here.
There are some related bugs still in more recent PyCharm versions, e.g. PyCharm 2021.2.2, here.
In general, when you found that some PyCharm warning is incorrect, I would first isolate a simple test case where it becomes more clear what PyCharm is actually incorrect about. When it is clear that PyCharm is wrong with the warning, then you should always fill a bug report about it (or maybe search for existing bug reports first). Here this is clear because PyCharm says you cannot do sth, while in fact you can, so sth is wrong.

Since it's agreed it's a bug, you can suppress it in Pycharm by the line:
# noinspection PyTypeHints

Overwriting methods via mixin pattern does not work as intended

I am trying to introduce a mod/mixin for a problem. In particular I am focusing here on a SpeechRecognitionProblem. I intend to modify this problem and therefore I seek to do the following:
class SpeechRecognitionProblemMod(speech_recognition.SpeechRecognitionProblem):
def hparams(self, defaults, model_hparams):
SpeechRecognitionProblem.hparams(self, defaults, model_hparams)
vocab_size = self.feature_encoders(model_hparams.data_dir)['targets'].vocab_size
p = defaults
p.vocab_size['targets'] = vocab_size
def feature_encoders(self, data_dir):
# ...
So this one does not do much. It calls the hparams() function from the base class and then changes some values.
Now, there are already some ready-to-go problems e.g. Libri Speech:
#registry.register_problem()
class Librispeech(speech_recognition.SpeechRecognitionProblem):
# ..
However, in order to apply my modifications I am doing this:
#registry.register_problem()
class LibrispeechMod(SpeechRecognitionProblemMod, Librispeech):
# ..
This should, if I am not mistaken, overwrite everything (with identical signatures) in Librispeech and instead call functions of SpeechRecognitionProblemMod.
Since I was able to train a model with this code I am assuming that it's working as intended so far.
Now here comes the my problem:
After training I want to serialize the model. This usually works. However, it does not with my mod and I actually know why:
At a certain point hparams() gets called. Debugging to that point will show me the following:
self # {LibrispeechMod}
self.hparams # <bound method SpeechRecognitionProblem.hparams of ..>
self.feature_encoders # <bound method SpeechRecognitionProblemMod.feature_encoders of ..>
self.hparams should be <bound method SpeechRecognitionProblemMod.hparams of ..>! It would seem that for some reason hparams() of SpeechRecognitionProblem gets called directly instead of SpeechRecognitionProblemMod. But please note that it's the correct type for feature_encoders()!
The thing is that I know this is working during training. I can see that the hyper-paramaters (hparams) are applied accordingly simply because the model's graph node names change through my modifications.
There is one specialty I need to point out. tensor2tensor allows to dynamically load a t2t_usr_dir, which are additional python modules which get loaded by import_usr_dir. I make use of that function in my serialization script as well:
if usr_dir:
logging.info('Loading user dir %s' % usr_dir)
import_usr_dir(usr_dir)
This could be the only culprit I can see at the moment although I would not be able to tell why this may cause the problem.
If anybody sees something I do not I'd be glad to get a hint what I'm doing wrong here.
So what is the error you're getting?
For the sake of completeness, this is the result of the wrong hparams() method being called:
NotFoundError (see above for traceback): Restoring from checkpoint failed.
Key transformer/symbol_modality_256_256/softmax/weights_0 not found in checkpoint
symbol_modality_256_256 is wrong. It should be symbol_modality_<vocab-size>_256 where <vocab-size> is a vocabulary size which gets set in SpeechRecognitionProblemMod.hparams.

So, this weird behavior came from the fact that I was remote debugging and that the source files of the usr_dir were not correctly synchronized. Everything works as intended but the source files where not matching.
Case closed.

AssertionError, altough the expected call looks same as actual call

I made a command in django which calls a function.
That function does a django orm call:
def get_notes():
notes = Note.objects.filter(number=2, new=1)
return [x.note for x in notes]
I want to patch the actual lookup:
#mock.patch('Note.objects.filter', autospec=True)
def test_get_all_notes(self, notes_mock):
get_notes()
notes_mock.assert_called_once_with(number=2, new=1)
I get the following assertion error:
AssertionError: Expected call: filter(number=2, new=1)
Actual call: filter(number=2, new=1)
I search on google and stackoverflow for hours, but I still haven't a clue.
Can anyone point me in the right direction, I think it might be an obvious mistake I'm making...

AFAIK you can't use patch() like this. Patch target should be a string in the form package.module.ClassName. I don't know much about django but I suppose Note is a class so Note.objects.filter is not something you can import and hence use in patch(). Also I don't think patch() can handle attributes. Actually I don't quite understand why the patch works at all.
Try using patch.object() which is specifically designed to patch class attributes. It implies Note is already imported in your test module.
#mock.patch.object(Note, 'objects')
def test_get_all_notes(self, objects_mock):
get_notes()
objects_mock.filter.assert_called_once_with(number=2, new=1)
I've removed autospec because I'm not sure it will work correctly in this case. You can try putting it back if it works.
Another option might be to use patch() on whatever you get with type(Note.objects) (probably some django class).
As I've said I don't know much about django so I'm not sure if these things work.

pylint bug - E1101 & E0102 upon use of #property + #foo.setter

I noticed pylint doesn't handle well the case of:
#property
def foo(self):
return self._bar.foo
#foo.setter
def foo(self, foo_val):
self._bar.foo = foo_val
Though it's a perfectly valid case syntax since python2.6
It says I defined foo twice, and doesn't understand the ".setter" syntax (Gives E1101 & E0102).
Is there a workaround for that without having to change the code? I don't want to disable the errors as they are important for other places.
Is there any other tool I can use that handles it better? I already checked pyflakes and it behaves the same way. PyDev's code analysis seems to handle this specific case better, but it doesn't check for conventions, refactoring, and other cool features pylint does, and I can't run it from an external script (or can I??)
Thanks!

If you don't want to disable the errors globally, you can disable them for these specific lines, for example:
def foo(self, foo_val): # pylint: disable-msg=E0102

This is ticket http://www.logilab.org/ticket/51222 on the pylint project. Monitor it's status.

Huh. Annoying. And all the major tools I could find (pyflakes, pylint, pychecker) exhibit this problem. It looks like the problem starts in the byte code, but I can't get dis to give me any byte code for object properties.
It looks like you would be better off if you used this syntax:
# Changed to longer member names to reduce pylint grousing
class HughClass(object):
def __init__(self, init_value):
self._hugh = init_value
def hugh_setter(self):
return self._hugh * 2
def hugh_getter(self, value):
self._hugh = value / 2
hugh = property(hugh_getter, hugh_setter)
Here's a nice blog article on it. LOL-quote:
Getters and setters belong to the sad
world of Java and C++.

This was reported as a bug in pyflakes, and it appears to be fixed in current trunk. So I guess the answer (now) is: pyflakes!