Testing python classes which leverage class attributes - python

I have a python class which uses class variables to maintain some instance-wide state. However, when attempting to test various scenarios via pytest I'm realizing that the class-level attributes are being shared across the tests since it's the same interpreter being used. Is there any pattern or way around this outside of running each class as a separate invocation of the pytest command or resetting every class attribute manually between tests? It's looking like a big no but wanted to ask and make sure.
p.s. For those who will immediately ask why I'm using class attributes. I'm using class attributes for state because this code will run on AWS Lambda. With this pattern I can cache objects in memory as class-attributes between lambda invocations and be assured that instance attributes are cleared each time the lambda runs. It's an attempt at using some OOP and writing base lambda classes with logging and various helpers already implemented that other devs can leverage

https://docs.python.org/3.3/library/imp.html#imp.reload
When reload(module) is executed:
Python modules’ code is recompiled and the module-level code
reexecuted, defining a new set of objects which are bound to names in
the module’s dictionary. The init function of extension modules is not
called a second time. As with all other objects in Python the old
objects are only reclaimed after their reference counts drop to zero.
The names in the module namespace are updated to point to any new or
changed objects. Other references to the old objects (such as names
external to the module) are not rebound to refer to the new objects
and must be updated in each namespace where they occur if that is
desired.

Related

Is docstring being declared every time the function is called?

The Python official document specifies that a docstring is a string literal that occurs at the beginning of a function. And it can be accessed using the __doc__ attribute.
If I have a function that will be called many many times, does that mean the docstring will be declared every time the function is called?
If this is the case, would it be more efficient to design docstring in such a way that it is stored in __doc__ but not being declared every time the function is called?
every time you start a python program, they are "important" into memory only "once", parsed so that every "object" properties are determined and all objects are put into their separate memory locations and then linked together in the memory to make it a whole running system (remember the object nature of python).
second behavior is when you don't restrict the python interpreter. If you import your files, then, in addition to the above steps, it writes these objects into more durable .pyc files under __pycache__ folder at the same level of the file.
In this process, new objects are created to have a __doc__ property object when certain keywords are parsed, mainly class and def. These __doc__ of each are then either kept empty, filled some default by inheritance, or if it has a docstring then it is written inside.
You can see this behavior on different objects, created with/out supplying a docstring, simply by using dir(objectname). To answer your question, you can use this command throughout your program.
However, this is true only for static written code. If you are trying to make objects on the fly, especially within loops, then your objects will be actively created and destroyed, thus there will be almost no optimization against them and docstrings will be created again and again.
consider these two:
def staticMethod():
pass
for i in range(5):
def activeMethod():
pass
print(staticMethod,"s")
print(activeMethod,"a")
while staticMethod is served from the same memory location, the memory address for activeMethod changes. you will see an altering between few values because python can still optimize since this one is a simple example.
So keep yourself aware of this distinct behavior, especially of loops.

How does python interpreter build objects [duplicate]

This question already has answers here:
Python (and Python C API): __new__ versus __init__ [duplicate]
(6 answers)
Closed 2 years ago.
I recently started following Python after studying Java. I'm confused with the way of python interpreter's object construction.
Compared to Java when we construct an object when simply provide our arguments, for Python that is the case too.
But I can't think why the __init()__ method requires a self parameter when we define it in our class.
I read this question and I got that the methods require a self parameter because python calls a method in the format ClassA.methodA(ObjectA, arg1, arg2).
But I really don't get why the __init()__ method require this.
Is it because the way that Python generate an object differs from the way that Java generates an object.
I really appreciate if someone can explain it to me.
Why must ‘self’ be used explicitly in method definitions and calls?
The idea was borrowed from Modula-3. It turns out to be very useful, for a variety of reasons.
First, it’s more obvious that you are using a method or instance attribute instead of a local variable. Reading self.x or self.meth() makes it absolutely clear that an instance variable or method is used even if you don’t know the class definition by heart. In C++, you can sort of tell by the lack of a local variable declaration (assuming globals are rare or easily recognizable) – but in Python, there are no local variable declarations, so you’d have to look up the class definition to be sure. Some C++ and Java coding standards call for instance attributes to have an m_ prefix, so this explicitness is still useful in those languages, too.
Second, it means that no special syntax is necessary if you want to explicitly reference or call the method from a particular class. In C++, if you want to use a method from a base class which is overridden in a derived class, you have to use the :: operator – in Python you can write baseclass.methodname(self, ). This is particularly useful for __init__() methods, and in general in cases where a derived class method wants to extend the base class method of the same name and thus has to call the base class method somehow.
Finally, for instance variables it solves a syntactic problem with assignment: since local variables in Python are (by definition!) those variables to which a value is assigned in a function body (and that aren’t explicitly declared global), there has to be some way to tell the interpreter that an assignment was meant to assign to an instance variable instead of to a local variable, and it should preferably be syntactic (for efficiency reasons). C++ does this through declarations, but Python doesn’t have declarations and it would be a pity having to introduce them just for this purpose. Using the explicit self.var solves this nicely. Similarly, for using instance variables, having to write self.var means that references to unqualified names inside a method don’t have to search the instance’s directories. To put it another way, local variables and instance variables live in two different namespaces, and you need to tell Python which namespace to use.
reference: https://docs.python.org/3/faq/design.html#why-must-self-be-used-explicitly-in-method-definitions-and-calls

Python/Django and services as classes

Are there any conventions on how to implement services in Django? Coming from a Java background, we create services for business logic and we "inject" them wherever we need them.
Not sure if I'm using python/django the wrong way, but I need to connect to a 3rd party API, so I'm using an api_service.py file to do that. The question is, I want to define this service as a class, and in Java, I can inject this class wherever I need it and it acts more or less like a singleton. Is there something like this I can use with Django or should I build the service as a singleton and get the instance somewhere or even have just separate functions and no classes?
TL;DR It's hard to tell without more details but chances are you only need a mere module with a couple plain functions or at most just a couple simple classes.
Longest answer:
Python is not Java. You can of course (technically I mean) use Java-ish designs, but this is usually not the best thing to do.
Your description of the problem to solve is a bit too vague to come with a concrete answer, but we can at least give you a few hints and pointers (no pun intended):
1/ Everything is an object
In python, everything (well, everything you can find on the RHS of an assignment that is) is an object, including modules, classes, functions and methods.
One of the consequences is that you don't need any complex framework for dependency injection - you just pass the desired object (module, class, function, method, whatever) as argument and you're done.
Another consequence is that you don't necessarily need classes for everything - a plain function or module can be just enough.
A typical use case is the strategy pattern, which, in Python, is most often implemented using a mere callback function (or any other callable FWIW).
2/ a python module is a singleton.
As stated above, at runtime a python module is an object (of type module) whose attributes are the names defined at the module's top-level.
Except for some (pathological) corner cases, a python module is only imported once for a given process and is garanteed to be unique. Combined with the fact that python's "global" scope is really only "module-level" global, this make modules proper singletons, so this design pattern is actually already builtin.
3/ a python class is (almost) a singleton
Python classes are objects too (instance of type type, directly or indirectly), and python has classmethods (methods that act on the class itself instead of acting on the current instance) and class-level attributes (attributes that belong to the class object itself, not to it's instances), so if you write a class that only has classmethods and class attributes, you technically have a singleton - and you can use this class either directly or thru instances without any difference since classmethods can be called on instances too.
The main difference here wrt/ "modules as singletons" is that with classes you can use inheritance...
4/ python has callables
Python has the concept of "callable" objects. A "callable" is an object whose class implements the __call__() operator), and each such object can be called as if it was a function.
This means that you can not only use functions as objects but also use objects as functions - IOW, the "functor" pattern is builtin. This makes it very easy to "capture" some context in one part of the code and use this context for computations in another part.
5/ a python class is a factory
Python has no new keyword. Pythonc classes are callables, and instanciation is done by just calling the class.
This means that you can actually use a class or function the same way to get an instance, so the "factory" pattern is also builtin.
6/ python has computed attributes
and beside the most obvious application (replacing a public attribute by a pair of getter/setter without breaking client code), this - combined with other features like callables etc - can prove to be very powerful. As a matter of fact, that's how functions defined in a class become methods
7/ Python is dynamic
Python's objects are (usually) dict-based (there are exceptions but those are few and mostly low-level C-coded classes), which means you can dynamically add / replace (and even remove) attributes and methods (since methods are attributes) on a per-instance or per-class basis.
While this is not a feature you want to use without reasons, it's still a very powerful one as it allows to dynamically customize an object (remember that classes are objects too), allowing for more complex objects and classes creation schemes than what you can do in a static language.
But Python's dynamic nature goes even further - you can use class decorators and/or metaclasses to taylor the creation of a class object (you may want to have a look at Django models source code for a concrete example), or even just dynamically create a new class using it's metaclass and a dict of functions and other class-level attributes.
Here again, this can really make seemingly complex issues a breeze to solve (and avoid a lot of boilerplate code).
Actually, Python exposes and lets you hook into most of it's inners (object model, attribute resolution rules, import mechanism etc), so once you understand the whole design and how everything fits together you really have the hand on most aspects of your code at runtime.
Python is not Java
Now I understand that all of this looks a bit like a vendor's catalog, but the point is highlight how Python differs from Java and why canonical Java solutions - or (at least) canonical Java implementations of those solutions - usually don't port well to the Python world. It's not that they don't work at all, just that Python usually has more straightforward (and much simpler IMHO) ways to implement common (and less common) design patterns.
wrt/ your concrete use case, you will have to post a much more detailed description, but "connecting to a 3rd part API" (I assume a REST api ?) from a Django project is so trivial that it really doesn't warrant much design considerations by itself.
In Python you can write the same as Java program structure. You don't need to be so strongly typed but you can. I'm using types when creating common classes and libraries that are used across multiple scripts.
Here you can read about Python typing
You can do the same here in Python. Define your class in package (folder) called services
Then if you want singleton you can do like that:
class Service(object):
instance = None
def __new__(cls):
if cls.instance is not None:
return cls.instance
else:
inst = cls.instance = super(Service, cls).__new__()
return inst
And now you import it wherever you want in the rest of the code
from services import Service
Service().do_action()
Adding to the answer given by bruno desthuilliers and TreantBG.
There are certain questions that you can ask about the requirements.
For example one question could be, does the api being called change with different type of objects ?
If the api doesn't change, you will probably be okay with keeping it as a method in some file or class.
If it does change, such that you are calling API 1 for some scenario, API 2 for some and so on and so forth, you will likely be better off with moving/abstracting this logic out to some class (from a better code organisation point of view).
PS: Python allows you to be as flexible as you want when it comes to code organisation. It's really upto you to decide on how you want to organise the code.

Pickling instances of a class whose instances are classes

I have a class whose instances are classes. Each instance has its own name. I can add that name to the enclosing module when the new class is created, which allows me to pickle.dump it. Originally, I didn't add the name to the module, but I had a reduce method in the top-level class. Unfortunately, there's some mysterious code in the pickle module that special-cases subclasses of type and looks for the name in the module rather than calling reduce. If that code were inside a try, and just continued to the reduce code on failure, I think using reduce would work fine.
Anyway, the problem with just adding the name is that on pickle.load, the name doesn't exist and the load fails. Presumably, I could add a getattr to the module to interpret the name and create the corresponding class, but my understanding is that not all versions of python support module-level getattr.
The reduce method in the class instances allows successfully pickling instances of those class instances by recreating the class instances on pickle.load, but pickling the class instances themselves doesn't work.
Other than making a slightly nonstandard pickle module, is there some reasonable solution to allow pickling of the class instances?

Do I have to import the class if an instance of the class will be used in Python?

I am using PRAW. In my code, an instance of a class in PRAW is declared in a module and return. Do I have to import the class or PRAW in my main module to be able to use its sub methods and variables?
The short answer is "no".
If you have an instance of some class, the definition of that class is already lying somewhere in the Python process - if it was defined in a module (in contrast with dynamically constructed using type), that module was imported somewhere else, and the class definition can be found there.
The "import" mechanism in Python is quite efficient, so if you do import a module that is not yet loaded into the current process, its code is actually run. Afterwards, a reference to that module is kept in a central registry, which is exposed in the sys.modules dictionary (that is a plain dictionary in the sys module).
Whenever new module imports, or name imports from modules that are already present there take place, all that Python does is to assign a variable in the scope the import is taking place, referring to the object already in memory - so, import on an "already known" object is just an assignment, quite the same thing the "=" symbol performs.
But when you have an instance of a class, it already contains a reference to its class, there is no need to bring it into the current namespace. (That reference is on the instance's __class__ attribute, btw).

Categories

Resources