I'm doing a Data Science project in Python in which many functions and objects have been refactored into classes. One of these classes is the Universe() which contains all names and attributes in our database. As you can imagine, it's huge (takes well over 5 minutes to load) and it's imported in at least two other classes (thus slowing the other class instantiations down considerably).
I'd like to make it so that in the running kernel this class is only instantiated once even when imported from other classes (thus minimizing Database interaction). I've thought of using static classes but from what I've read python developers absolutely hate static classes and consider it non-pythonic.
Are there any other more pythonic solutions to this problem? How would you define a class that encapsulates data and minimizes Database interaction for all other classes that use it?
You can create and instantiate a class in its own module, at module level. That module will only be imported once, and thus the class is only instantiated once.
Example:
file: uni.py:
class _Universe:
def __init__(self):
print('init')
self.get_database_info()
def get_database_info(self):
print('db info')
Universe = _Universe()
file: unia.py:
import unib
from uni import Universe
file: unib.py:
from uni import Universe
Executing file unia.py suggest the Universe class is imported twice, but looking at the output (from the print functions), you'll see instantiating this happens once: Python is smart enough to re-use the already imported module:
python unia.py:
output:
init
db info
Note that I'm using _Universe, so the class instance can be called Universe, to ease code changes
If you have code where Universe() is instantiated (outside the uni module), you could cheat somewhat and add the __call__() method to _Universe. Though it'd be better and clearer to rewrite those little bits of code from e.g. universe = Universe() to Universe:
class _Universe:
def __init__(self):
print('init')
self.get_database_info()
def get_database_info(self):
print('db info')
def __call__(self):
return self
and now in e.g. unia.py, you could have:
import unib
from uni import Universe
universe = Universe()
without repercussions.
Since that, however, may lead to confusion (why "instantiate" an instance?), it's better not to. Perhaps just for an initial quick test and timing of results.
Related
I use RaspberryPi3 with Python to Remote control GPIO of other RPIs.
I created a class to initialize connections and pin for all Pis:
class relay(pigpio.pi):
def __init__(self,ip_addr):
pigpio.pi.__init__(self)
self.GPIO=[4,5,6,12]
self.rpi=pigpio.pi(ip_addr)
for t in range(len(self.GPIO)):
self.rpi.write(self.GPIO[t],0)
def switch_state(self,i,state):
self.rpi.write(self.GPIO[i],state)
pi_1=relay('192.168.2.112') # creating first Rpi link
pi_2=relay('192.168.2.113') # creating second Rpi link
x=0
pi_1.switch_state(x,0)
how can I inherit pigpio module's attributes into relay ? in order not to create switch_state as I did, but to use read, write and more that belong to pigpio
If I'm right you want to extend a module to a class by inheritance.
If that is true, you can not perform that without hacky things but anyway, you should not do that.
Module and classes are not designed to be used in that way.
You better keep using module's functions as expected, I see no good reasons to map module's function within a class.
You could just map it by hand like:
import spam
class Foo():
def egg(self, *args, **kwargs):
return spam.egg(*args, **kwargs)
But again, I'm not sure there is a valid reason to do that - but there is plenty of valid reasons to not..
I'm fairly new to testing in Python and I've come upon some weird behavior which I can't understand.
I have mocked class MockedClass and I'm using it in two test classes like this (both classes are in the same file):
First class:
class TestClass1(unittest.TestCase):
def setUp(self):
self.mocked_object_1 = MockedClass()
Second class:
class TestClass2(unittest.TestCase):
def setUp(self):
self.mocked_object_2 = MockedClass()
Mocked objects are passed as arguments to tested functions. What happens now is that when these two tests are run separately (in different files), all tests in those classes are successful, but when they are both run in the same file, all test cases in TestClass2 fail.
I managed to solve this (somehow) by declaring global temp_mocked_object and doing this:
class TestClass1(unittest.TestCase):
def setUp(self):
temp_mocked_object = self.mocked_object_1
self.mocked_object_1 = MockedClass()
def tearDown(self):
self.mocked_object_1 = temp_mocked_object
But I don't see why would this problem even occur, much less do I understand why this solution worked. What confuses me the most is that second test fails despite different mocked object names.
Mocked class is structured this way:
class MockedClass(someScript.OriginalClass):
def __init__(self):
# some unimportant objects are initialized here
I'm working in older Python version (2.7), but I think that doesn't matter.
So I would greatly appreciate some kind of explanation about what's going on here, since I couldn't find the answer elsewhere.
Background:
I have been working on a game in Python, and in order to keep everything clean, organized and in a pythonic way, I have a folder containing multiple python files, each containing one big class, for example "MapEngine" or "NPCEngine".
From main.py, I am loading each class from each file and "glueing everything together with a "Game" class, such as:
from folder import *
class Game:
def __init__(self):
self.MapEngine ＝ MapEngine.MapEngine()
...
def loop(self):
...
Since classes such as "CollisionEngine" requires data from other classes such as, "MapEngine", I usually assign some variables in the former (i.e. CollisionEngine) to the latter (i.e MapEngine), in order to use MapEngine's loaded map data or functions:
class CollisionEngine:
def __init__(self, MapClass, ...):
self.MapEngine ＝ MapClass
Problem:
Well, since many classes have to be linked to others, it became hard after a while to figure out which class to load first in order to assign variables. Furthermore, classes like "EventEngine" need to have access to every other class. My code became hard to read, and I have trouble when 2 classes are equally important to each other.
Question:
I have heard of class inheritance, but I do not think it can be applied here because each class is very different as in its function. Therefore, is there a way to beautifully link every class together, as if it was all part of one big class? In other words, is there a way to refer to variables from other classes, from within a class?
(My thoughts: Perhaps, I can write a class called "Request", and it will act as a top level class manager. Although, I think I will have to use functions such as exec() or eval(), which are not efficient and are somewhat dangerous.)
This is my first post, I've tried to be as explicit as possible, please ask me for clarification, & thank you for your reply!
Consider separating your project into layers - that should help you keep things more organised and make the imports more natural.
The principle is that lower layers of your code "cake" shouldn't depend on (read: import) upper layers of your code.
For example you might have a foundation layer which contains common data structures, util classes and algorithms that are used in lots of your code at various layers.
Then you might have a model layer which depends on the foundation layer (i.e. data structures/utils/algorithms) but nothing else. The model layer provides models of objects within the domain.
You might then have a game layer which depends on the model layer (so it would be quite reasonable for modules in your game layer to import things from the model layer, but not vice versa).
Well, after many tries, I have figured out a (sketchy) way of solving my problem. Of course, as eddiewould suggested, I will have a better organization and multiple layers for my code, but if one would like to have multiple classes all linked together, simply include a variable to the main class (that called every class) to every class. I believe that a code snippet will explain it better:
main.py
engine_folder
----> engine_1.py
----> engine_2.py
in main.py, engine_1 and engine_2 are loaded:
from engine_folder import engine_1, engine_2
class game:
def __init__(self):
self.engine_1 = engine_1.engine(self, ...)
self.engine_2 = engine_2.engine(self, ...)
#where engine_1.engine and engine_2.engine are
#two classes that need to transfer data between
#each other
def run(self):
self.engine_1.run()
self.engine_2.run()
Notice how engine_1.engine's first argument is self, which refers to the top level class which called this class. Now, in engine_1, if we would want to print a variable from engine_2, the class would look similar to this:
class engine:
def __init__(self, top_level_class, ...):
self.top_level_class = top_level_class
def run(self):
print self.top_level_class.engine_2.random_var
This is very beautiful (besides the fact that print self.top_level_class.engine_2.random_var is very long), but compared to something like:
class EventEngine:
def __init__(self, Camera_Class, Map_Class, Character_Class, Skill_Class,
Interface_Class, Sprite_Class, Dialog_Class, Game_Class,
Item_Class):
self.ItemEngine = Item_Class
self.GameEngine = Game_Class
self.DialogEngine = Dialog_Class
self.SpriteEngine = Sprite_Class
self.SkillEngine = Skill_Class
self.CameraEngine = Camera_Class
self.MapEngine = Map_Class
self.CharacterEngine = Character_Class
self.IEngine = Interface_Class
The new version:
class EventEngine:
def __init__(self, top_level_class):
self.top = top_level_class
#a var from Map_Class can be called as such:
#self.top.MapEngine.map[0][1]
#and I can call variables from every class, not only those I
#have loaded like before
is much better and much cleaner.
In a large python project (openerp) I encounter several times the following pattern:
In a module, a class with its methods is defined. Then, in the same module and immediately after the class definition, an instance of the class is instantiated, that is then called from other modules.
# in module_A.py:
class ClassA(object):
def __init__(self, default="Hello world!"):
self.default = default
def my_method(self, data):
print self.default
print data
object_a = ClassA()
To me it looks simpler to define the methods as module functions, without the class lookup overload:
# in module_B.py:
default = "Hello world!"
def my_method(data):
print default
print data
Seen from other modules, the usage is very similar:
from module_a import object_a as prefix
prefix.my_method("I'm so objective!")
versus:
import module_b as prefix
prefix.my_method("I'm so modular!")
Is there any rationale to prefer pattern A over pattern B? Or is pattern B more pythonic?
Sometimes, you want different clients to be able to use your module with different settings in such a way that they don't conflict with each other. For example, Python's random module provides a bunch of random number generation functions that are actually bound methods of a hidden Random instance. Most users don't care too much what algorithm generates their random numbers or whether other modules asking for random numbers will change the sequence. However, users who do care can get their own Random object and generate sequences of random numbers that won't be affected by other modules asking for random numbers.
Sometimes, something that's global now might not always be global. For example, if you're working on a planetary-scale RTS, you might have a Planet class with one instance, because the battle only happens on one planet. However, you don't want to rule out the possibility of building something like Planetary Annihilation, with battles stretching across entire solar systems and dropping extinction-event asteroids as superweapons. If you get rid of the Planet class and make its methods and attributes module-level, it'll be much harder to go back and add more planets later.
Sometimes, it's more readable to have objects doing things instead of modules. For example, suppose module joebob defines two objects evil_overlord_bob and good_guy_joe.
class Bob(object):
def slaughter_everything(self):
print "Muahahaha! Die for my amusement!"
class Joe(object):
def stop_bob(self):
print "I won't let you hurt those innocents!"
evil_overlord_bob = Bob()
good_guy_joe = Joe()
Suppose Bob and Joe are very unique people. It's unthinkable that you'd want to create another object anything like Bob or Joe. In that case, you could move slaughter_everything and stop_bob to module-level and get rid of the Bob and Joe classes and objects entirely. However, then you'd be writing
joebob.slaughter_everything()
joebob.stop_bob()
It's much clearer what's going on if you can say
evil_overlord_bob.slaughter_everything()
good_guy_joe.stop_bob()
even if you'll never need to instantiate Bob's equally-evil twin brother greg_the_fleshripper.
Among other benefits, using classes allows you to use the introspection on the instances, which is something you cannot do with functions.
In a more general way, both approaches are "pythonic". Use one of the other really depends on the type of project (small/big, with/without GUI, ...)
I'm developing a PyQT4 application, and it's getting pretty hard for me to navigate through all of the code at once. I know of the import foo statement, but I can't figure out how to make it import a chunk of code directly into my script, like the BASH source foo statement.
I'm trying to do this:
# File 'functions.py'
class foo(asd.fgh):
def __init__(self):
print 'foo'
Here is the second file.
# File 'main.py'
import functions
class foo(asd.fgh):
def qwerty(self):
print 'qwerty'
I want to include code or merge class decelerations from two separate files. In PHP, there is import_once('foo.php'), and as I mentioned previously, BASH has source 'foo.sh', but can I accomplish this with Python?
Thanks!
For some reason, my first thought was multiple inheritance. But why not try normal inheritance?
class foo(functions.foo):
# All of the methods that you want to add go here.
Is there some reason that this wont work?
Since you just want to merge class definitions, why don't you do:
# main.py
import functions
# All of the old stuff that was in main.foo is now in this class
class fooBase(asd.fgh):
def qwerty(self):
print 'qwerty'
# Now create a class that has methods and attributes of both classes
class foo(FooBase, functions.foo): # Methods from FooBase take precedence
pass
or
class foo(functions.foo, FooBase): # Methods from functions.foo take precedence
pass
This takes advantage of pythons capability for multiple inheritance to create a new class with methods from both sources.
You want execfile(). Although you really don't, since redefining a class, uh... redefines it.
monkey patching in python doesn't work in nearly the same way. This is normally considered poor form, but if you want to do it anyways, you can do this:
# File 'functions.py'
class foo(asd.fgh):
def __init__(self):
print 'foo'
the imported module remains unchanged. In the importing module, we do things quite differently.
# File 'main.py'
import functions
def qwerty(self):
print 'qwerty'
functions.foo.qwerty = qwerty
Note that there is no additional class definition, just a bare function. we then add the function as an attribute of the class.