I have a code with several functions defined which I call from a main container code. Each new function uses variables obtained with the previous functions, so it looks kind of like this:
import some_package
import other_package
import first_function as ff
import secon_function as sf
import third_function as tf
import make_plot as mp
# Get values for three variables from first function
var_1, var_2, var_3 = ff()
# Pass some of those values to second function and get some more
var4, var5 = sf(var_1, var_3)
# Same with third function
var_6, var_7, var_8, var_9 = tf(var_2, var_4, var_5)
# Call plotting function with (almost) all variables
mp(var_1, var_2, var_3, var_5, var_6, var_7, var_8, var_9)
Is this more pythonic than using global variables? The issue with this methodology is that if I add/remove a new variable from a given function I'm forced to modify four places: the function itself, the call to that function in the main code, the call to the make_plot function in the main and the make_plotfunction itself. Is there a better or more recommended way to do this?
What about putting them in a class?
class Foo(object):
def ff(self):
self.var_1, self.var_2, self.var_3 = ff()
def sf(self):
self.var_4, self.var_5 = sf(self.var_1, self.var_2)
def tf(self):
self.var_6, self.var_7, self.var_8, self.var_9 = tf(self.var_2, self.var_4, self.var_5)
def plot(self):
mp(self.var_1, self.var_2, self.var_3,
self.var_5, self.var_6, self.var_7, self.var_8, self.var_9)
foo = Foo()
foo.ff()
foo.sf()
foo.tf()
foo.plot()
Maybe some of these methods should be module-level functions that take a Foo instance, maybe some of these attributes should be variables passed around separately, maybe there are really 2, or even 4, different classes here rather than 1, etc. But the idea—you've replaced 9 things to pass around with 1 or 2.
I'd suggest that what you want is a data structure which is filled in by the various functions and then passed into make_plot at the end. This largely applies in whatever language you're using.
Related
how do you mock a global variable in pytest? Here is a pair of example files:
File being tested, call it main.py:
MY_GLOBAL = 1
def foo():
return MY_GLOBAL*2
def main()
# some relevant invokation of foo somewhere here
if __name__=='__main__':
main()
File that is testing, call it test_main.py:
from main import foo
class TestFoo(object):
def test_that_it_multiplies_by_global(self):
# expected=2, we could write, but anyway ...
actual = foo()
assert actual == expected
This is just a dummy example of course, but how would you go about mocking MY_GLOBAL and giving it another value?
Thanks in advance i've been kind of breaking my head over this and i bet it's really obvious.
The global variable is an attribute of the module, which you can patch using patch.object:
import main
from unittest.mock import patch
class TestFoo(object):
def test_that_it_multiplies_by_global(self):
with patch.object(main, 'MY_GLOBAL', 3):
assert main.foo(4) == 12 # not 4
However, you want to make sure you are testing the right thing. Is foo supposed to multiply its argument by 1 (and the fact that it uses a global variable with a value of 1 an implementation detail), or is it supposed to multiply its argument by whatever value MY_GLOBAL has at the time of the call? The answer to that question will affect how you write your test.
It's useful to distinguish between module-level constants and global variables. The former are pretty common, but the latter are an anti-pattern in Python. You seem to have a module-level constant (read-only access to the var in normal production code). Global variables (R/W access in production code) should generally be refactored if possible.
For module constants:
If you can do so, it's generally more maintainable to refactor the functions that depend on module constants. This allows direct testing with alternate values as well as a single source of truth for the "constants" and backward compatibility. A minimal refactor is as simple as adding an optional parameter in each function that depends on the "constant" and doing a simple search-and-replace in that function, e.g.:
def foo(value=MY_GLOBAL):
return value*2
All other code can continue to call foo() as normal, but if you want to write tests with alternate values of MY_GLOBAL, you can simply call foo(value=7484).
If what you want is an actual global, (with the global keyword and read/write access during production code, try these alternatives.
In Python, is it always advisable to access import-ed variables without going through the parameters of a function? Doesn't this violate the black-box paradigm?
For instance: given the statement from collections import deque, I wouldn't expect that one would instantiate a deque object to be passed as a parameter along with every function. Instead, I would expect that a deque would be instantiated as needed.
Suppose, though, that the imported object didn't belong to the canonical libraries in Python. Would it be preferred to access such an object through the parameters of a function, or through the global scope?
Edit:
To help illustrate what I mean, take for instance the code below:
from collections import deque
def my_func():
# this seems to be OK
nodes = deque()
On the other hand, suppose that we had some other kind of object. Would this be encouraged in Python?
from my_module import SomeClass
def my_func():
# SomeClass accessed through global scope
instance_of_some_class = SomeClass()
Doesn't the above violate black-box coding? Alternatively:
from my_module import SomeClass
def my_func(some_class):
# SomeClass accessed through local scope
some_class.do_a_thing()
def main():
# I suppose SomeClass() is being accessed globally here...but this is the crux of my question nonetheless.
instance_of_some_class = SomeClass()
my_func(instance_of_some_class)
I realize that as a matter of design, this may be open to opinion; mainly, I was curious if there is a prescribed recommendation in Python.
In my python package I have an entry_point run.py file which takes the seed (e.g. 42) and the cuda device (e.g. "cuda:0") as command line argument.
Since both of these variables are used throughout the entire package at different places, I don't want to pass them as arguments from function to function. Hence, I did the following:
utils.py:
import random
import numpy as np
import torch
def set_device(device: str):
global _DEVICE
_DEVICE = torch.device(device)
def get_device() -> torch.device:
return _DEVICE
def set_seed_number(seed: int):
global _SEED
_SEED = seed
def set_seeds():
torch.manual_seed(_SEED)
random.seed(_SEED)
np.random.seed(_SEED)
And then within run.py I set these variables once by calling:
from package.utils import set_device, set_seed_number
...
set_device(device)
set_seed_number(seed=seed)
Now I can import and call the get_device()and set_seeds method from anywhere in my package and I don't have to pass these variables as arguments.
So far this approach works fine, but after reading that using globals in python is strongly discouraged I am wondering if there is a more pythonic way of achieving the above discribed goal?
I already thought of having a dedicated Singleton class, which dynamically would instantiate those constants but I am not exactly sure if and how that would work and if it would be considered more "pythonic" after all.
Thanks already for your answers and maybe you can point me to some patterns that seem applicable in this situation. I can only guess that I am not the first one trying to achieve the above discribed goal.
I can't honestly see a problem with global if it is used sparingly and only when there is a strong reason to do so. (I think the strong discouragement aganst global is because it is often abused.)
But as regards your proposed custom class, there is no need to instantiate it -- you can just set class variables.
main.py
import settings
settings.set_foo(3)
print(settings.Settings.foo)
settings.py
class Settings:
pass
def set_foo(x):
Settings.foo = x
This is no different in principle from putting your data items inside some other mutable collection e.g. a dictionary and then setting them inside functions in the module that defines it (or another that imports it).
main.py
import settings
settings.set_foo(3)
print(settings.settings['foo'])
settings.py
settings = {}
def set_foo(x):
settings['foo'] = x
I am trying to define some constants at the top of my file. There are no classes in the file, just imports, constants, and methods. Possibly due to poor design, I want to use a method inside of this file to set a constant. For example:
MY_CONSTANT = function(foo, bar)
def function(foo, bar):
return 6
In this example, I want MY_CONSTANT to get assigned the int 6. This is a simplified version of the problem, as my function actually makes many expensive calls and I only want that function to be called once. I plan to use the constant inside of a loop.
This does not work because I get the following error:
NameError: name 'function' is not defined
Is there a better design for this, or how can I use a method call to set my constant?
You are trying to call a function before it has been defined:
def function(foo, bar):
return 6
MY_CONSTANT = function(foo=None, bar=None)
>>>MY_CONSTANT
6
Edit: I set foo=None and bar=None going into function because I'm not sure where you have those defined either.
So I have a lot of functions in a file called definitions.py
My main file main.py which accesses those definitions and also has functions of its own using those functions.
Now in main.py I have 'from definitions import *'
Both files rely on a set of 15 initial variables, these variables I have placed in definitions.py, this is all well and good I have all my functions working fine, the problem arises when I want to use my application as a model, where I will want to change some of the variables to see how the output differs.
Essentially I want my initial variables to be in a sort of bowl which is accessed each time a function is called and I can swap and change the values in this bowl which means the next function that is called uses the updated variables.
The problem I'm having at the moment is, I think, because the variables are written in definitions.py that's that and I can't change them.
Even in python shell I can put n1 equal to something else, execute a function that uses n1 but it will use the old n1, not the new one, I think because the variables haven't changed in the definition.py file.
Is there some sort of way to have live access variables that I don't know about? Thank you.
You should use a class. For example, if your definitions.py file has:
variable1 = 3
variable2 = 'stuff'
def spam(arg1):
return 'spam' + arg1
def eggs(arg1,arg2):
return 'eggs' + arg1 + arg2
change it to
class Definitions():
def __init__():
self.variable1 = 3
self.variable2 = 'stuff'
def spam(self,arg1):
return 'spam' + arg1
def eggs(self,arg1,arg2):
return 'eggs' + arg1 + arg2
Now, from your main.py file, you can import in a slightly different way and sweep multiple parameter values:
import definitions
for parameter in xrange(0,10):
defs = definitions.Definitions()
defs.variable1 = parameter
# do some stuff and store the result
# compare the various results
Remember that now your functions are inside a class, so instead of calling spam('mail'), you should call defs.spam('mail'), and so on.