Develop Reference

Pass the same dataframe over and over again to different classes and functions in Python

The first step of my program is to get the data.
After that I am finding myself passing this data to all the different classes (with also a config dictionary variable) over and over again.
So I am wondering if there is a better way to just store the data somewhere and make it available to all classes and functions, without passing them as a parameter.
Thank you
Edit: here is a code example
go.py
config = {
'mode' : 'single',
'data' : { },
'strategy' : { },
'view' : { }
}
stratego.start(config)
stratego.py
def start(config):
data = dt.Data(config['data'])
if (config['data']['type'] == 'yahoo'):
df = data.get_yahoo_data()
elif (config['data']['type'] == 'csv'):
df = data.get_csv_data()
else:
return False
trades = str.Strategy(df, config['strategy'])
tradeBook = trades.run()
but then I am realising that the problem is my main function (start). If I run the main code not in a function I have all my instances available in the global. Is that right? Is it correct to do this way or it is better to wrap the program in a main function?

If really you don't want to pass it as an argument you could define it as a variable in a python file and import this variable where you define your fonction. You should be able to use this variable in the function without passing it in argument.

EDIT: Refactored code according to code update by OP
Ok since you use a strategy pattern you can actually do that using a strategy like design pattern
stratego.py
def start(*strategies):
for strategy in strategies:
strategy.run()
go.py
from functools import lru_cache, wraps
from abc import ABC, abstractmethod
import stratego
#lru_cache()
def get_data(filepath):
# Load data from filepath
data = ...
return data
#lru_cache()
def get_data_with_config(**data_config):
# Load data based on data_config
data = get_data(data_config['filepath'])
if (data_config['type'] == 'yahoo'):
df = data.get_yahoo_data()
elif (data_config['type'] == 'csv'):
df = data.get_csv_data()
...
return df
class Strategy(ABC):
def __init__(self, config):
self.config = config
#abstractmethod
def run(self):
pass
class YahooStrategy(Strategy):
def __init__(self, config):
config = config.copy()
config['data']['type'] = 'yahoo'
super().__init__(config)
def run(self):
df = get_data_with_config(**self.config['data'])
# Do sth with data
class CsvStrategy(Strategy):
def __init__(self, config):
config = config.copy()
config['data']['type'] = 'csv'
super().__init__(config)
def run(self):
df = get_data_with_config(**self.config['data'])
# Do sth with data
class FunctionStrategy(Strategy):
def __init__(self, config, func):
super().__init__(config)
self.func = func
def run(self):
return self.func(self.config)
def strategy_decorator(func):
#wraps(func)
def wrapper(config):
return FunctionStrategy(config, func)
return wrapper
#strategy_decorator
def some_strategy_function(config):
df = get_data_with_config(**config['data'])
# Do smth with data
# With one strategy
strategy = YahooStrategy({'data': {'filepath': 'data.csv', ...}})
stratego.run(strategy)
# Multiple strategies
strategies = [
YahooStrategy({'data': {'filepath': 'data.csv', ...}}),
CsvStrategy({'data': {'filepath': 'data2.csv', ...}}),
some_strategy_function({'data': {'filepath': 'data4.csv', ...}})
]
stratego.run(*strategies)

If you're thinking pass by reference vs pass by value then I would suspect you are newer to Python. To my understanding, all variables are passed by reference. That is, you aren't copying the actual data every time you call a function with parameters.
If you're thinking more along the lines of global variables, you can do something like this:
globvar = 0
def set_globvar_to_one():
global globvar # Needed to modify global copy of globvar
globvar = 1
def print_globvar():
print(globvar) # No need for global declaration to read value of globvar

Calling a function within a function in a loop - python

I didn't find a solution in "similar" questions. I like to call a function in a function in a loop. the solution will not be to set "parameters" in front of the function (def inner) I try to call. I like to call the "def inner" in the end of the loop. Thanks!
def meta1():
def inner():
print("hello inner2")
parameters = {"Name": "XYZ","max_time": 35}
return parameters
def meta2():
def inner():
print("hello inner2")
parameters = {"Name": "XXX","max_time": 25}
return parameters
the loop over the functions
for func in [meta1, meta2]:
x = func()
print(x['Name'])
print("here is some other code between. The solution is not to change the functions!")
print(func(inner())) #here i need some help, thanks

Not clear why you want/need to do this but if each meta also returns its inner function then it is easy.
def meta1():
def printer():
print("hello xyz")
parameters = {"Name": "XYZ","max_time": 35}
return parameters,printer
def meta2():
def printer():
print("hello xxx")
parameters = {"Name": "XXX","max_time": 25}
return parameters,printer
for func in [meta1, meta2]:
x,inner = func()
print(x['Name'])
print("here is some other code between. The solution is not to change the functions!")
print(inner())

While it is not clear what you want to do, this code may help
You can copy and paste it into an Idle window, and run it.
The point of my answer is to introduce you to classes, which in Python may be the best fit for the level of abstraction you want. Classes are a major feature of Python, and when you learn about them you will also discover static members and so on. But for now ...
def meta1():
def inner():
print("hello inner2")
parameters = {"Name": "XYZ","max_time": 35}
return parameters
class Meta1ClassExample:
def inner(self):
print("hello inner2")
parameters = {"Name": "XYZ","max_time": 35}
return parameters
if __name__ == '__main__':
try:
meta1.inner() #AttributeError: 'function' object has no attribute 'inner'
except AttributeError:
print("As expected, looking for the inner function is an error")
metaclass = Meta1ClassExample()
p = metaclass.inner()
print (p)

If your goal is to use the inner functions outside meta1 and meta2, respectively, you can try something like this:
def meta1():
def printer():
print("hello xyz")
parameters = {"Name": "XYZ","max_time": 35}
return printer, parameters
def meta2():
def printer():
print("hello xxx")
parameters = {"Name": "XXX","max_time": 25}
return printer, parameters
for func in [meta1, meta2]:
printer_fct, x = func()
print(x['Name'])
print("here is some other code between. The solution is not to change the functions!")
print(printer_fct()) #here i need some help, thanks
Edit: In the last line, you would probably want a simple call to the function printer_fct(), since the application of print only outputs None because printer_fct() does not return anything.

How to combine for_each and if_

I am starting to learn liteflow and wonder if it is possible to somehow combine for_each and if_. I tried the following:
from liteflow.core import *
class Hello(StepBody):
def run(self, context: StepExecutionContext) -> ExecutionResult:
print("Hello")
return ExecutionResult.next()
class DoStuff(StepBody):
def run(self, context: StepExecutionContext) -> ExecutionResult:
print(f"doing stuff...{context.execution_pointer.context_item}")
return ExecutionResult.next()
class Goodbye(StepBody):
def run(self, context: StepExecutionContext) -> ExecutionResult:
print("Goodbye")
return ExecutionResult.next()
class MyWorkflow(Workflow):
def id(self):
return "MyWorkflow"
def version(self):
return 1
def build(self, builder: WorkflowBuilder):
builder\
.start_with(Hello)\
.for_each(lambda data, context: ["abc", "def", "xyz"])\
.if_(lambda d, c: not d.startswith("x"))\
.do(lambda x: x.start_with(DoStuff))\
.then(Goodbye)
host = configure_workflow_host()
host.register_workflow(MyWorkflow())
host.start()
wid = host.start_workflow("MyWorkflow", 1, None)
input()
host.stop()
This is not working ('NoneType' object has no attribute 'startswith'), if I replace the condition with something like 0 < 1 DoStuff is exceuted once instead of three times without the if_ and even prints the "wrong" output.
I know that I can eliminate the if_ by using a LC in for_each in this simple example ([x for x in ["abc", "def", "xyz"] if not x.startswith("x")]), but imagine a more complex workflow.
So how do I combine for_each and if_ in a way to get access to the loop variable from for_each in if_ and also pass the correct context to do? Or is this impossible and I have to check the condition in the run method of my steps?

Wrapping a Python function that uses with

Say have a python function foo() that uses some resource and is meant to be called as follows:
with foo(x,y,z) as f:
doSomething(f)
So far so good. Now lets say foo takes in a complex set of arguments based on a variety of factors, and I'd like to define a wrapper function to make things simpler. Something like:
def simple_foo():
if x:
return foo(a,b,c)
else:
return foo(d,e,f)
Now, I'd like to use simple_foo in place of foo, like:
with simple_foo() as f:
doSomething(f)
However, unsurprisingly, this does not work. How can I write simple_foo() to get this behavior?

Decorate function foo() with contextmanager (doc):
from contextlib import contextmanager
#contextmanager
def foo(a, b, c):
try:
yield a + b + c
finally:
pass
def simple_foo(x):
if x:
return foo(1, 2, 3)
return foo(4, 5, 6)
with simple_foo(True) as v:
print(v)
with simple_foo(False) as v:
print(v)
Prints:
6
15

You can do by writing a custom context manager that internally calls that function, try code given below:
class SimpleFoo:
def __init__(self,x,y,z, option):
self.x = x
self.y = y
self.z = z
self.option = option
def __enter__(self):
if self.option:
return foo(self.x,self.y,self.z)
else:
return foo(self.y,self.z,self.x)
def __exit__(self, type, value, traceback):
if type != None:
print("Error in SimpleFoo")
print("Error Type :", type)
print("Error Value :", value)
print("Error Traceback :", traceback)
self.status = value
Now if you want to use this, use it as below:
with SimpleFoo(1,2,3,True) as foo:
doSomething(foo)
I hope this helps.

Way to call method depending on variable?

I already have a working, but in my oppinion not beautiful solution for a part of a long script.
My script uses several similar methods, that differ too much to combine. However I came to a point where I want to call one of those methods depending on a given variable.
The names of the methods are build up like this:
def read_A():
#doing sth
def read_B():
#doing sth else
def read_C():
etc.
Now I would like to call those methods in a pythonic way, when the letter ('A', 'B', 'C', ...) is given as a variable.
A non-pythonic solution would be:
if var == "A":
read_A()
if var == "B":
read_B() .....
And I hope to find a more pythonic solution that allows me to call those methods simply like this:
var = "A"
read_var() #This would call the method 'read_A()'
Please mind that the code above is only an image of what I hope to do, it is not a working example!

I dont see an issue with just using
if var == 'A':
read_a()
but if you'd like to make it more 'pythonic' you could map your variables to the methods using a dictionary and execute it based on the result of what's stored in your dictionary:
def read_a():
print('Running method read_a')
def read_b():
print('Running method read_b')
switch = {'A': read_a, 'B': read_b}
case = 'A'
switch.get(case)()
>> 'Running method read_a'
case = 'B'
switch.get(case)()
>> 'Running method read_b'

Stick the functions in a dictionary, and use the dictionary to dispatch to the chosen one:
read = {'A': read_a, 'B': read_b, 'C': read_c}
choice = 'A'
read[choice]()
On that last line, you lookup the function that matches your choice in the dictionary, then you immediately call it.

you may use next construction:
def execute_func(x):
return {
'0':read_A(),
'1':read_B()
}[x]
Set your variables instead '0' and '1' or more and pass your params to execute_func().

You can do it in this way if you have many functions named read_a, read_b...etc, instead of writing huge dictionary.
def read_a():
print('Running method read_a')
def read_b():
print('Running method read_b')
def read_c():
print("running method read_c")
def read_d():
print("running method read_d")
............
............
def read_z():
print("running method read_z")
def _read_var(var):
method = "read_{}".format(var.lower())
try:
eval(method)()
except NameError:
raise NotImplementedError
var = "A"
_read_var(var)# will invoke read_a method

"""
modified from
https://stackoverflow.com/questions/65163600/how-to-call-a-class-method-given-its-name
"""
class MyClass(object):
def __init__(self):
pass
def call_method_by_string(self, method_name):
getattr(self, method_name)() # call local method based on string
def get_year(self):
print("here")
if __name__ == "__main__":
mc = MyClass()
mc.call_method_by_string(method_name="get_year")