Develop Reference

Importing Functions From Another Directory Using A Function?

I'm using Python 3.9
I have the following structure:
parent_folder
|
|-->. useful_functions.py .
|-->. script_folder_1 .
|
|-->. script_1.py
|-->. function_import.py
|-->. script_folder_2 .
|
|-->. script_2.py
|-->. function_import.py
script_1.py and script_2.py are very similar (but not identical), and so both want to use the functions stored in useful_functions.py
I have worked out that I can import the functions from useful_functions.py by putting the following at the top of script_1.py and script_2.py:
import sys
import os
local_dir = os.getcwd()
dir_up = os.path.dirname(local_dir)
sys.path.append(dir_up)
import useful_functions as uf
This works, meaning it allows script_1.py to call functions from useful_function.py as uf.function_name(arguments)
Since this text block is identical in script_1.py and script_2.py, I wanted to pull it out and make it a function in a different file function_import.py, since that way if I need to modify it I can modify the standalone file in script_folder_1 and copy it to script_folder_2.
However, when I create function_import.py, function_import.py is able to "locally" access the functions from useful_functions.py, but script_1.py is not able to access the functions from useful_functions.py.
Details:
def function_import():
import sys
import os
import re
local_dir = os.getcwd()
dir_up = os.path.dirname(local_dir)
sys.path.append(dir_up)
import useful_functions as uf
print(uf.test_function())
script_1.py:
import function_import
function_import.function_import()
uf.test_function()
When I run script_1.py in the terminal, uf.test_function() returns 3, the placeholder value, inside of function_import(), but then when script_1.py calls uf.test_function(), I get:
NameError: name 'uf' is not defined
Is there a way to allow a function inside a script to import modules for the script to use? Or is there some other way entirely that I should be doing this?
I read these posts and they didn't seem to have a good solution to my problem:
https://stackoverflow.com/search?q=import+from+parent+directory+python&s=5452f193-c78d-4966-b69a-896fb6a5a5f8
Import Script from a Parent Directory

This doesn't define a function inside function_import.py but it does the job
function_import.py:
import sys
import os
import re
local_dir = os.getcwd()
dir_up = os.path.dirname(local_dir)
sys.path.append(dir_up)
import useful_functions as uf
useful_functions.py:
def test_function():
print('In the test function')
script_1.py:
from function_import import uf
uf.test_function()
Outputs In the test function

It's not the perfect solution but you can wrap your function into another one like so :
function_import.py:
def function_import():
import sys
import os
import re
local_dir = os.getcwd()
dir_up = os.path.dirname(local_dir)
sys.path.append(dir_up)
import useful_functions as uf
#print(uf.test_function())
def uf_test_function(*args, **kwargs):
uf.test_function(*args, **kwargs)
script_1.py:
import function_import
function_import.function_import()
function_import.uf_test_function()

Conserving a state in a large python program without passing variable as parameter

I have a large project where I load in some variables from a config.ini, such as the target_directory of the data that is generated. This target_directory is used all over the project but it is not always set in the same place. It can be set by config.ini, but it can be overridden by command line parameters or also by unit tests.
What is the best way to preserve that state in a global way, so I don't have to pass it into every function of my program.
The program consists of many different modules and packages and I was wondering if there's a best practice how this could be handled.

You could set state into a module-level variable using a dictionary, or any other type of container. These values are not static, and can be modified/mutated by any code that has access to them (unless you use an immutable container, like a tuple or namedtuple). You can import the whole conf.py module with import conf, or just pull in the configuration container with from conf import CONFIG
# this is conf.py
import argparse
import configparser
CONFIG = {}
def build_config():
# ...compose your config from cli, ini file, etc
CONFIG['option_one'] = 'hello world'
return CONFIG
...
# this is a.py
import conf
def do_something_with_config_value():
return conf.CONFIG['option_one']
...
# this is b.py
import a
import conf
if __name__ == '__main__':
conf.build_config()
print(a.do_something_with_config_value())

This seems to work:
main.py
import c2
from c3 import func
c2.v1 = 1
c2.v2 = 1
c2.v3 = 1
def edit():
c2.v1 = 11
if __name__ == "__main__":
edit()
func()
c2.py (the configuration container)
v1 = 2
v2 = 2
c3.py
import c2
def func():
print (c2.v1)
print (c2.v2)
print (c2.v3)
output:
11
1
1

How to get variables from another file in python [duplicate]

This question already has answers here:
How can I import a module dynamically given its name as string?
(10 answers)
Closed 8 years ago.
For example I have a variable config in config.py. I want to use the variable config in main.py. And the config.py must be pass to main.py through command line. like follows:
python ./main.py ./config.py
I know in lua I can use dofile function. How can I do this in Python
Fixed by Dynamic module import in Python

disclaimer: i cant comment
can you just import the variable in main.py ?
from .config import config
#do stuff
the only other way i could imagine this would work, if you would pipe the line to the main.py file and parse it by hand or something:
cat config.py | grep 'config = ' | sed -e 's,config = ,,g' | python main.py
thou this will only work if config is only used once in the file and the value it represents is behind the = and you know if it is a string or a int etc.

# main.py
import sys
import os
if __name__ == "__main__":
var_name = "config"
arg = sys.argv[1]
module_path = os.path.realpath(os.path.dirname(arg))
if module_path not in sys.path:
sys.path.insert(0, module_path)
module_name = os.path.basename(arg).replace(".py", "")
module = __import__(module_name)
config_var = getattr(module, var_name)
# use config_var

Running a function in Python if a user chooses it (EasyGui)

I'm using EasyGui to allow a user to select multiple options. Each option is a function which they can run if they select it. I'm trying to use dictionaries as suggested in other threads but I'm having trouble implementing it (Module object is not callable error). Is there something I'm missing?
from easygui import *
import emdtest1
import emdtest2
import emdtest3
EMDTestsDict = {"emdtest1":emdtest1,
"emdtest2":emdtest2,
"emdtest3":emdtest3}
def main():
test_list = UserSelect()
for i in range(len(test_list)):
if test_list[i] in EMDTestsDict.keys():
EMDTestsDict[test_list[i]]()
def UserSelect():
message = "Which EMD tests would you like to run?"
title = "EMD Test Selector"
tests = ["emdtest1",
"emdtest2",
"emdtest3"]
selected_master = multchoicebox(message, title, tests)
return selected_master
if __name__ == '__main__':
main()

You're putting modules into the dict, when you want to put functions in it. What you're doing is the equivalent of saying
import os
os()
Which, of course, makes no sense. If emdtest1, emdtest2, and emdtest3 are .py files with functions in them, you want:
from emdtest1 import function_name
Where function_name is the name of your function.

You need to import the functions rather than the module ... for example , if you have a file called emdtest1 with a defined function emdtest1, you'd use:
from emdtest1 import emdtest1

Import custom modules on IPython.parallel engines with sync_imports()

I've been playing around with IPython.parallel and I wanted to use some custom modules of my own, but haven't been able to do it as explained on the cookbook using dview.sync_imports(). The only thing that has worked for me was something like
def my_parallel_func(args):
import sys
sys.path.append('/path/to/my/module')
import my_module
#and all the rest
and then in the main just to
if __name__=='__main__':
#set up dview...
dview.map( my_parallel_func, my_args )
The correct way to do this would in my opinion be something like
with dview.sync_imports():
import sys
sys.path.append('/path/to/my/module')
import my_module
but this throws an error saying there is no module named my_module.
So, what is the right way of doing it using dview.sync_imports()??

The problem is that you're changing the PYTHONPATH just in the local process running the Client, and not in the remote processes running in the ipcluster.
You can observe this behaviour if you run the next piece of code:
from IPython.parallel import Client
rc = Client()
dview = rc[:]
with dview.sync_imports():
import sys
sys.path[:] = ['something']
def parallel(x):
import sys
return sys.path
print 'Local: ', sys.path
print 'Remote: ', dview.map_sync(parallel, range(1))
Basically all the modules that you want to use with sync_imports must already be in the PYTHONPATH.
If it's not in the PYTHONPATH then you must add it to the path in the function that you execute remotely, and then import the module in the function.