Develop Reference

Python import function from another file via argparse

I'm writing a small utility function which takes in input arguments of the location of a Python file, and also a function to call within the Python file
For example src/path/to/file_a.py
def foo():
...
In the utility function, I'm parsing the arguments like so:
python ./util.py --path src/path/to/file_a.py --function foo
and the foo function needs to be used later in the util.py in another library:
def compile():
compiler.compile(
function=foo,
etc
)
What's the best way of importing the foo function via argparse?
Some initial thoughts:
util.py:
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--function", type=str)
return parser.parse_args()
def compile(path, function):
import path
compiler.compile(
function=path.function,
etc
)
if __name__ == "__main__":
args = get_args()
compile(
path=args.path
function=args.function
)
however importing via argparse, and adding it to the function does not seem to work nicely.
There's also the idea of using sys.path.append:
def compile(path, function):
import sys
sys.path.append(path)
but then I'm unsure of how to import the foo function from that.

This problem can be rephrased as "how do I import a python file given a path?" To do that, we can use https://stackoverflow.com/a/67692/5666087. Here is a code example that incorporates the answer to that question with your needs.
import argparse
import importlib.util
import sys
def get_function_object(path_to_pyfile: str, funcname: str):
spec = importlib.util.spec_from_file_location("tmpmodulename", path_to_pyfile)
module = importlib.util.module_from_spec(spec)
sys.modules["tmpmodulename"] = module
spec.loader.exec_module(module)
if not hasattr(module, funcname):
raise AttributeError(f"Cannot find function '{funcname}'in imported module")
# TODO: Consider testing whether this object is a callable.
return getattr(module, funcname)
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--function", type=str)
return parser.parse_args()
if __name__ == "__main__":
args = get_args()
function = get_function_object(args.path, funcname=args.function)
compiler.compile(function=funtion)

How to test if __name__ == "__main__" with passing command line arguments?

Hi I want to test my executable module main.py.
In this module there is function main() that takes two arguments:
# main.py
def main(population_size: int, number_of_iterations: int):
...
At the bottom of this module there is logic that takes command line arguments and executes main function:
# main.py
if __name__ == "__main__":
# create parser and handle arguments
PARSER = argparse.ArgumentParser()
PARSER.add_argument("--populationSize",
type=int,
default=-1,
help="Number of individuals in one iteration")
PARSER.add_argument("--numberOfIterations",
type=int,
default=-1,
help="Number of iterations in one run")
# parse the arguments
ARGS = PARSER.parse_args()
main(ARGS.populationSize, ARGS.numberOfIterations)
I want to test passing command line arguments. My test method that doesn't work:
# test_main.py
#staticmethod
#mock.patch("argparse.ArgumentParser.parse_args")
#mock.patch("main.main")
def test_passing_arguments(mock_main, mock_argparse):
"""Test passing arguments."""
mock_argparse.return_value = argparse.Namespace(
populationSize=4, numberOfIterations=3)
imp.load_source("__main__", "main.py")
mock_main.assert_called_with(4, 3)
The error that I get is that mock_main is not called. I don't know why. To my understanding I mocked main function from main module. Mock of main function is neccessary becouse it's time consuming, and what I want to only test here is that parameters are passed correctly.
From this post I took way of mocking argparse module.

Like all code you want to test, wrap it in a function.
def parse_my_args(argv=None):
PARSER = argparse.ArgumentParser()
PARSER.add_argument("--populationSize",
type=int,
default=-1,
help="Number of individuals in one iteration")
PARSER.add_argument("--numberOfIterations",
type=int,
default=-1,
help="Number of iterations in one run")
# parse the arguments
return PARSER.parse_args(argv)
if __name__ == '__main__':
args = parse_my_args()
main(args.populationSize, args.numberOfIterations)
ArgumentParser.parse_args processes whatever list of strings you pass it. When you pass None, it uses sys.argv[1:] instead.
Now you can test parse_my_args simply by passing whatever list of arguments you want.
# test_main.py
#staticmethod
def test_passing_arguments():
"""Test passing arguments."""
args = parse_my_args(["--populationSize", "4", "--numberOfIterations", "3"])
assert args.populationSize == 4
assert args.numberOfIterations == 3
If you further want to verify that the correct arguments are passed to main, wrap that in a function and use mock as you did above.
def entry_point(argv=None):
args = parse_my_args(argv)
main(args.populationSize, args.numberOfIterations)
if __name__ == '__main__':
entry_point()
and
#staticmethod
#mock.patch("main.main")
def test_passing_arguments(mock_main):
"""Test passing arguments."""
entry_point(["--populationSize", "4", "--numberOfIterations", "3"])
mock_main.assert_called_with(4, 3)

I usually write my command-line code like this. First rename your existing main function to something else, like run() (or whatever):
def run(population_size: int, number_of_iterations: int):
...
Then write a main() function which implements the command-line interface and argument parsing. Have it accept argv as an optional argument which is great for testing:
def main(argv=None):
parser = argparse.ArgumentParser()
...
args = parser.parse_args(argv)
run(args.popuplation_size, args.number_of_iterations)
Then in the module body just put:
if __name__ == '__main__':
sys.exit(main())
Now you have a proper main() function that you can easily test without fussing about the context in which it was called or doing any sort of weird monkeypatching, e.g. like:
main(['--populationSize', '4', '--numberOfIterations', '3'])

Python: most elegant way of accessing __version__ inside main()

I am creating a small Python 3 program that takes command line arguments using argparse. This library has an automatic way of handling version flags. I am trying to figure out what is the most elegant way of accessing version inside my main() function.
#!/usr/bin/env python3
import argparse
def main():
__version__ = '0.1.0'
parser = argparse.ArgumentParser()
parser.add_argument('input',
type=str,
parser.add_argument('-v', '--version',
action='version',
version='%(prog)s ' + __version__
)
args = parser.parse_args()
# main code
print('foo bar')
if __name__ == '__main__':
main()
So what is the most elegant solution here?
1) do what I did above and just live with it.
2) leave __version__ outside main() and pass it as an argument to main() (though I always thought it was bad practise to use arguments in a main() function)
3) leave __version__ and all parser setup outsidemain()and pass the variableargs` to main() as an argument (bad practise again).
4) use global variables
Or am I mistaken that it is bad practise to send arguments to main()? Or alternatively, am I missing another solution altogether?

#!/usr/bin/env python3
import argparse
__version__ = '0.1.0'
def main():
# optionally put this argparse code in its own function
parser = argparse.ArgumentParser()
parser.add_argument('input',
type=str)
parser.add_argument('-v', '--version',
action='version',
version='%(prog)s ' + __version__
)
args = parser.parse_args()
# main code
print(args)
print('foo bar')
if __name__ == '__main__':
main()
testing
1136:~/mypy$ python3 stack57822847.py foobar
Namespace(input='foobar')
foo bar
1136:~/mypy$ python3 stack57822847.py -v
stack57822847.py 0.1.0

Call a module containing command line arguments from another module

Consider a following python files, is there any way to pass cl arguments toother module on import? (calling os.system is not desired)
# A.py
if __name__ == "__main__":
# -- resolve command line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--name', type=str, required=True)
parser.add_argument('--out_file', type=str, required=True)
args = parser.parse_args()
# -- do some operations
# -- save results in `out_file`
#B.py
import A
# how to pass `name` and `out_file` in main?

The correct way is of course to change A.py to have a main function taking arguments as parameters as you were suggested in other answers.
So you really should use:
A.py:
# A.py
def main(args):
# -- resolve command line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--name', type=str, required=True)
parser.add_argument('--out_file', type=str, required=True)
args = parser.parse_args(args)
# -- do some operations
# -- save results in `out_file`
if __name__ == "__main__":
main(sys.argv)
B.py:
import A
import sys
A.main([sys.argv[0], '--name', 'NAME_X', '--out_file', 'FILE.YY'])
That being said, sys.argv is mutable, so it is possible to change it before calling ArgumentParser.parse_args.
So this is possible (even if a bit more hacky):
A.py:
# A.py
def main():
# -- resolve command line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--name', type=str, required=True)
parser.add_argument('--out_file', type=str, required=True)
args = parser.parse_args() # always use sys.argv
# -- do some operations
# -- save results in `out_file`
if __name__ == "__main__":
main()
B.py:
import A
import sys
sys.argv = [sys.argv[0], '--name', 'NAME_X', '--out_file', 'FILE.YY'])
A.main()

# A.py
def main():
# -- resolve command line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--name', type=str, required=True)
parser.add_argument('--out_file', type=str, required=True)
args = parser.parse_args()
# -- do some operations
# -- save results in `out_file`
return out_file
if __name__ == "__main__":
main()
#B.py
import A
def main():
out_file = A.main()
# how to pass `name` and `out_file` in main?

In a.py you need to move the main stuff to a function, for example a def main(). I also added an arguments=None parameter to main() that receives the args from b.py.
# a.py
import argparse
def main(arguments=None):
# -- resolve command line arguments
parser = argparse.ArgumentParser()
parser.add_argument('--name', type=str, required=True)
parser.add_argument('--out_file', type=str, required=True)
args = parser.parse_args(arguments)
# -- do some operations
# -- save results in `out_file`
if __name__ == "__main__":
main()
And then you can pass arguments to that function in b.py like so
#b.py
from a import main
main(['--name', 'some_name', '--out_file', 'file.txt'])

Argument checks best practices

I have a simple code that needs at least 1 argument. Right now my code format looks something like this:
import modules
# argparse stuff
parser = argparse.ArgumentParser()
parser.add_argument(-m)
parser.add_argument(-u)
args = parser.parse_args()
# check the number of arguments
if len(sys.argv) > 3:
sys.exit()
if len(sys.argv) == 1:
sys.exit()
class Program:
def A():
def B():
def C():
if __name__ == '__main__':
try:
Program()
The code works as intended, but I'd like to know how I can rewrite my code to be 'pythonic'. Do I put the argument checks under the 'if name' statement? If so, how? thanks.

I would suggest not looking at sys.argv, especially if you're already using a CLI parsing library.
Argprase has a pile of ways to enforce requirements, but if none of those fit your needs you can looks at your 'args' object.
Personally, I would suggest not running functions, like parse_args(), in the global scope of that file. Instead I would suggest (at minimum) to just wrap what you've got in a function called main, then call 'main()' after 'if __name__ == '__main__'
Argparse examples:
if '-m' and '-u' are mutually exclusive
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-m')
group.add_argument('-u')
args = parser.parse_args() # will output a error message if '-m' or '-u' isn't supplied
If a specific arg is required always
parser = argparse.ArgumentParser()
parser.add_argument('-m', required=True) # must always give '-m'
Or just looking at the 'args' object
parser = argparse.ArgumentParser()
parser.add_argument('-m')
parser.add_argument('-u')
args = parser.parse_args()
if not (args.m or args.u):
sys.exit(1) # should exit non-zero on failures
main wrapping example:
import modules
class Program:
def A():
def B():
def C():
def main():
parser = argparse.ArgumentParser()
parser.add_argument(-m)
parser.add_argument(-u)
args = parser.parse_args()
if not (args.m or args.u):
sys.exit(1)
try:
Program()
except SomeException:
# handle it
pass # b/c I don't know what you need here
if __name__ == '__main__':
main()

Checking the number of arguments after argparse doesn't make much sense. If there's some error, argparse will handle that, so you don't really have to replicate it.
Do put the arguments check after if __name__ check - just in case you want to import the module without executing.
Otherwise, it's just standard code as you'd see in argparse documentation. Nothing really wrong with it.