I'm writing a small utility function which takes in input arguments of the location of a Python file, and also a function to call within the Python file
For example src/path/to/file_a.py
def foo():
...
In the utility function, I'm parsing the arguments like so:
python ./util.py --path src/path/to/file_a.py --function foo
and the foo function needs to be used later in the util.py in another library:
def compile():
compiler.compile(
function=foo,
etc
)
What's the best way of importing the foo function via argparse?
Some initial thoughts:
util.py:
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--function", type=str)
return parser.parse_args()
def compile(path, function):
import path
compiler.compile(
function=path.function,
etc
)
if __name__ == "__main__":
args = get_args()
compile(
path=args.path
function=args.function
)
however importing via argparse, and adding it to the function does not seem to work nicely.
There's also the idea of using sys.path.append:
def compile(path, function):
import sys
sys.path.append(path)
but then I'm unsure of how to import the foo function from that.
This problem can be rephrased as "how do I import a python file given a path?" To do that, we can use https://stackoverflow.com/a/67692/5666087. Here is a code example that incorporates the answer to that question with your needs.
import argparse
import importlib.util
import sys
def get_function_object(path_to_pyfile: str, funcname: str):
spec = importlib.util.spec_from_file_location("tmpmodulename", path_to_pyfile)
module = importlib.util.module_from_spec(spec)
sys.modules["tmpmodulename"] = module
spec.loader.exec_module(module)
if not hasattr(module, funcname):
raise AttributeError(f"Cannot find function '{funcname}'in imported module")
# TODO: Consider testing whether this object is a callable.
return getattr(module, funcname)
def get_args():
parser = argparse.ArgumentParser()
parser.add_argument("--path", type=str)
parser.add_argument("--function", type=str)
return parser.parse_args()
if __name__ == "__main__":
args = get_args()
function = get_function_object(args.path, funcname=args.function)
compiler.compile(function=funtion)
I have a small python script and I want to add a handful of tests.
For a larger system I would have modules and separate tests from the system under test, but for this tiny thing I want to keep it all in one .py file. Then I can run:
> foo.py --test
to run the tests and
> foo.py
to run my script normally.
I got this far:
import unittest
import sys
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--test", action="store_true")
args = parser.parse_args()
if (args.test):
return unittest.main()
class TestBasicFunction(unittest.TestCase):
def test(self):
self.assertTrue(True)
if __name__ == '__main__':
sys.exit(main())
But it fails because unittest.main() tries to parse the arguments to my script.
I am not picky about unit test framework. py test, nose, whatever is fine.
How can I do this?
Note that there are reasons you might want to consider separating unittests from the code it is testing.
However, here is how you could do it:
import unittest
import sys
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument("--test", action="store_true")
args = parser.parse_args()
print(args)
if args.test:
sys.argv.remove('--test')
return unittest.main(argv=sys.argv)
class TestBasicFunction(unittest.TestCase):
def test(self):
self.assertTrue(True)
if __name__ == '__main__':
sys.exit(main())
I am trying to work around a problem I have encountered in a piece of code I need to build on. I have a python module that I need to be able to import and pass arguments that will then be parsed by the main module. What I have been given looks like this:
#main.py
if __name__ == '__main__'
sys.argv[] #pass arguments if given and whatnot
Do stuff...
What I need is to add a main() function that can take argument(s) and parse them and then pass them on like so:
#main.py with def main()
def main(args):
#parse args
return args
if __name__ == '__main__':
sys.argv[] #pass arguments if given and whatnot
main(sys.argv)
Do stuff...
To sum up: I need to import main.py and pass in arguments that are parsed by the main() function and then give the returned information to the if __name_ == '__main_' part.
EDIT
To clarify what I am doing
#hello_main.py
import main.py
print(main.main("Hello, main"))
ALSO I want to still be able to call main.py from shell via
$: python main.py "Hello, main"
Thus preserving the name == main
Is what I am asking even possible? I have been spending the better part of today researching this issue because I would like to, if at all possible, preserve the main.py module that I have been given.
Thanks,
dmg
Within a module file you can write if __name__ == "__main__" to get specific behaviour when calling that file directly, e.g. via shell:
#mymodule.py
import sys
def func(args):
return 2*args
#This only happens when mymodule.py is called directly:
if __name__ == "__main__":
double_args = func(sys.argv)
print("In mymodule:",double_args)
One can then still use the function when importing to another file:
#test.py
import mymodule
print("In test:",mymodule.func("test "))
Thus, calling python test.py will result in "In test: test test ", while calling python mymodule.py hello will result in "In mymodule: hello hello ".
I have a simple code that needs at least 1 argument. Right now my code format looks something like this:
import modules
# argparse stuff
parser = argparse.ArgumentParser()
parser.add_argument(-m)
parser.add_argument(-u)
args = parser.parse_args()
# check the number of arguments
if len(sys.argv) > 3:
sys.exit()
if len(sys.argv) == 1:
sys.exit()
class Program:
def A():
def B():
def C():
if __name__ == '__main__':
try:
Program()
The code works as intended, but I'd like to know how I can rewrite my code to be 'pythonic'. Do I put the argument checks under the 'if name' statement? If so, how? thanks.
I would suggest not looking at sys.argv, especially if you're already using a CLI parsing library.
Argprase has a pile of ways to enforce requirements, but if none of those fit your needs you can looks at your 'args' object.
Personally, I would suggest not running functions, like parse_args(), in the global scope of that file. Instead I would suggest (at minimum) to just wrap what you've got in a function called main, then call 'main()' after 'if __name__ == '__main__'
Argparse examples:
if '-m' and '-u' are mutually exclusive
parser = argparse.ArgumentParser()
group = parser.add_mutually_exclusive_group(required=True)
group.add_argument('-m')
group.add_argument('-u')
args = parser.parse_args() # will output a error message if '-m' or '-u' isn't supplied
If a specific arg is required always
parser = argparse.ArgumentParser()
parser.add_argument('-m', required=True) # must always give '-m'
Or just looking at the 'args' object
parser = argparse.ArgumentParser()
parser.add_argument('-m')
parser.add_argument('-u')
args = parser.parse_args()
if not (args.m or args.u):
sys.exit(1) # should exit non-zero on failures
main wrapping example:
import modules
class Program:
def A():
def B():
def C():
def main():
parser = argparse.ArgumentParser()
parser.add_argument(-m)
parser.add_argument(-u)
args = parser.parse_args()
if not (args.m or args.u):
sys.exit(1)
try:
Program()
except SomeException:
# handle it
pass # b/c I don't know what you need here
if __name__ == '__main__':
main()
Checking the number of arguments after argparse doesn't make much sense. If there's some error, argparse will handle that, so you don't really have to replicate it.
Do put the arguments check after if __name__ check - just in case you want to import the module without executing.
Otherwise, it's just standard code as you'd see in argparse documentation. Nothing really wrong with it.
In Python I have a module myModule.py where I define a few functions and a main(), which takes a few command line arguments.
I usually call this main() from a bash script. Now, I would like to put everything into a small package, so I thought that maybe I could turn my simple bash script into a Python script and put it in the package.
So, how do I actually call the main() function of myModule.py from the main() function of MyFormerBashScript.py? Can I even do that? How do I pass any arguments to it?
It's just a function. Import it and call it:
import myModule
myModule.main()
If you need to parse arguments, you have two options:
Parse them in main(), but pass in sys.argv as a parameter (all code below in the same module myModule):
def main(args):
# parse arguments using optparse or argparse or what have you
if __name__ == '__main__':
import sys
main(sys.argv[1:])
Now you can import and call myModule.main(['arg1', 'arg2', 'arg3']) from other another module.
Have main() accept parameters that are already parsed (again all code in the myModule module):
def main(foo, bar, baz='spam'):
# run with already parsed arguments
if __name__ == '__main__':
import sys
# parse sys.argv[1:] using optparse or argparse or what have you
main(foovalue, barvalue, **dictofoptions)
and import and call myModule.main(foovalue, barvalue, baz='ham') elsewhere and passing in python arguments as needed.
The trick here is to detect when your module is being used as a script; when you run a python file as the main script (python filename.py) no import statement is being used, so python calls that module "__main__". But if that same filename.py code is treated as a module (import filename), then python uses that as the module name instead. In both cases the variable __name__ is set, and testing against that tells you how your code was run.
Martijen's answer makes sense, but it was missing something crucial that may seem obvious to others but was hard for me to figure out.
In the version where you use argparse, you need to have this line in the main body.
args = parser.parse_args(args)
Normally when you are using argparse just in a script you just write
args = parser.parse_args()
and parse_args find the arguments from the command line. But in this case the main function does not have access to the command line arguments, so you have to tell argparse what the arguments are.
Here is an example
import argparse
import sys
def x(x_center, y_center):
print "X center:", x_center
print "Y center:", y_center
def main(args):
parser = argparse.ArgumentParser(description="Do something.")
parser.add_argument("-x", "--xcenter", type=float, default= 2, required=False)
parser.add_argument("-y", "--ycenter", type=float, default= 4, required=False)
args = parser.parse_args(args)
x(args.xcenter, args.ycenter)
if __name__ == '__main__':
main(sys.argv[1:])
Assuming you named this mytest.py
To run it you can either do any of these from the command line
python ./mytest.py -x 8
python ./mytest.py -x 8 -y 2
python ./mytest.py
which returns respectively
X center: 8.0
Y center: 4
or
X center: 8.0
Y center: 2.0
or
X center: 2
Y center: 4
Or if you want to run from another python script you can do
import mytest
mytest.main(["-x","7","-y","6"])
which returns
X center: 7.0
Y center: 6.0
It depends. If the main code is protected by an if as in:
if __name__ == '__main__':
...main code...
then no, you can't make Python execute that because you can't influence the automatic variable __name__.
But when all the code is in a function, then might be able to. Try
import myModule
myModule.main()
This works even when the module protects itself with a __all__.
from myModule import * might not make main visible to you, so you really need to import the module itself.
I had the same need using argparse too.
The thing is parse_args function of an argparse.ArgumentParser object instance implicitly takes its arguments by default from sys.args. The work around, following Martijn line, consists of making that explicit, so you can change the arguments you pass to parse_args as desire.
def main(args):
# some stuff
parser = argparse.ArgumentParser()
# some other stuff
parsed_args = parser.parse_args(args)
# more stuff with the args
if __name__ == '__main__':
import sys
main(sys.argv[1:])
The key point is passing args to parse_args function.
Later, to use the main, you just do as Martijn tell.
The answer I was searching for was answered here: How to use python argparse with args other than sys.argv?
If main.py and parse_args() is written in this way, then the parsing can be done nicely
# main.py
import argparse
def parse_args():
parser = argparse.ArgumentParser(description="")
parser.add_argument('--input', default='my_input.txt')
return parser
def main(args):
print(args.input)
if __name__ == "__main__":
parser = parse_args()
args = parser.parse_args()
main(args)
Then you can call main() and parse arguments with parser.parse_args(['--input', 'foobar.txt']) to it in another python script:
# temp.py
from main import main, parse_args
parser = parse_args()
args = parser.parse_args([]) # note the square bracket
# to overwrite default, use parser.parse_args(['--input', 'foobar.txt'])
print(args) # Namespace(input='my_input.txt')
main(args)
Assuming you are trying to pass the command line arguments as well.
import sys
import myModule
def main():
# this will just pass all of the system arguments as is
myModule.main(*sys.argv)
# all the argv but the script name
myModule.main(*sys.argv[1:])
I hit this problem and I couldn't call a files Main() method because it was decorated with these click options, eg:
# #click.command()
# #click.option('--username', '-u', help="Username to use for authentication.")
When I removed these decorations/attributes I could call the Main() method successfully from another file.
from PyFileNameInSameDirectory import main as task
task()