I have multiple Python scripts which use docopt.
My issue is that the available options for the two scripts differ slightly - one option is present in one script, but not the other.
I've included a minimum working example below.
If I run:
python main.py --num=7 --name=John
the script does not run, as --name=John is also passed to module1.py, where it is not a valid option.
With my actual script, I have several imports after docopt parses the arguments, and so I cannot simply move the docopt call to the bottom of the script (if __name__ == '__main__':). If I do this, the imports in the imported script never get called, and I get undefined name errors.
I have found a workaround, but I don't think it is good practice at all.
What I am doing is adding:
if __name__ == '__main__':
arguments = docopt.docopt(__doc__, version=0.1)
just after the import docopt.
However, I believe that having two of these statements in a script is bad practice. I cannot think of any other workarounds at this time though.
Can someone suggest a better solution? Thanks in advance.
main.py
"""
main.py
Usage:
main.py [--num=<num>] [--name=<name>] [--lib=<lib-dir>]
main.py -h | --help
main.py --version
Options:
--num=<num> A number
--name=<name> A name
--lib=<lib-dir> Path to the directory containing lib
--version
"""
import docopt
arguments = docopt.docopt(__doc__, version=0.1)
library_path = os.path.abspath(arguments['--lib'])
sys.path.insert(1, library_path)
NUM = arguments['--num']
from other_file import x, y
from module1 import function
def main():
print 'In main()'
function()
print NUM
if __name__ == '__main__':
print '{} being executed directly'.format(__name__)
main()
module1.py:
"""
module1.py
Usage:
module1.py [--num=<num>] [--lib=<lib-dir>]
module1.py -h | --help
module1.py --version
Options:
--num=<num> A number
--lib=<lib-dir> Path to the directory containing lib
--version
"""
import docopt
arguments = docopt.docopt(__doc__, version=0.1)
library_path = os.path.abspath(arguments['--lib'])
sys.path.insert(1, library_path)
NUM = arguments['--num']
from other_file import z
def main():
print 'In main()'
print NUM
def function():
print 'In function in {}'.format(__name__)
# print NUM
if __name__ == '__main__':
print '{} being executed directly'.format(__name__)
main()
EDIT:
I forgot to mention that the other_file module has many different versions. Because of this, one of the docopt options is the path to the file. This is then added to sys.path as follows:
library_path = os.path.abspath(arguments['--lib'])
sys.path.insert(1, library_path)
For this reason, the import of docopt in the global scope is needed to add the path to the other_file module to my system path.
The global variable (NUM below, DEBUG in my actual file) I can live without.
The clean solution is to refactor your code so it doesn't rely on a global, neither in main.py nor module1.py:
"""
main.py
Usage:
main.py [--num=<num>] [--name=<name>]
main.py -h | --help
main.py --version
Options:
--num=<num> A number
--name=<name> A name
--version
"""
from other_file import x, y
from module1 import function
def main(num):
print 'In main()'
function(num)
print num
if __name__ == '__main__':
import docopt
arguments = docopt.docopt(__doc__, version=0.1)
NUM = arguments['--num']
print '{} being executed directly'.format(__name__)
main(NUM)
And:
"""
module1.py
Usage:
module1.py [--num=<num>]
module1.py -h | --help
module1.py --version
Options:
--num=<num> A number
--version
"""
from other_file import z
def main(num):
print 'In main()'
print num
def function(num):
print 'In function in {}'.format(__name__)
print num
if __name__ == '__main__':
import docopt
arguments = docopt.docopt(__doc__, version=0.1)
NUM = arguments['--num']
print '{} being executed directly'.format(__name__)
main(NUM)
Related
I'm running unit tests for my program.py file. I found an error: python3 -m unittest: error: unrecognized arguments:.
I believe the error comes from the argparse library I'm using where the target code is expecting some arguments.
Target file: program.py:
import argparse
parse = argparse.ArgumentParser(description="Command line program.")
parse.add_argument("--key", type=str,
help="Enter key")
parse.add_argument("--output", type=str,
help="Path to place results.")
args = parse.parse_args()
def program():
# Use args here
def writefile():
# Uses args and write to file
if __name__ == "__main__":
program()
Test file: program_test.py:
import unittest
import program
class TestProgram(unittest.TestCase):
def setUp(self):
self.argv_list = ["--key", "somefile.txt",
"--output", "myresultfile.txt"]
def test_program_stuff(self):
# See "Things I've tried"
program.writefile(...)
Command:
me#mylinux:myprogram$ env/bin/python3 -m unittest -v program_test.py
usage: python3 -m unittest [-h] [--key KEY] [--output OUTPUT]
python3 -m unittest: error: unrecognized arguments: -v program_test.py
Things I've tried:
Mock the argparse.Namespace with argparse.Namespace(key="key.txt", output="result.txt")
Manipulate sys.args in test_program_stuff by sys.args.append(self.argv_list)
I've looked at solutions to unit testing argparse but none have helped so I'm thinking it may not be the same issue:
Python argparse "unrecognized arguments" error
argparse fails when called from unittest test
argparse and unittest python
Note: I do realize that this is a duplicate of Pytest unrecognized arguments when importing file with argparse, however, that question was unfortunately not answered. It is also a duplicate of How to call function with argparse arguments in unittests?, however, he doesn't want to provide arguments and instead wants to call another function defined in the module.
Essentially, the problem can be reduced to the following:
# main.py
import argparse
parse = argparse.ArgumentParser()
parse.add_argument("--foo", action="store_true")
args = parse.parse_args()
and
# tests.py
import main
If we run that with python -m unittest ./tests.py, we receive the following output:
usage: python -m unittest [-h] [--foo FOO]
python -m unittest: error: unrecognized arguments: ./tests.py
The problem is that if you import something, all the top level code will run during the import. Usually, this isn't a problem because in a library you only really define functions to be used by other programs, however, in your case the parse.parse_args() runs.
This can be resolved by guarding this logic similar to what you already did:
import argparse
def main():
parse = argparse.ArgumentParser()
parse.add_argument("--foo")
args = parse.parse_args()
if __name__ == "__main__":
main()
Here, __name__ will contain the name of the module which would be "main" if it is imported or "__main__" if it is run directly. Therefore, the main() function will not be called during the unit test. Read more about this here.
However, in your specific case, it seems that you want to test the functionality of the main() function (or rather the code that you have in your main module.)
There are generally two ways to achieve this:
You can simply accept the arguments in main():
import argparse
import sys
def main(argv):
parse = argparse.ArgumentParser()
parse.add_argument("--foo", action="store_true")
args = parse.parse_args(argv[1:])
if __name__ == "__main__":
main(sys.argv)
Then you are able to provide these arguments in the unit test as well:
import unittest
import main
class Tests(unittest.TestCase):
def test_whatever(self):
main.main(["main.py", "--foo"])
You can use a mock-framework to change the value of sys.argv. In this situation would seem a bit over-engineered. However, if you are interested, that is answered here.
The parse.parse_args is outside of a method so will get run when the file is imported by the unittest file. This is why it is complaining that it doesn't have the expected arguments.
A solution is to move the parse_args into a function. e.g:
import argparse
import sys
def parse_args(sys_args):
parse = argparse.ArgumentParser(description="Command line program.")
parse.add_argument("--key", type=str,
help="Enter key")
parse.add_argument("--output", type=str,
help="Path to place results.")
return parse.parse_args(sys_args)
def program(key, output):
# Use args here
use_key(key)
writefile(output)
def use_key(key):
print(f"I have the key: {key}")
def writefile(filename):
# Uses args and write to file
print(f"I will write to file: {filename}")
if __name__ == "__main__":
parsed_args = parse_args(sys.argv[1:])
program(parsed_args.key, parsed_args.output)
This then allows things to test individually. e.g:
import io
import unittest
from unittest import mock
import program
class TestProgram(unittest.TestCase):
def setUp(self):
self.argv_list = ["--key", "somefile.txt",
"--output", "myresultfile.txt"]
def test_program_parse(self):
# See "Things I've tried"
args = program.parse_args(self.argv_list)
self.assertEqual("somefile.txt", args.key)
self.assertEqual("myresultfile.txt", args.output)
def test_program(self):
with mock.patch('sys.stdout', new=io.StringIO()) as fake_out:
program.use_key('val1')
self.assertEqual('I have the key: val1\n', fake_out.getvalue())
def test_writefile(self):
with mock.patch('sys.stdout', new=io.StringIO()) as fake_out:
program.writefile('junk.txt')
self.assertEqual('I will write to file: junk.txt\n', fake_out.getvalue())
if __name__ == '__main__':
unittest.main(verbosity=2)
I'm trying to execute file1.py from file2 .py by using exec function.
exec(open('file1.py').read())
Now I want to pass parameter target='target.yaml' to file1.py from exec function. How I can do that ?
Please help
You can use subprocess module:
file1:
import subprocess
print("Running file1.py")
subprocess.run("python file2.py target.yaml", shell=True)
exit(0) # file2 will be opened in a new window
file2:
import sys
yaml_file = sys.argv[1]
print("Running file2.py : yaml target: " + yaml_file)
output:
Running file1.py
Running file2.py : yaml target: target.yaml
While the other answer is a very good solution, if you are writing both files yourself you should consider importing one into the other and calling the function from the other file directly.
file1.py:
import otherfile
argument = "Hello world!"
otherfile.fancy_function(argument)
otherfile.py:
def fancy_function(arg):
print(arg)
if __name__ == "__main__":
# If the file is called directly like `python otherfile.py` this will be executed
fancy_function("I have my own world too!")
Hi I am writing python for the first time:
I have a existing getprop.py script that loads the property file and prints the value of a given property:
import sys
import util
if len(sys.argv) < 3:
print "Error! Usage is: getprop.py [propfile] [propname]"
sys.exit(1)
props = util.loadprops(sys.argv[1])
if sys.argv[2] in props:
print props(sys.argv[2]);
Now I need to get the value of a property in another py script, so I modified the above script such that I do not disturb its functionality and I can use it in another script:
import sys
import util
def getpropvalue(propfile, propname):
props = util.loadprops(propfile)
if propname in props:
return props[propname]
if len(sys.argv) < 3:
print "Error! Usage is: getprop.py [propfile] [propname]"
sys.exit(1)
else:
print getpropvalue(sys.argv[1], sys.argv[2]);
and then in other script I import getprop and call the method like getprop.getpropvalue(FILE_NAME, PROP_NAME)and it prints the value of the property on the screen.
why does it prints the value? Is there any better way to solve this problem?
There is a way to run the script only if it was called directly. Add those lines to the end of your getprop code:
if __name__ == "__main__":
main()
This way the main function is only going to be called if you run the script directly (not importing). Is that what you're looking for?
Some explanation: every running script has a __name__ variable that will be set to "__main__" if you run the script from an IDE or console like python script.py
Change your getprop.py to this:
import sys
import util
def getpropvalue(propfile, propname):
props = util.loadprops(propfile)
if propname in props:
return props[propname]
if __name__ == '__main__':
if len(sys.argv) < 3:
print "Error! Usage is: getprop.py [propfile] [propname]"
sys.exit(1)
else:
print getpropvalue(sys.argv[1], sys.argv[2]);
This will prevent the code from being executed when it is imported.
As an example of my problem consider the following situation:
A user launches a program from command line with a syntax similar to this
python prog.py "hi there" hi=Hello there=World!
The output of the program is "Hello World!".
My question refers to the part of parsing the arguments. How can I save the information contained in the "hi=Hello there=World!" part in order to use it? From there I should be able to do something with it.
I don't have the slightest idea of what the parameters are going to be so the solution needs to be as generic as possible.
you have to do something like this... save this as "argtest.py":
import sys
def main(x):
print(x)
if __name__ == "__main__":
print sys.argv #notice this is whatever you put in the arguments
print sys.argv[0] # this is the name of the file you used
main(sys.argv[1])
this will pass the first arg into main()
so if you are running from command line
python argtest.py hello
your output will be
hello
in your main() you would of course define whatever you want to do with the argument
personally i do my parsing in under the if __name__=="__main__": line usually something like arguments = [x.split('=') for x in sys.argv[1:]] #if you want to seperate arguments like "pie=apple fart=stinky"
sys.argv is a list of whatever you put after python argtest.py space seperated
for example if you do python argtest.py apple pie comma poop
sys.argv[1] == 'apple'
sys.argv[2] == 'pie'
sys.argv[3] == 'comma'
sys.argv[4] == 'poop'
Using docopt
install docopt:
$ pip install docopt
Write the code of prog.py:
"""
Usage:
prog.py <greeting> <name>
"""
if __name__ == "__main__":
from docopt import docopt
args = docopt(__doc__)
print args
print "------"
name = args["<name>"]
greeting = args["<greeting>"]
print "{greeting} {name}!".format(greeting=greeting, name=name)
Use it:
First see usage instructions:
$ python prog.py
Usage:
prog.py <greeting> <name>
Then use it:
$ python prog.py Ola Mundi
{'<greeting>': 'Ola',
'<name>': 'Mundi'}
------
Ola Mundi!
I ended up using simple string processing:
import sys
entry = {}
for i in sys.argv[1:]:
pos = i.find("=")
entry[i[:pos]] = i[pos+1:]
I'm using Python 2.7, and I have the following simple script, which expects one command line argument:
#!/usr/bin/env python
import sys
if (len(sys.argv) == 2):
print "Thanks for passing ", sys.argv[1]
else:
print "Oops."
I can do something like this from the command line:
My-Box:~/$ ./useArg.py asdfkjlasdjfdsa
Thanks for passing asdfkjlasdjfdsa
or this:
My-Box:~/$ ./useArg.py
Oops.
I would like to do something similar from the interactive editor:
>>> import useArg asdfasdf
File "<stdin>", line 1
import useArg asdfasdf
^
SyntaxError: invalid syntax
but I don't know how. How can I pass a parameters to import/reload in the interactive editor ?
You can't. Wrap your code inside the function
#!/usr/bin/env python
import sys
def main(args):
if (len(args) == 2):
print "Thanks for passing ", args[1]
else:
print "Oops."
if __name__ == '__main__':
main(sys.argv)
If you execute your script from command line you can do it like before, if you want to use it from interpreter:
import useArg
useArg.main(['foo', 'bar'])
In this case you have to use some dummy value at the first position of the list, so most of the time much better solution is to use argparse library. You can also check the number of command line arguments before calling the main function:
import sys
def main(arg):
print(arg)
if __name__ == '__main__':
if len(sys.argv) == 2:
main(sys.argv[1])
else:
main('Oops')
You can find good explanation what is going on when you execute if __name__ == '__main__': here: What does if __name__ == "__main__": do?