I have a .py file following the normal code structure
def main( args ):
.......
.......
if __name__ == "__main__":
parser = argparse.ArgumentParser(description = “ forecasting example”)
parser.add_argument("--train-window", default=2160, type=int)
parser.add_argument("--test-window", default=336, type=int)
parser.add_argument("--stride", default=168, type=int)
parser.add_argument("-n", "--num-steps", default=501, type=int)
parser.add_argument("-lr", "--learning-rate", default=0.05, type=float)
parser.add_argument("--dct", action="store_true")
parser.add_argument("--num-samples", default=100, type=int)
parser.add_argument("--log-every", default=50, type=int)
parser.add_argument("--seed", default=1234567890, type=int)
args = parser.parse_args()
main(args)
I was trying to run this program in Jupyter notebook, but it will get errors such as
usage: ipykernel_launcher.py [-h] [--train-window TRAIN_WINDOW]
[--test-window TEST_WINDOW] [--stride STRIDE]
[-n NUM_STEPS] [-lr LEARNING_RATE] [--dct]
[--num-samples NUM_SAMPLES]
[--log-every LOG_EVERY] [--seed SEED]
ipykernel_launcher.py: error: unrecognized arguments: -f C:\Users\AppData\Roaming\jupyter\runtime\kernel-4c744f03-3278-4aaf-af5e-50c96e9e41cd.json
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
my question is that, what are the right approaches or the modifications I need to make if I want to run a python program, which setup input parameters using argparse type of mechanism, in Jupyter Notebook?
Your code should be indented differently so you can import it into your notebook, or into another Python script. The whole point of the if __name__ == "__main__": block is that it gets executed immediately when Python parses the file; the condition is true only when you run the file directly, not when you import it. But the block needs to be indented differently, so that it's not inside any def or class or other block structure.
The way to use this from a notebook, then, is to call main (or whichever other functions from the imported code you want to run) with your desired parameters.
In this case, main has been designed to expect an Argparse object as its argument, which is quite unfortunate. A better design would simply do the argument parsing inside main, and expose a different function or set of functions for reuse as a library.
Assuming your main function's internals look something like
def main(args):
print(
real_main(args.train_window, args.test_window,
stride=args.stride, num_steps=args.num_steps,
learning_rate=args.learning_rate,
dct=args.dct, num_samples=args.num_samples,
log_every=args.log_every, seed=args.seed))
and supposing you wanted to run the equivalent of
python thatfile.py -n 23 -lr 0.7--dct --num-samples 2300
the equivalent code in your notebook would look like
from thatfile import real_main as that_main
print(that_main(2160, 336, num_steps=23,
learning_rate=0.7, dct=True,
num_samples=2300))
where the first two values are simply copied from the argparse defaults, and I obviously had to speculate a great deal about which parameters are required and which are optional keyword parameters, and whether they are named identically to the argparse field names.
Related
I have a python script say A which has some arguments specified using argparse in main.
.
.
def main(args):
# use the arguments
.
.
if __name__ == '__main__':
parser = argparse.ArgumentParser(..)
parser.add_argument(
'-c',
'--classpath',
type=str,
help='directory with list of classes',
required=True)
# some more arguments
args = parser.parse_args()
main(args)
I've written another python script say B, which uses flask to run a web-application on localhost.
I'm trying to import the script A in B as:
from <folder> import A
How do I give in the arguments that are required in A for running script B?
I want to run A inside script B by passing parameters via the main flask python script (viz., the script B).
I want to use all the functionality of A, but I'd rather not change the structure of A or copy-paste the same exact code inside B.
I've been trying something like:
#app.route(...)
def upload_file():
A.main(classpath = 'uploads/')
but that doesn't seem to work. I'm taking inspiration from this SO answer, but I guess I'm missing something.
Does anybody have some idea on how to do this efficiently?
The answer which I've linked helped me to made it work for my code. Quite simply, efficient use of kwargs can help solve it.
.
.
def main(**kwargs):
file_path_audio = kwargs['classpath']
# use the other arguments
.
.
if __name__ == '__main__':
parser = argparse.ArgumentParser(..)
parser.add_argument(
'-c',
'--classpath',
type=str,
help='directory with list of classes',
required=True)
# some more arguments
kwargs = parser.parse_args()
main(**kwargs)
And for the flask script, simply use,
#app.route(...)
def upload_file():
A.main(classpath = 'uploads/', ..) # have to add all the arguments with their defaults
I haven't found any other way than to state all the default arguments while using the main function.
I am completely new to Python, just started today and getting to grips with Python. Running it in Visual Studio btw.
Came across the import argParse and this is where things got a bit confusing for me.
I have some code I'm trying to get to work but it won't and I am quite clueless.
I'm getting an error on the code args = parser.parse_args() and I have no idea why either. Code is below
import math
import argparse
parser = argparse.ArgumentParser(description='calculate')
parser.add_argument('radius', type=int, help="radius plzz")
parser.add_argument('height', type=int, help="height plzz")
args = parser.parse_args()
def cylinder_volume(radius, height):
vol = (math.pi) * (radius ** 2) * height
return vol
if __name__ == '__main__':
print(cylinder_volume(args.radius, args.height))
I do have an idea of what's going on in this code but I don't know why it won't run as expected?
Maybe because I'm using Visual Studio? Maybe I need to import something else..
I have an image of the error!
args = parser.parse_args() parses the command line arguments (accessible as the sys.argv list) and makes the first argument args.radius and the second argument args.height, per the calls to the add_argument method. So all you need to do is to run the script from the command line with two integer arguments, e.g.:
script_name.py 123 456
or to test it in an IDE such as Visual Studio, you can pass a list of arguments to parse_args instead:
args = parser.parse_args(['123', '456'])
which outputs:
21673294.79680895
you can add 'dest=(str)' to send the argument as an attribute of args.
parser.add_argument(
'radius',
type=int,
help="radius plzz",
dest='radius'
)
parser.add_argument(
'height',
type=int,
help="height plzz",
dest='height'
)
Then you can call the arguments as you did in:
print(cylinder_volume(args.radius, args.height))
There are two python scripts, 'script1.py' and 'script2.py'.
'script1.py' uses OptionParser to parse command line arguments.
The contents of 'script1.py' look something like this
from optparse import OptionParser
def main():
parser = OptionParser()
parser.add_option("-o", "--option1")
parser.add_option("-p", "--option2")
(opts, args) = parser.parse_args()
# Do things with the options
if __name__ == '__main__':
main()
To run it on a command line. It is run with:
python script1.py -o Option1 -p Option2
'script2.py' also uses OptionParser implemented in the same way but with a different set of options.
'script2.py' also has 'script1.py' imported as a module.
I would like to run the main of script1.py from script2.py. What is the best way to do this?
One way I got this to work is by changing the main of script1.py to take OptionParser as an arguement.
def main(OptionParser):
...
...
...
if __name__ == '__main__':
main(OptionParser)
And making sure the OptionParser for both the scripts have exactly the same options. If I do that then I can just pass the OptionParser object from script2 into script1 as follows:
script1.main(OptionParser)
Is there a way to achieve the same result without making the OptionParser in both the scripts the same.
Ideally, I would like it to work as follows:
script1.main(option1="Option1", option2="Option2")
This way I can run script1 from the command line as well as from another script.
Edit:
I'm also aware I can used subprocess and os.system() to execute the python script. I'm wondering if there are neater ways to design the interaction between the two scripts.
Edit 2:
As per Mig's suggestion I moved the option parser out of main.
scrip1.py looks as follows now
def main(option1, option2):
# Do main things
if __name__ == '__main__':
parser = OptionParser()
parser.add_option("-o", "--option1")
parser.add_option("-p", "--option2")
(opts, args) = parser.parse_args()
main(option1=opts.option1, option2=opts.option2)
Now from script2.py after importing script1.py as a module I can call main of script1 script1.main(option1="Option1", option2="Option2").
If you have functions that are supposed to work both as main script and as imported, then I would not use opt parser in it. There are many ways to do this but you can have a main that only takes care of your opt parser and then passing right arguments to the function which is really responsible for the job. Do you see what I mean?
Then in this case calling it from the command line will take the arguments from an opt parser, but if you use it as a library, then you call the function doing the job instead.
Another way to do this is which is pretty similar is to keep main as the function doing the real job, but you create the opt parser in the if __name__ == '__main__': block at the end. You build your opt parser in the this block and call main with the arguments it needs.
All in all the principle is to separate the real job from the option parsing.
I don't know all the details of your application, so it may not be the answer you are looking for, but it is quite a common thing to do in many programming languages.
So I have created my argparse which has two different flags. One is -a and the other one is -b. When I run my script damage.py with a specific flag, I want it to be able to execute a function depending on what flag is passed. For example if I pass damage.py -t, it will run the function tester() as shown in my import and print hello, where as if I pass -d it will run another function. So far my code is as follows:
import argparse
def main():
parser = argparse.ArgumentParser()
parser.add_argument("-a", "--export-date", action="store_true", required=True)
parser.add_argument("-b", "--execute-test", action="store_true", required=False)
if __name__ == '__main__':
main()
Rather than saving these values to a variable first you can access them directly like this:
if args.export_date:
# Do something with date
if args.execute_test:
tester()
This means that when you run your program like python damage.py -dt it will run both the code in the date block as in the tester block.
I have a script that finds test names and is widely used in our company. It operates on the command line like so:
find_test.py --type <type> --name <testname>
Inside the script is the equivalent of:
import argparse
parser = argparse.ArgumentParser(description='Get Test Path')
parser.add_argument('--type', dest='TYPE', type=str, default=None, help="TYPE [REQUIRED]")
parser.add_argument('--name', dest='test_name', type=str, default=None, help="Test Name (Slow)")
parser.add_argument('--id', dest='test_id', type=str, default=None, help="Test ID (Fast)")
parser.add_argument('--normalise', dest='normalise', action="store_true", default=False, help="Replace '/' with '.' in Test Name")
args = parser.parse_args()
(Not sure what all these arguments do, I personally only use the first two). These lines are then proceeded by the code that uses these arguments.
I want to refactor this script so I can import it as a module, but also preserve its command line functionality - since lots of people use this script and it is also called in some of our csh scripts.
I have refactored it so far, like so:
def _main():
<all the former code that was in find_test.py>
if __name__ == "__main__":
_main()
And this still runs fine from the command line. But I don't know how then in my parent script I pass arguments with the relevant switches into this.
How do I refactor this further and then call it in parent script?
Is this possible?
I'd also rather not use docopts which i've read is the new argparse unless necessary - i.e. can't be done with argparse, since it's not installed company wide and this can be an arduous procedure.
You shouldn't just move all the code directly into a function; that doesn't help at all.
What you should do is move the code that needs to run whatever happens into a function. (And since it is the external interface, it should not begin with _.) The code that only needs to run from the command line - ie the parser stuff - should stay in the __name__ == '__main__' block, and it should pass its results to main().
So:
def main(TYPE, test_name, test_id=None, normalise=False):
# ... body of script...
if __name__ == "__main__":
parser = ...
...
args = parser.parse_args()
main(**vars(args))
(And docopt isn't the new anything; it's an external library which some people like.)