I am trying to code a module for my deep learning model. I wish to store these arguments using argeparse. There is some problem occuring in args = parser.parse_args()! Also What are the benefits of using the argparse library?
import numpy as np
import argparse
if __name__ == '__main__':
parser = argparse.ArgumentParser()
parser.add_argument('--dataset', type=str,
help='Dataset')
parser.add_argument('--epoch', type=int, default=40,
help='Training Epochs')
parser.add_argument('--node_dim', type=int, default=64,
help='Node dimension')
parser.add_argument('--num_channels', type=int, default=2,
help='number of channels')
parser.add_argument('--lr', type=float, default=0.005,
help='learning rate')
parser.add_argument('--weight_decay', type=float, default=0.001,
help='l2 reg')
parser.add_argument('--num_layers', type=int, default=2,
help='number of layer')
parser.add_argument('--norm', type=str, default='true',
help='normalization')
parser.add_argument('--adaptive_lr', type=str, default='false',
help='adaptive learning rate')
args = parser.parse_args()
print(args)
The above code is a part of full code, as given in the link below
Error:
usage: ipykernel_launcher.py [-h] [--dataset DATASET] [--epoch EPOCH]
[--node_dim NODE_DIM]
[--num_channels NUM_CHANNELS] [--lr LR]
[--weight_decay WEIGHT_DECAY]
[--num_layers NUM_LAYERS] [--norm NORM]
[--adaptive_lr ADAPTIVE_LR]
ipykernel_launcher.py: error: unrecognized arguments: -f /Users/anshumansinha/Library/Jupyter/runtime/kernel-1e4c0f41-a4d7-4388-be24-640dd3411f56.json
An exception has occurred, use %tb to see the full traceback.
SystemExit: 2
The above code is a part of a code, the full code can be seen on the following github link : link
You called ipykernel_launcher.py with the argument -f. But -f is not in your list of arguments.
Is -f the argument, which you want to use for the input file? Then you should add something like this to your code:
parser.add_argument('--file', '-f', type=str)
The benefit of argparse is that you don't have to write by hand the code for parsing the arguments. You can try this. It is more code than one normally thinks. Especially if you have positional arguments.
Related
How can I run this?
ap = argparse.ArgumentParser()
ap.add_argument("-p", "--shape-predictor", required=True,
help="path to facial landmark predictor")
ap.add_argument("-a", "--alarm", type=str, default="",
help="path alarm .WAV file")
ap.add_argument("-w", "--webcam", type=int, default=0,
help="index of webcam on system")
args = vars(ap.parse_args())
While I run this on Spyder it gives
usage: [-h] -p SHAPE_PREDICTOR [-a ALARM] [-w WEBCAM]
: error: the following arguments are required: -p/--shape-predictor
An exception has occurred, use %tb to see the full traceback.
How can I solve this issue?
From #hiroprotagonist's comment posted above:
In spyder ctrl+F6 should allow you to pass command-line args when you execute your script. see here https://stackoverflow.com/a/26766414/4954037.
When I run the following code:
def read_args():
parser = default_parser()
parser.add_argument('--tensorboard-dir', type=str, default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000, help="Use N training examples.")
return parser.parse_args()
def main():
flags = readargs()
I have the following error output:
The following arguments are required: --name
However when I add the --name argument:
def read_args():
parser = default_parser()
parser.add_argument('--name', type=str, default='cifar10test')
parser.add_argument('--tensorboard-dir', type=str, default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000, help="Use N training examples.")
return parser.parse_args()
def main():
flags = readargs()
is also creating problem.
Any ideas?
It appears that default_parser contains a --name argument which is required. What you're doing in your second example is defining the argument twice - once in default_parser and once in your program. Instead, you should be passing a --name argument when calling your program from the command line.
Example:
python cifar.py -N=1200 --tensorboard-dir=file.txt --name=cool_name
Alternatively, you could remove default_parser and construct your own ArgumentParser:
`parser = argparse.ArgumentParser()`
Full working demo:
import argparse
def read_args():
parser = argparse.ArgumentParser()
parser.add_argument('--tensorboard-dir', type=str,
default='/tmp/cifar10/tensorboard')
parser.add_argument('-N', type=int, default=50000,
help="Use N training examples.")
return parser.parse_args()
def main():
flags = vars(read_args())
# You can access your args as a dictionary
for key in flags:
print("{} is {}".format(key, flags[key]))
main()
The parser returns a Namespace object, but we can access its (much simpler) internal dictionary using vars(Namespace). You can then get your arguments by accessing the dictionary, for example, flags['N']. Note that tensorboard-dir becomes tensorboard_dir inside your python program to avoid issues with the subtraction operator.
Call it from the command line (I'm using Bash):
python cifar.py -N=1200 --tensorboard-dir=file.txt
Output:
tensorboard_dir is file.txt
N is 1200
I'm trying to build a script that uses subparsers arguments. However, I can not pass any of the sub-arguments as a parameter. Resulting in "invalid choice:" for any input combination.
Example input:
python3 preprocess.py -d ../data/acm/ tf -l en
Complete Output:
usage: preprocess.py [-h] [-k FOLDS] -d DATASETDIR [DATASETDIR ...] {tf} ...
preprocess.py: error: invalid choice: 'en' (choose from 'tf')
The code is
parser = argparse.ArgumentParser(description='Split input dataset into k folds of cross-validation.')
parser.add_argument('-k', '--folds', default=10, help='Number of folds for K fold cross-validation.', type=int)
required_args = parser.add_argument_group('required arguments')
required_args.add_argument('-d','--datasetdir', type=str, nargs='+', help='Dataset path (For more info: readme.txt)', required=True)
parser_subparsers = parser.add_subparsers(title="Representations", description="Choose the representations")
parser_tf = parser_subparsers.add_parser('tf', help='TF helper')
parser_tf.add_argument('-l', '--language', type=str, help='Language', default='en', choices=['en'])
parser_tf.add_argument('-s', '--stopword', type=bool, help='Skip stopwords', default=True)
args = parser.parse_args()
Since --datasetdir has nargs="+" the other argument(s) are being slurped up as additional dataset dir, rather than invoking the subparser.
CLI suggestion: change datasetdir into a plain old positional argument, with ability to separate paths using os.pathsep. It will be difficult to wrangle argparse into what you wanted to do, and using optional arguments with required=True is a code smell in the first place.
New interface will look something like this:
python3 preprocess.py ../data/acm/:/dir2:/dir3 tf -l en
I am trying to pass a .py file to ipython notebook environment.
I have never had to deal directly with argparse before. How do I rewrite the main() function?
I tried to delete the line of def main(): and keep the rest of the code.
But args = parser.parse_args()" returned an error:
ipykernel_launcher.py: error: unrecognized arguments: -f.
And when I run . %tb:
showing this
def main():
parser = argparse.ArgumentParser()
parser.add_argument('--data_dir', type=str, default='data/tinyshakespeare',
help='data directory containing input.txt')
parser.add_argument('--input_encoding', type=str, default=None,
help='character encoding of input.txt, from https://docs.python.org/3/library/codecs.html#standard-encodings')
parser.add_argument('--log_dir', type=str, default='logs',
help='directory containing tensorboard logs')
parser.add_argument('--save_dir', type=str, default='save',
help='directory to store checkpointed models')
parser.add_argument('--rnn_size', type=int, default=256,
help='size of RNN hidden state')
parser.add_argument('--num_layers', type=int, default=2,
help='number of layers in the RNN')
parser.add_argument('--model', type=str, default='lstm',
help='rnn, gru, or lstm')
parser.add_argument('--batch_size', type=int, default=50,
help='minibatch size')
parser.add_argument('--seq_length', type=int, default=25,
help='RNN sequence length')
parser.add_argument('--num_epochs', type=int, default=50,
help='number of epochs')
parser.add_argument('--save_every', type=int, default=1000,
help='save frequency')
parser.add_argument('--grad_clip', type=float, default=5.,
help='clip gradients at this value')
parser.add_argument('--learning_rate', type=float, default=0.002,
help='learning rate')
parser.add_argument('--decay_rate', type=float, default=0.97,
help='decay rate for rmsprop')
parser.add_argument('--gpu_mem', type=float, default=0.666,
help='%% of gpu memory to be allocated to this process. Default is 66.6%%')
parser.add_argument('--init_from', type=str, default=None,
help="""continue training from saved model at this path. Path must contain files saved by previous training process:
'config.pkl' : configuration;
'words_vocab.pkl' : vocabulary definitions;
'checkpoint' : paths to model file(s) (created by tf).
Note: this file contains absolute paths, be careful when moving files around;
'model.ckpt-*' : file(s) with model definition (created by tf)
""")
args = parser.parse_args()
train(args)
You can try args = parser.parse_args(args=[]).
It's better to use #nbro 's answer for Jupyter execution.
args = parser.parse_args(args=[])
If you want to manage parameters as class format, you can try this.
class Args:
data = './data/penn'
model = 'LSTM'
emsize = 200
nhid = 200
args=Args()
As #nbro suggested, the following command should work:
args = parser.parse_args(args=[])
In addition, if you have required arguments in your parser, set them inside the list:
args = parser.parse_args(args=['--req_1', '10', '--req_2', '10'])
Where you previously used:
import argparse
parser = argparse.ArgumentParser(description="Dummy parser")
parser.add_argument("--req_1", type=int, required=True, help="required int 1")
parser.add_argument("--req_2", type=int, required=True, help="required int 2")
You can also see from the notebook all params:
print("see all args:", args)
print("use one arg:", args.req_1)
You can find more information in the docs: Parsing arguments
An example is:
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('echo')
args = parser.parse_args(['aa']) # actually you don't have to write (args=['aa'])
print(args.echo)
the output should be
>>> aa
I am using Python argparse to take in parameters through the CLI. I have tried using the following but when I don't give any one of the arguments, it gives the output as None. I want then default ones to be the ones provided in const=. Please take a look.
parser = argparse.ArgumentParser()
parser.add_argument('--input', nargs='?', const='testInput')
parser.add_argument('--target', nargs='?', const='testTarget')
parser.add_argument('--msg', nargs='?', const='helloFromTheOtherSide')
args = parser.parse_args()
print args.input
If I don't give input, it prints it as None as I said. I want it to print TestInput instead..
Use the default argument:
parser = argparse.ArgumentParser()
parser.add_argument('--input', nargs='?', default='testInput')
parser.add_argument('--target', nargs='?', default='testTarget')
parser.add_argument('--msg', nargs='?', default='helloFromTheOtherSide')
args = parser.parse_args()
print args.input
With
parser.add_argument('--input', nargs='?', default='testInput', const='aConst')
you have a 3 way choice
prog # input='testInput'
prog --input # input='aConst'
prog --input myfile # input='myfile'
If you don't need that aConst option, omit the nargs='?'. Since it is a flagged argument it is already optional. It doesn't need the `?'.
parser.add_argument('--input', default='testInput')