I have a commandline script that works perfectly fine. Now I want to make my tool more intuitive.
I have:
parser.add_argument("-s",help = "'*.sam','*.fasta','*.fastq'", required=True)
right now, python script.py -s savefile.sam works but I would like it to be python script.py > savefile.sam
parser.add_argument("->",help = "'*.sam','*.fasta','*.fastq'", required=True)
does not work as it gives: error: unrecognized arguments: -
can I do this with argparse or should I settle for -s?
> savefile.sam is shell syntax and means "send output to the file savefile.sam". Argparse won't even see this part of the command because the shell will interpret it first (assuming you issue this command from a suitable shell).
While your command does make sense, you shouldn't try to use argparse to implement it. Instead, if an -s isn't detected, simply send the script's output to stdout. You can achieve this by setting the default for -s:
parser.add_argument("-s",
type=argparse.FileType("w"),
help="'*.sam','*.fasta','*.fastq'",
default=sys.stdout)
This way, you can run python script.py > savefile.sam, and the following will happen:
The shell will evaluate python script.py.
argparse will see no additional arguments, and will use the default sys.stdout.
Your script will send output to stdout.
The shell will redirect the script's output from stdout to savefile.sam.
Of course, you can also send the stdout of the script into the stdin the another process using a pipe.
Note that, using FileType, it's also legal to use -s - to specify stdout. See here for details.
In a sense your argparse works
import argparse
import sys
print sys.argv
parser=argparse.ArgumentParser()
parser.add_argument('->')
print parser.parse_args('-> test'.split())
print parser.parse_args()
with no arguments, it just assigns None to the > attribute. Note though that you can't access this attribute as args.>. You'd have to use getattr(args,'>') (which is what argparse uses internally). Better yet, assign this argument a proper long name or dest.
1401:~/mypy$ python stack29233375.py
['stack29233375.py']
Namespace(>='test')
Namespace(>=None)
But if I give a -> test argument, the shell redirection consumes the >, as shown below:
1405:~/mypy$ python stack29233375.py -> test
usage: stack29233375.py [-h] [-> >]
stack29233375.py: error: unrecognized arguments: -
1408:~/mypy$ cat test
['stack29233375.py', '-']
Namespace(>='test')
Only the - passes through in argv, and on to the parser. So it complains about unrecognized arguments. An optional positional argument could have consumed this string, resulting in no errors.
Notice that I had to look at the test file to see the rest of the output - because the shell redirected stdout to test. The error message goes to stderr, so it doesn't get redirected.
You can change the prefix char from - (or in addition to it) with an ArgumentParser parameter. But you can't use such a character alone. The flag must be prefix + char (i.e. 2 characters).
Finally, since this argument is required, do you even need a flag? Just make it a positional argument. It's quite common for scripts to take input and/or output file names as positional arguments.
Related
I am designing a tool to meet some spec. I have a scenario where I want the argument to contain - its string. Pay attention to arg-1 in the below line.
python test.py --arg-1 arg1Data
I am using the argparse library on python27. For some reason the argparse gets confused with the above trial.
My question is how to avoid this? How can I keep the - in my argument?
A sample program (containing the -, if this is removed everything works fine):
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--arg-1", help="increase output verbosity")
args = parser.parse_args()
if args.args-1:
print "verbosity turned on"
Python argparse module replace dashes by underscores, thus:
if args.arg_1:
print "verbosity turned on"
Python doc (second paragraph of section 15.4.3.11. dest) states:
Any internal - characters will be converted to _ characters to make
sure the string is a valid attribute name.
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("--arg-1", help="increase output verbosity")
parser.add_argument("arg-2")
args = parser.parse_args()
print(args)
produces:
1750:~/mypy$ python stack34970533.py -h
usage: stack34970533.py [-h] [--arg-1 ARG_1] arg-2
positional arguments:
arg-2
optional arguments:
-h, --help show this help message and exit
--arg-1 ARG_1 increase output verbosity
and
1751:~/mypy$ python stack34970533.py --arg-1 xxx yyy
Namespace(arg-2='yyy', arg_1='xxx')
The first argument is an optional. You can use '--arg-1' in commandline, but the value is stored as args.arg_1. Python would interpret args.arg-1 as args.arg - 1. There's a long history of unix commandlines allowing flags with a -. It tries to balance both traditions.
It leaves you in full control of the positionals dest attribute, and does not change the - to _. If you want to access that you have to use the getattr approach. There is bug/issue discussing whether this behavior should be changed or not. But for now, if you want to make it hard on yourself, that's your business.
Internally, argparse accesses the namespace with getattr and setattr to minimize restrictions on the attribute names.
A non-Python program will call a Python program with both input and output arguments. Input may be a file reference or a string redirected to stdin in the non-Python program. Output may be a file or stdout.
argparse.FileType seems ready to handle this as it already has the special - to direct to stdin/stdout. In fact, using - to direct to stdout works but the implementation/syntax for stdin is what I don't know.
Examples calls in the non-Python code:
python mycode.py - output.txt
python mycode.py - -
What does the non-Python code do after that? Print/stdout an input string?
What does the Python code do after that?
I will always need to distinguish where both args are going (i.e. input and output) so using default="-" nor default=sys.stdin in add_argument won't work because they require an absent argument.
Here's what I have so far:
parser = argparse.ArgumentParser()
parser.add_argument('read_fref', type=argparse.FileType('r'))
parser.add_argument('write_fref', type=argparse.FileType('w'))
parser_ns = parser.parse_args()
with parser_ns.read_fref as f_r:
read_f = json.load(f_r)
output = {'k': 'v'}
with parser_ns.write_fref as f_w:
json.dump(output, f_w)
I'm having trouble understanding what you are asking. I understand what Python and argparse are doing, but I don't quite understand what you are trying to do.
Your sample looks like it would run fine when called from a Linux shell. With the the - arguments, it should accept input from the keyboard, and display it on the screen. But those arguments are most often used with shell redirection controls >, <, | (details vary with shell, sh, bash, etc).
But if you are using the shell to redirect stdin or stdout to/from files, you could just as well give those files as commandline arguments.
If you are bothered by required/default issue, consider making these arguments flagged (also called optionals):
parser.add_argument('-r','--readfile', type=argparse.FileType('r'), default='-')
parser.add_argument('-w','--writefile', type=argparse.FileType('w'), default='-')
With this change, these calls are the same
python mycode.py -r - <test.json
python mycode.py <test.json
python mycode.py -r test.json
all writing to the screen (stdout). That could be redirected in similar ways.
To take typed input:
python mycode.py
{...}
^D
I have a question about argparse.
Here is part of my code:
(...)
def main():
parser = argparse.ArgumentParser()
parser.add_argument('-g', '--gfirst', dest="a", type=int, required=True)
args = parser.parse_args()
print args.a #Testing
print args.a #Testing
if __name__ == '__main__':
main()
print "3"
unittest.main(verbosity=2)
print "4"
(...)
I am trying to set 'a' as a required value to execute the test cases, because I will need this value in the future. However...
$ python regular_test.py --gfirst 2
2
2
3
option --gfirst not recognized
Usage: regular-test.py [options] [test] [...]
Options:
-h, --help Show this message
-v, --verbose Verbose output
-q, --quiet Minimal output
-f, --failfast Stop on first failure
-c, --catch Catch control-C and display results
-b, --buffer Buffer stdout and stderr during test runs
...as you can see, the program accepts the argument and prints it, but the test case itself does not execute. I've inserted some prints to show whats executing and what isnt.
What am I doing wrong?
Thanks in advance
ps.: I am using python 2.7.3
ps2.:The tests were running properly (before adding argparse to the program.)
unittest.main() itself parses command-line arguments and it cannot understand/recognize your custom defined arguments (see parseArgs() method of the TestProgram class).
Instead, run tests using TextTestRunner:
runner = unittest.TextTestRunner(verbosity=2)
result = runner.run(unittest.makeSuite(MyTestCase))
Also see related threads:
argparse and unittest python
Call a python unittest from another script and export all the error messages
Your problem is that the unit-tester wants to own any command-line arguments. It might make it complicated to prescribe your own arguments.
Technically, unit-tests should contain everything they need in order to run, and shouldn't depend on arguments. You might consider moving any environment-related configuration (like a DB hostname, for example) to environment variables.
My two cents.
I would like to be able to log the command used to run the current python script within the script itself. For instance this is something I tried:
#test.py
import sys,subprocess
with open('~/.bash_history','r') as f:
for line in f.readlines():
continue
with open('logfile','r') as f:
f.write('the command you ran: %s'%line.strip('\n'))
However the .bash_history does not seem to be ordered in chronological order. What's the best recommended way to achieve the above for easy logging? Thanks.
Update: unfortunately sys.argv doesn't quite solve my problem because I need to use process subtitution as input variables sometimes.
e.g. python test.py <( cat file | head -3)
What you want to do is not universally possible. As devnull says, the history file in bash is not written for every command typed. In some cases it's not written at all (user sets HISTFILESIZE=0, or uses a different shell).
The command as typed is parsed and processed long before your python script is invoked. Your question is therefore not related to python at all. Wether what you want to do is possible or not is entirely up to the invoking shell. bash does not provide what you want.
If your can control the caller's shell, you could try using zsh instead. There, if you setopt INC_APPEND_HISTORY, zsh will append to its history file for each command typed, so you can do the parse history file hack.
One option is to use sys.argv. It will contain a list of arguments you passed to the script.
import sys
print 'Number of arguments:', len(sys.argv), 'arguments.'
print 'Argument List:', str(sys.argv)
Example output:
>python test.py
Number of arguments: 1 arguments.
Argument List: ['test.py']
>python test.py -l ten
Number of arguments: 3 arguments.
Argument List: ['test.py', '-l', 'ten']
As you can see, the sys.argv variable contains the name of the script and then each individual parameter passed. It does miss the python portion of the command, though.
I have the following code, which runs a command with arguments
subprocess.call(["schtasks.exe", "/Query","/V","/FO","csv",">>", r"D:/temp/fo3.csv"])
However I am having problems with the >> part of the argument, the command does not seem to like it, it gives the following message
ERROR Invalid argument /option - >>
So my question is how do i get the >> argument to work?
when you pass arguments as a list like that, subprocess passes each argument to the program. What is probably happening is that your program (schtasks.exe) is seeing the argument >> and it doesn't know what to do with it. When you run this in the shell, >> is interpreted by the shell as redirection and so your program never sees it.
You have 2 options, pass the arguments as a string and use shell = True. This is not recommended if you are accepting user input which can modify the string you're passing to subprocess as it would lead to a security risk in your program.
Your second option is to pass an open file object to stdout, e.g.
f = open(r"D:/temp/fo3.csv",'a')
subprocess.call(["schtasks.exe", "/Query","/V","/FO","csv"], stdout = f)
Output redirection (the >>) is a shell feature and won't work if you call the program directly. Use shell=True in the call.