Does anybody know what is the equivalent of batch script's %* in python ?
Clarification: in a batch file %* is all parameters specified in the command line -- this is very useful if you want to forward the parameters to another program.
My Problem is that I want to convert one batch call to a python call :
Batch call is :
trial.bat %*
what is the equivalent python call ?(P.S. I know I can just go through the whole sys.argv list and append parameters in some string and pass that to the batch file, but I am looking for a simpler solution here)
I tried following :
os.system('trial.bat '+sys.argv)
os.system('trial.bat '+sys.argv[1:])
But that is not working. I tried similar thing using argparse as well. Its not working either. Please help.
sys.argv[1:] is pretty close. The thing is that argv is a list of arguments, and [1:] is a slice of a list, which is again a list. If you want a string with all arguments combined, you can join them again:
os.system('trial.bat ' + ' '.join(sys.argv[1:]))
Or even better, you use the subprocess module which method’s accept a list of arguments:
subprocess.check_call(['trial.bat'] + sys.argv[1:])
Subprocess is much more flexible when handling parameters and will also behave similar to the parsing in argv. As an example, when calling a script with the arguments foo "hello world" bar, argv will contain this:
>>> sys.argv[1:]
['foo', 'hello world', 'bar']
Now if we were to simply join this list, we would get a single string
>>> ' '.join(sys.argv[1:])
'foo hello world bar'
As you can see, the information of the compound argument hello world is lost, resulting in a completely different meaning.
When using subprocess however, you keep your list and subprocess will automatically make sure to pass these arguments correctly to the called program. So the called program will be able to get hello world as a combined argument too.
You want subprocess.Popen (or one of it's convenience wrappers):
import subprocess
import sys
process = subprocess.Popen(['trial.bat'] + sys.argv[1:])
process.wait()
It's definitely preferred to os.system. The advantage here is that commandline arguments which may need to be quoted to keep their meaning effectively stay quoted. Additionally, this alternative is probably safer than os.system since it avoids creating a subshell.
If you want to use os.system, you need to put the command line back together manually. Python has already parsed the command line apart into separate arguments (or MSVCRT has done it on Python's behalf). This means you need to not just concatenate them back together, but also quote them appropriately.
There is nothing in the stdlib that handles the "quote them appropriately" exactly the way MSVCRT wants. That's partly because Windows quoting is actually ambiguous; there are some cases where it is impossible to round-trip things. But, for simple cases, either POSIX-style quoting (with shlex.quote) or just sticking explicit quotes around each argument will work. So, either of these:
args = ' '.join(shlex.quote(arg) for arg in [program] + sys.argv[1:])
args = ' '.join('"{}"'.format(arg) for arg in [program] + sys.argv[1:])
Then:
os.system(args)
But using subprocess is better than os.system. One reason is that you don't have to fiddle with quoting things; you can just do this:
subprocess.check_call([program] + sys.argv[1:], shell=True)
Someone still needs to put the list of arguments back together in a string so it can be passed to the shell, but now that "someone" is the subprocess module rather than your code.
Related
I have a python script that I'd like to supply with an argument (usually) containing wildcards, referring to a series of files that I'd like to do stuff with. Example here:
#!/usr/bin/env python
import argparse
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-i', action="store", dest="i")
results = parser.parse_args()
print 'argument i is: ', results.i
list_of_matched_files = glob.glob(results.i)
In this case, everything works great if the user adds quotes to the passed argument like so:
./test_script.py -i "foo*.txt"
...but often times the users forget to add quotes to the argument and are stumped when the list only contains the first match because UNIX already expanded the list and argparse only then gets the first list element.
Is there a way (within the script) to prevent UNIX from expanding the list before passing it to python? Or maybe even just to test if the argument doesn't contain quotes and then warn the user?
No. Wildcards are expanded by the shell (Bash, zsh, csh, fish, whatever) before the script even runs, and the script can't do anything about them. Testing whether the argument contains quotes also won't work, as the shell similarly strips the quotes from "foo*.txt" before passing the argument to the script, so all Python sees is foo*.txt.
Its not UNIX that is doing the expansion, it is the shell.
Bash has an option set -o noglob (or -f) which turns off globbing (filename expansion), but that is non-standard.
If you give an end-user access to the command-line then they really should know about quoting. For example, the commonly used find command has a -name parameter which can take glob constructs but they have to be quoted in a similar manner. Your program is no different to any other.
If users can't handle that then maybe you should give them a different interface. You could go to the extreme of writing a GUI or a web/HTML front-end, but that's probably over the top.
Or why not prompt for the filename pattern? You could, for example, use a -p option to indicate prompting, e.g:
import argparse
import glob
parser = argparse.ArgumentParser()
parser.add_argument('-i', action="store", dest="i")
parser.add_argument('-p', action="store_true", default=False)
results = parser.parse_args()
if results.p:
pattern = raw_input("Enter filename pattern: ")
else:
pattern = results.i
list_of_matched_files = glob.glob(pattern)
print list_of_matched_files
(I have assumed Python 2 because of your print statement)
Here the input is not read by the shell but by python, which will not expand glob constructs unless you ask it to.
You can disable the expansion using set -f from the command line. (re-enable with set +f).
As jwodder correctly says though, this happens before the script is run, so the only way I can think of to do this is to wrap it with a shell script that disables expansion temporarily, runs the python script, and re-enables expansion. Preventing UNIX from expanding the list before passing it to python is not possible.
Here is an example for the Bash shell that shows what #Tom Wyllie is talking about:
alias sea='set -f; search_function'
search_function() { perl /home/scripts/search.pl $# ; set +f; }
This defines an alias called "sea" that:
Turns off expansion ("set -f")
Runs the search_function function which is a perl script
Turns expansion back on ("set +f")
The problem with this is that if a user stops execution with ^C or some such then the expansion may not be turned back on leaving the user puzzling why "ls *" is not working. So I'm not necessarily advocating using this. :).
This worked for me:
files = sys.argv[1:]
Even though only one string is on the command line, the shell expands the wildcards and fills sys.argv[] with the list.
I've GUI where if i press a button "process" calls a python script which needs one command line argument. so i used subprocess.Popen like this:
subprocess.Popen(['chbif /home/vnandak/Work/VS30_Cradle/SV000_sv000/cradle_vs30_dkaplus01_fwd_dl140606_fem140704_v00_mod1.bif'],shell=True)
chbif is the alias for the .py script
This works fine but now i want to choose a file of my choice by browsing using askfilename() func from tkFileDailog() module. how can i do this?
I thought of doing it like this:
def bifcall():
name13= askopenfilename()
subprocess.Popen(['chbif', name13],shell=True)
But if i use this it throws an error saying that it does'nt have one more command line argument which is the file name
If you pass shell=True, the first argument should be a string, and it will be interpreted as the shell would interpret it. If shell=False, then the first argument should be a list, which sort of bypasses the shell.
Here's an answer to another question that explains it well: https://stackoverflow.com/a/15109975/451201
Because if you use shell=True, you can only use one string as the first argument.
Try to change
subprocess.Popen(['chbif', name13],shell=True)
to
subprocess.Popen('chbif ' + name13,shell=True)
But it is not recommended to use shell=True for security reasons (See https://docs.python.org/2/library/subprocess.html#frequently-used-arguments for details).
I am a bit confused as to how to get this done.
What I need to do is call an external command, from within a Python script, that takes as input several arguments, and a file name.
Let's call the executable that I am calling "prog", the input file "file", so the command line (in Bash terminal) looks like this:
$ prog --{arg1} {arg2} < {file}
In the above {arg1} is a string, and {arg2} is an integer.
If I use the following:
#!/usr/bin/python
import subprocess as sbp
sbp.call(["prog","--{arg1}","{arg2}","<","{file}"])
The result is an error output from "prog", where it claims that the input is missing {arg2}
The following produces an interesting error:
#!/usr/bin/python
import subprocess as sbp
sbp.call(["prog","--{arg1} {arg2} < {file}"])
all the spaces seem to have been removed from the second string, and equal sign appended at the very end:
command not found --{arg1}{arg2}<{file}=
None of this behavior seems to make any sense to me, and there isn't much that one can go by from the Python man pages found online. Please note that replacing sbp.call with sbp.Popen does not fix the problem.
The issue is that < {file} isn’t actually an argument to the program, but is syntax for the shell to set up redirection. You can tell Python to use the shell, or you can setup the redirection yourself.
from subprocess import *
# have shell interpret redirection
check_call('wc -l < /etc/hosts', shell=True)
# set up redirection in Python
with open('/etc/hosts', 'r') as f:
check_call(['wc', '-l'], stdin=f.fileno())
The advantage of the first method is that it’s faster and easier to type. There are a lot of disadvantages, though: it’s potentially slower since you’re launching a shell; it’s potentially non-portable because it depends on the operating system shell’s syntax; and it can easily break when there are spaces or other special characters in filenames.
So the second method is preferred.
I am trying to use the grep command in a python script using the subprocess module.
Here's what I have:
userid = 'foo12'
p = subprocess.Popen(['grep', "%s *.log"%userid], stdout=subprocess.PIPE)
And it returns nothing.
I am not entirely sure what I am doing wrong so can someone please explain. The current method that I am using that works is by adding the shell=true which makes it output the correct output but as the help pages have pointed out it is unsafe. I need help trying to make this work so that my script isn't unsafe.
I think you're running up against two problems:
This call:
p = subprocess.Popen(['grep', "%s *.log"%userid]...
will not work as expected without shell=True because the list of arguments are being passed directly to os.execvp, which requires each item to be a single string representing an argument. You've squished two separate arguments together into a single string (in other words, grep is interpreting "foo12 *.log" as the pattern to search, and not pattern+file list).
You can fix this by saying:
p = subprocess.Popen(['grep', userid, '*.log']...)
The second issue is that, again without shell=True, execvp doesn't know what you mean by *.log and passes it directly along to grep, without going through the shell's wildcard expansion mechanism. If you don't want to use shell=True, you can instead do something like:
import glob
args = ['grep', userid]
args.extend(glob.glob('*.log')
p = subprocess.Popen(args, ...)
Here are two tested pieces of code to model from:
>>> print subprocess.check_output(['grep', 'python', 'api_talk.txt'])
Discuss python API patterns
Limitations of python
Introspection in python
>>> print subprocess.check_output('grep python *.txt', shell=True)
Use the latter if you want the shell to do wildcard expansion for you. When shell is True, be sure to put the whole command in a single string rather than a list of separate fields.
I am assuming you want to grep for 'foo12' in all files that end with '.log', to get this to work with just subprocess you will need to change your code to the following:
userid = 'foo12'
p = subprocess.Popen('grep %s *.log' % userid, stdout=subprocess.PIPE, shell=True)
shell=True is necessary for the wildcard expansion, and when that option is set you need to provide a string command instead of a list.
Also, make sure when you are providing a list of arguments that each argument is a separate entry in the list, your initial code would have been equivalent to the following:
grep 'foo12 *.log'
I'm currently teaching myself Python and was just wondering (In reference to my example below) in simplified terms what the sys.argv[1] represents. Is it simply asking for an input?
#!/usr/bin/python3.1
# import modules used here -- sys is a very standard one
import sys
# Gather our code in a main() function
def main():
print ('Hello there', sys.argv[1])
# Command line args are in sys.argv[1], sys.argv[2] ..
# sys.argv[0] is the script name itself and can be ignored
# Standard boilerplate to call the main() function to begin
# the program.
if __name__ == '__main__':
main()
You may have been directed here because you were asking about an IndexError in your code that uses sys.argv. The problem is not in your code; the problem is that you need to run the program in a way that makes sys.argv contain the right values. Please read the answers to understand how sys.argv works.
If you have read and understood the answers, and are still having problems on Windows, check if Python Script does not take sys.argv in Windows fixes the issue. If you are trying to run the program from inside an IDE, you may need IDE-specific help - please search, but first check if you can run the program successfully from the command line.
I would like to note that previous answers made many assumptions about the user's knowledge. This answer attempts to answer the question at a more tutorial level.
For every invocation of Python, sys.argv is automatically a list of strings representing the arguments (as separated by spaces) on the command-line. The name comes from the C programming convention in which argv and argc represent the command line arguments.
You'll want to learn more about lists and strings as you're familiarizing yourself with Python, but in the meantime, here are a few things to know.
You can simply create a script that prints the arguments as they're represented. It also prints the number of arguments, using the len function on the list.
from __future__ import print_function
import sys
print(sys.argv, len(sys.argv))
The script requires Python 2.6 or later. If you call this script print_args.py, you can invoke it with different arguments to see what happens.
> python print_args.py
['print_args.py'] 1
> python print_args.py foo and bar
['print_args.py', 'foo', 'and', 'bar'] 4
> python print_args.py "foo and bar"
['print_args.py', 'foo and bar'] 2
> python print_args.py "foo and bar" and baz
['print_args.py', 'foo and bar', 'and', 'baz'] 4
As you can see, the command-line arguments include the script name but not the interpreter name. In this sense, Python treats the script as the executable. If you need to know the name of the executable (python in this case), you can use sys.executable.
You can see from the examples that it is possible to receive arguments that do contain spaces if the user invoked the script with arguments encapsulated in quotes, so what you get is the list of arguments as supplied by the user.
Now in your Python code, you can use this list of strings as input to your program. Since lists are indexed by zero-based integers, you can get the individual items using the list[0] syntax. For example, to get the script name:
script_name = sys.argv[0] # this will always work.
Although interesting, you rarely need to know your script name. To get the first argument after the script for a filename, you could do the following:
filename = sys.argv[1]
This is a very common usage, but note that it will fail with an IndexError if no argument was supplied.
Also, Python lets you reference a slice of a list, so to get another list of just the user-supplied arguments (but without the script name), you can do
user_args = sys.argv[1:] # get everything after the script name
Additionally, Python allows you to assign a sequence of items (including lists) to variable names. So if you expect the user to always supply two arguments, you can assign those arguments (as strings) to two variables:
user_args = sys.argv[1:]
fun, games = user_args # len(user_args) had better be 2
So, to answer your specific question, sys.argv[1] represents the first command-line argument (as a string) supplied to the script in question. It will not prompt for input, but it will fail with an IndexError if no arguments are supplied on the command-line following the script name.
sys.argv[1] contains the first command line argument passed to your script.
For example, if your script is named hello.py and you issue:
$ python3.1 hello.py foo
or:
$ chmod +x hello.py # make script executable
$ ./hello.py foo
Your script will print:
Hello there foo
sys.argv is a list.
This list is created by your command line, it's a list of your command line arguments.
For example:
in your command line you input something like this,
python3.2 file.py something
sys.argv will become a list ['file.py', 'something']
In this case sys.argv[1] = 'something'
Just adding to Frederic's answer, for example if you call your script as follows:
./myscript.py foo bar
sys.argv[0] would be "./myscript.py"
sys.argv[1] would be "foo" and
sys.argv[2] would be "bar" ... and so forth.
In your example code, if you call the script as follows ./myscript.py foo , the script's output will be "Hello there foo".
Adding a few more points to Jason's Answer :
For taking all user provided arguments: user_args = sys.argv[1:]
Consider the sys.argv as a list of strings as (mentioned by Jason). So all the list manipulations will apply here. This is called "List Slicing". For more info visit here.
The syntax is like this: list[start:end:step]. If you omit start, it will default to 0, and if you omit end, it will default to length of list.
Suppose you only want to take all the arguments after 3rd argument, then:
user_args = sys.argv[3:]
Suppose you only want the first two arguments, then:
user_args = sys.argv[0:2] or user_args = sys.argv[:2]
Suppose you want arguments 2 to 4:
user_args = sys.argv[2:4]
Suppose you want the last argument (last argument is always -1, so what is happening here is we start the count from back. So start is last, no end, no step):
user_args = sys.argv[-1]
Suppose you want the second last argument:
user_args = sys.argv[-2]
Suppose you want the last two arguments:
user_args = sys.argv[-2:]
Suppose you want the last two arguments. Here, start is -2, that is second last item and then to the end (denoted by :):
user_args = sys.argv[-2:]
Suppose you want the everything except last two arguments. Here, start is 0 (by default), and end is second last item:
user_args = sys.argv[:-2]
Suppose you want the arguments in reverse order:
user_args = sys.argv[::-1]
sys.argv is a list containing the script path and command line arguments; i.e. sys.argv[0] is the path of the script you're running and all following members are arguments.
To pass arguments to your python script
while running a script via command line
> python create_thumbnail.py test1.jpg test2.jpg
here,
script name - create_thumbnail.py,
argument 1 - test1.jpg,
argument 2 - test2.jpg
With in the create_thumbnail.py script i use
sys.argv[1:]
which give me the list of arguments i passed in command line as
['test1.jpg', 'test2.jpg']
sys.argv is a attribute of the sys module. It says the arguments passed into the file in the command line. sys.argv[0] catches the directory where the file is located. sys.argv[1] returns the first argument passed in the command line. Think like we have a example.py file.
example.py
import sys # Importing the main sys module to catch the arguments
print(sys.argv[1]) # Printing the first argument
Now here in the command prompt when we do this:
python example.py
It will throw a index error at line 2. Cause there is no argument passed yet. You can see the length of the arguments passed by user using if len(sys.argv) >= 1: # Code.
If we run the example.py with passing a argument
python example.py args
It prints:
args
Because it was the first arguement! Let's say we have made it a executable file using PyInstaller. We would do this:
example argumentpassed
It prints:
argumentpassed
It's really helpful when you are making a command in the terminal. First check the length of the arguments. If no arguments passed, do the help text.
sys.argv will display the command line args passed when running a script or you can say sys.argv will store the command line arguments passed in python while running from terminal.
Just try this:
import sys
print sys.argv
argv stores all the arguments passed in a python list. The above will print all arguments passed will running the script.
Now try this running your filename.py like this:
python filename.py example example1
this will print 3 arguments in a list.
sys.argv[0] #is the first argument passed, which is basically the filename.
Similarly, argv[1] is the first argument passed, in this case 'example'.