Scratching my head... this curl command will work fine from the command line when I copy it from here and paste it in my Windows 7 command line, but I can't get it to execute in my Python 2.7.9 script. Says the system cannot find the specified file. Popen using 'ping' or something like that works just fine, so I'm sure this is a goober typo that I'm just not seeing. I would appreciate a separate set of eyes and any comments as to what is wrong.
proc = subprocess.Popen("curl --ntlm -u : --upload-file c:\\temp\\test.xlsx http://site.domain.com/sites/site/SiteDirectory/folder/test.xlsx")
Have a look at second two paragraphs of the subprocess.Popen documentation if you haven't already:
args should be a sequence of program arguments or else a single string. By default, the program to execute is the first item in args if args is a sequence. If args is a string, the interpretation is platform-dependent and described below. See the shell and executable arguments for additional differences from the default behavior. Unless otherwise stated, it is recommended to pass args as a sequence.
On Unix, if args is a string, the string is interpreted as the name or path of the program to execute. However, this can only be done if not passing arguments to the program. [emphasis mine]
Instead you should pass in a list in which each argument to the program (including the executable name itself) is given as a separate item in the list. This is generally going to be safer in a cross-platform context anyways.
Update: I see now that you're using Windows in which case the advice on UNIX doesn't apply. On Windows though things are even more hairy. The best advice remains to use a list :)
Update 2: Another possible issue (and in fact the OP's issue as reported in the comments on this answer) is that because the full path to the curl executable was not given, it may not be found if the Python interpreter is running in an environment with a different PATH environment variable.
I would like to be able to log the command used to run the current python script within the script itself. For instance this is something I tried:
#test.py
import sys,subprocess
with open('~/.bash_history','r') as f:
for line in f.readlines():
continue
with open('logfile','r') as f:
f.write('the command you ran: %s'%line.strip('\n'))
However the .bash_history does not seem to be ordered in chronological order. What's the best recommended way to achieve the above for easy logging? Thanks.
Update: unfortunately sys.argv doesn't quite solve my problem because I need to use process subtitution as input variables sometimes.
e.g. python test.py <( cat file | head -3)
What you want to do is not universally possible. As devnull says, the history file in bash is not written for every command typed. In some cases it's not written at all (user sets HISTFILESIZE=0, or uses a different shell).
The command as typed is parsed and processed long before your python script is invoked. Your question is therefore not related to python at all. Wether what you want to do is possible or not is entirely up to the invoking shell. bash does not provide what you want.
If your can control the caller's shell, you could try using zsh instead. There, if you setopt INC_APPEND_HISTORY, zsh will append to its history file for each command typed, so you can do the parse history file hack.
One option is to use sys.argv. It will contain a list of arguments you passed to the script.
import sys
print 'Number of arguments:', len(sys.argv), 'arguments.'
print 'Argument List:', str(sys.argv)
Example output:
>python test.py
Number of arguments: 1 arguments.
Argument List: ['test.py']
>python test.py -l ten
Number of arguments: 3 arguments.
Argument List: ['test.py', '-l', 'ten']
As you can see, the sys.argv variable contains the name of the script and then each individual parameter passed. It does miss the python portion of the command, though.
I was reading programming python 4th edition by Mark Luze, Oreilly, by teaching myself.
There's an example on how to fork a child process, which I do not quite understand:
os.execlp('python', 'python', 'child.py', #other args#)
In an interactive shell(like bash), I know I can type python child.py #args# to ask python interpreter to run child.py with args.
Why are there TWO 'python' in the execlp() function? If I put only one python in the function, I would get an error complainting cannot find file or directory, which is the 1st args of child.py
The first argument is the program to execute (found on the PATH). The rest are the sys.argv arguments to the program.
The first such argument is the program name used to invoke it, and the display value used in the OS process list. It is the value of sys.argv[0] in a python script.
First of all, execlp is rarely used today. In most cases, you'd use the subprocess module, like this:
subprocess.call(['python', 'child.py'])
The first argument of execlp is the file you want to execute.
The latter arguments form the argument array to that program (sys.argv in Python). The first argument is then the name the program got invoked with. For example, Python sets the name to '-c' if the program is being run with the -c option. Similarly, grep behaves differently depending on the first argument, so that users can execute rgrep to imply grep -r.
I'm currently teaching myself Python and was just wondering (In reference to my example below) in simplified terms what the sys.argv[1] represents. Is it simply asking for an input?
#!/usr/bin/python3.1
# import modules used here -- sys is a very standard one
import sys
# Gather our code in a main() function
def main():
print ('Hello there', sys.argv[1])
# Command line args are in sys.argv[1], sys.argv[2] ..
# sys.argv[0] is the script name itself and can be ignored
# Standard boilerplate to call the main() function to begin
# the program.
if __name__ == '__main__':
main()
You may have been directed here because you were asking about an IndexError in your code that uses sys.argv. The problem is not in your code; the problem is that you need to run the program in a way that makes sys.argv contain the right values. Please read the answers to understand how sys.argv works.
If you have read and understood the answers, and are still having problems on Windows, check if Python Script does not take sys.argv in Windows fixes the issue. If you are trying to run the program from inside an IDE, you may need IDE-specific help - please search, but first check if you can run the program successfully from the command line.
I would like to note that previous answers made many assumptions about the user's knowledge. This answer attempts to answer the question at a more tutorial level.
For every invocation of Python, sys.argv is automatically a list of strings representing the arguments (as separated by spaces) on the command-line. The name comes from the C programming convention in which argv and argc represent the command line arguments.
You'll want to learn more about lists and strings as you're familiarizing yourself with Python, but in the meantime, here are a few things to know.
You can simply create a script that prints the arguments as they're represented. It also prints the number of arguments, using the len function on the list.
from __future__ import print_function
import sys
print(sys.argv, len(sys.argv))
The script requires Python 2.6 or later. If you call this script print_args.py, you can invoke it with different arguments to see what happens.
> python print_args.py
['print_args.py'] 1
> python print_args.py foo and bar
['print_args.py', 'foo', 'and', 'bar'] 4
> python print_args.py "foo and bar"
['print_args.py', 'foo and bar'] 2
> python print_args.py "foo and bar" and baz
['print_args.py', 'foo and bar', 'and', 'baz'] 4
As you can see, the command-line arguments include the script name but not the interpreter name. In this sense, Python treats the script as the executable. If you need to know the name of the executable (python in this case), you can use sys.executable.
You can see from the examples that it is possible to receive arguments that do contain spaces if the user invoked the script with arguments encapsulated in quotes, so what you get is the list of arguments as supplied by the user.
Now in your Python code, you can use this list of strings as input to your program. Since lists are indexed by zero-based integers, you can get the individual items using the list[0] syntax. For example, to get the script name:
script_name = sys.argv[0] # this will always work.
Although interesting, you rarely need to know your script name. To get the first argument after the script for a filename, you could do the following:
filename = sys.argv[1]
This is a very common usage, but note that it will fail with an IndexError if no argument was supplied.
Also, Python lets you reference a slice of a list, so to get another list of just the user-supplied arguments (but without the script name), you can do
user_args = sys.argv[1:] # get everything after the script name
Additionally, Python allows you to assign a sequence of items (including lists) to variable names. So if you expect the user to always supply two arguments, you can assign those arguments (as strings) to two variables:
user_args = sys.argv[1:]
fun, games = user_args # len(user_args) had better be 2
So, to answer your specific question, sys.argv[1] represents the first command-line argument (as a string) supplied to the script in question. It will not prompt for input, but it will fail with an IndexError if no arguments are supplied on the command-line following the script name.
sys.argv[1] contains the first command line argument passed to your script.
For example, if your script is named hello.py and you issue:
$ python3.1 hello.py foo
or:
$ chmod +x hello.py # make script executable
$ ./hello.py foo
Your script will print:
Hello there foo
sys.argv is a list.
This list is created by your command line, it's a list of your command line arguments.
For example:
in your command line you input something like this,
python3.2 file.py something
sys.argv will become a list ['file.py', 'something']
In this case sys.argv[1] = 'something'
Just adding to Frederic's answer, for example if you call your script as follows:
./myscript.py foo bar
sys.argv[0] would be "./myscript.py"
sys.argv[1] would be "foo" and
sys.argv[2] would be "bar" ... and so forth.
In your example code, if you call the script as follows ./myscript.py foo , the script's output will be "Hello there foo".
Adding a few more points to Jason's Answer :
For taking all user provided arguments: user_args = sys.argv[1:]
Consider the sys.argv as a list of strings as (mentioned by Jason). So all the list manipulations will apply here. This is called "List Slicing". For more info visit here.
The syntax is like this: list[start:end:step]. If you omit start, it will default to 0, and if you omit end, it will default to length of list.
Suppose you only want to take all the arguments after 3rd argument, then:
user_args = sys.argv[3:]
Suppose you only want the first two arguments, then:
user_args = sys.argv[0:2] or user_args = sys.argv[:2]
Suppose you want arguments 2 to 4:
user_args = sys.argv[2:4]
Suppose you want the last argument (last argument is always -1, so what is happening here is we start the count from back. So start is last, no end, no step):
user_args = sys.argv[-1]
Suppose you want the second last argument:
user_args = sys.argv[-2]
Suppose you want the last two arguments:
user_args = sys.argv[-2:]
Suppose you want the last two arguments. Here, start is -2, that is second last item and then to the end (denoted by :):
user_args = sys.argv[-2:]
Suppose you want the everything except last two arguments. Here, start is 0 (by default), and end is second last item:
user_args = sys.argv[:-2]
Suppose you want the arguments in reverse order:
user_args = sys.argv[::-1]
sys.argv is a list containing the script path and command line arguments; i.e. sys.argv[0] is the path of the script you're running and all following members are arguments.
To pass arguments to your python script
while running a script via command line
> python create_thumbnail.py test1.jpg test2.jpg
here,
script name - create_thumbnail.py,
argument 1 - test1.jpg,
argument 2 - test2.jpg
With in the create_thumbnail.py script i use
sys.argv[1:]
which give me the list of arguments i passed in command line as
['test1.jpg', 'test2.jpg']
sys.argv is a attribute of the sys module. It says the arguments passed into the file in the command line. sys.argv[0] catches the directory where the file is located. sys.argv[1] returns the first argument passed in the command line. Think like we have a example.py file.
example.py
import sys # Importing the main sys module to catch the arguments
print(sys.argv[1]) # Printing the first argument
Now here in the command prompt when we do this:
python example.py
It will throw a index error at line 2. Cause there is no argument passed yet. You can see the length of the arguments passed by user using if len(sys.argv) >= 1: # Code.
If we run the example.py with passing a argument
python example.py args
It prints:
args
Because it was the first arguement! Let's say we have made it a executable file using PyInstaller. We would do this:
example argumentpassed
It prints:
argumentpassed
It's really helpful when you are making a command in the terminal. First check the length of the arguments. If no arguments passed, do the help text.
sys.argv will display the command line args passed when running a script or you can say sys.argv will store the command line arguments passed in python while running from terminal.
Just try this:
import sys
print sys.argv
argv stores all the arguments passed in a python list. The above will print all arguments passed will running the script.
Now try this running your filename.py like this:
python filename.py example example1
this will print 3 arguments in a list.
sys.argv[0] #is the first argument passed, which is basically the filename.
Similarly, argv[1] is the first argument passed, in this case 'example'.
From the python docs:
The various exec*() functions take a
list of arguments for the new program
loaded into the process. In each case,
the first of these arguments is passed
to the new program as its own name
rather than as an argument a user may
have typed on a command line. For the
C programmer, this is the argv[0]
passed to a program’s main(). For
example, os.execv('/bin/echo', ['foo',
'bar']) will only print bar on
standard output; foo will seem to be
ignored.
Can someone please help me understand this? What do I need to do if I want to run my own program with some parameters?
UNIX, where all these exec things come from, separated the program executable file from the program name, so that your process could have any arbitrary name.
The first argument is the program that will run. This must exist. The next argument is what your process running the program will be called, what will be in argv[0], and what comes up in the ps (process list) output.
So, if I did (in C, but it maps to Python as well):
execl ("/usr/bin/sleep", "notsleep", "60", NULL);
This would run the program /usr/bin/sleep but it would show up in the process list as notsleep. argv[0] would be notsleep and argv[1] (the actual argument) would be 60. Often, the first two parameters will be identical but it's by no means required.
That's why the first argument of your list is (seemingly) ignored. It's the name to give to the process, not the first argument to it.
A more correct way to do it would be:
os.execv('/bin/echo', ['echo', 'foo', 'bar'])