Develop Reference

How to pass command-line arguments to a python script from within another script?

I've got a python script that is named code.py and this takes two command line arguments --par_value xx --out_dir /path/to/dir where par_value is a certain parameter and out_dir is the output directory. I want to run this script for different values of the input parameter.
So I want to use the array:
par_vector = np.array([1,2,3])
and have the output file for each named, say, "output_1", "output_2" and "output_3". So I'm trying to write a python script where I can create that array above and an array of output strings, say, called output_strings so I can automatically run something along the like:
for i,j in zip(par_vector, output_strings):
python code.py --par_value i --out_dir j
But this does not run so I'm unable to figure out how to make such an automation of my code work rather than repeatedly calling the script code.py from the terminal. Any advice is welcome!

It might be a bit convoluted, but a way you could do it is to generate either .bat or .sh files (depending on your operating system) and call them with Popen from the subprocess library.
from subprocess import Popen, PIPE
for i,j in zip(par_vector, output_strings):
cmd = """python code.py --par_value {} --out_dir {}""".format(i,j)
temp_filename = "{}{}.bat".format(i,) #swap .bat for .sh if not on Windows
with open(temp_filename, 'wb') as out_file:
out_file.write(cmd)
execution = Popen([temp_filename], stdout=PIPE, stderr=PIPE)
results = execution.communicate()
# add conditions based on the output generated by results if you want to use it to verify, etc.
os.remove(temp_filename)
Obviously this needs to be changed to match your exact needs for the file name and location, etc..

I would use os.system for this if you're OK with the fact that os.system() is blocking. This would look something like:
import os
for i,j in zip(par_vector, output_strings):
os.system(f"python code.py --par_value {i} --out_dir {j}")

How to run Open Pose binary (.exe) from within a Python script?

I am making a body tracking application where I want to run Open Pose if the user chooses to track their body movements. The OpenPose binary file can be run like so:
bin\OpenPoseDemo.exe --write_json 'path\to\dump\output'
So, in my Python script, I want to have a line of code that would run Open Pose, instead of having to ask the user to manually run OpenPose by opening a separate command line window. For that, I have tried:
import os
os.popen(r"C:\path\to\bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'")
But this gives the following error:
Error:
Could not create directory: 'C:\Users\Admin\Documents\Openpose\. Status error = -1. Does the parent folder exist and/or do you have writing access to that path?
Which I guess means that OpenPose can be opened only by going inside the openpose directory where the bin subdirectory resides. So, I wrote a shell script containing this line:
bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'
and saved it as run_openpose_binary.sh in the openpose directory (i.e., the same directory where bin is located).
I then tried to run this shell script from within my Python script like so:
import subprocess
subprocess.call(['sh', r'C:\path\to\openpose\run_openpose_binary.sh'])
and this gives the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I also tried the following:
os.popen(r"C:\path\to\openpose\run_openpose_binary.sh")
and
os.system(r"C:\path\to\openpose\run_openpose_binary.sh")
These do not produce any error, but instead just pop up a blank window and closes.
So, my question is, how do I run the OpenPoseDemo.exe from within my Python script?

For your last method, you're missing the return value from os.popen, which is a pipe. So, what you need is something like:
# untested as I don't have access to a Windows system
import os
with os.popen(r"/full/path/to/sh C:/path/to/openpose/run_openpose_binary.sh") as p:
# pipes work like files
output_of_command = p.read().strip() # this is a string
or, if you want to future-proof yourself, the alternative is:
# untested as I don't have access to a Windows system
popen = subprocess.Popen([r'/full/path/to/sh.exe', r'/full/path/to/run_openpose_binary.sh')], stdin=subprocess.PIPE, stdout=subprocess.PIPE,encoding='utf-8')
stdout, stderr = popen.communicate(input='')
Leave a comment if you have further difficulty.

I've had to fight this battle several times and I've found a solution. It's likely not the most elegant solution but it does work, and I'll explain it using an example of how to run OpenPose on a video.
You've got your path to the openpose download and your path to the video, and from there it's a 3-line solution. First, change the current working directory to that openpose folder, and then create your command, then call subprocess.run (I tried using subprocess.call and that did not work. I did not try shell=False but I have heard it's a safer way to do so. I'll leave that up to you.)
import os
import subprocess
openpose_path = "C:\\Users\\me\\Desktop\\openpose-1.7.0-binaries-win64-gpu-python3.7-flir-3d_recommended\\openpose\\"
video_path = "C:\\Users\\me\\Desktop\\myvideo.mp4"
os.chdir(openpose_path)
command = "".join(["bin\\OpenPoseDemo.exe", " -video ", video_path])
subprocess.run(command, shell=True)

Trying to figure out how to save output from a command to a file path

I can run commands in prompt from a python script but I want to save the output to a specific file. This script will pull a directory listing:
import subprocess
subprocess.call('dir', shell=True)
and write to a file where my python program was ran:
import sys
sys.stdout = open('badfile', 'w')
print('test')
However I'm trying to combine the two so that the results from the Dir command will be appended to the file "badfile" in a specified location such as C:\users\idiot\somerandomdirectory and not just the default location.

Append output to a file
Q. I'm trying to combine the two so that the results from the Dir command will be appended to the file "badfile"
A. The Python answer is below this one ...
The Command Line Way ...
This can be done in python, but there are probably easier ways to achieve what you want. Here is one to consider:
The usual method is to use the Command Prompt or shell to do these things. You could still have a python script doing things, but then you run it from the command prompt and send the output somewhere else. The easiest way, the way that was designed for this exact case, is to use 'redirection.' You 'catch' the output from the program or script and send it somewhere else. This process is called 'redirection.'
This is how to do it for your example:
C:\myfolder> dir *.* >> C:\some\random\directory\badfile.txt
If you wanted to erase the file before sending your text, you would use a single > symbol instead of the >> notation.
Redirection
The general syntax for your example is:
COMMAND >> FILENAME (append COMMAND output to the end of FILENAME)
The >> symbol between the COMMAND and FILENAME is a 'redirection operator.'
A redirection operator is a special character that can be used with a command, like a Command Prompt command or DOS command, to either redirect the input to the command or the output from the command.
By default, when you execute a command, the input comes from the keyboard and the output is sent to the Command Prompt window. Command inputs and outputs are called command handles.
Here are some examples:
command > filename Redirect command output to a file
command >> filename APPEND into a file
command < filename Type a text file and pass the text to command
commandA | commandB Pipe the output from commandA into commandB
commandA & commandB Run commandA and then run commandB
commandA && commandB Run commandA, if it succeeds then run commandB
commandA || commandB Run commandA, if it fails then run commandB
I mostly use macOS nowadays, but the ideas are similar.
Here is a cheatsheet for Windows.
Here is a cheatsheet Linux and macOS.
The Pythonic Way
As for python, do this:
import subprocess
with open('C:/temp/badfile.txt', mode='at',) as f:
f.write(subprocess.check_output(['dir','*.*']).decode())
There. Done. Python really is great.
To have a program that takes in any command line arguments and writes the results to sp.log, like this:
sp dir *.* /w
create a python script called sp like this:
#!/usr/bin/env python3
# -*- coding: UTF-8 -*-
from subprocess import check_output
from sys import argv
if len(argv) > 1:
with open('sp.log', mode='at',) as f:
try:
f.write(check_output(argv[1:]).decode())
except Exception as e:
print(e)
There are a lot of other things you could add, like checking default encoding, making it work with windows/macOS/linux, adding error checking, adding debugging information, adding command line options ...
Here is a GIST of a longer and more detailed version that I threw together to play with:
https://gist.github.com/skeptycal

python subprocess call failing while same command line call works fine

I am trying to replace a command line call by a Python script using subprocess:
path_to_executable = r'c:\UK\app\Debug\lll.exe'
x = subprocess.call([path_to_executable, args])
args is a string that looks like this:
-unemp Base -rate Base -scen_name Base -print_progress 0 -rate_date 1 -hpa Base -dealpath C:\data\ -nthread 4 -deallist C:\config\all.txt -outdir c:\outdir\Base
The call is working when run from the command line, however failing with the same arguments in subprocess with the following error:
FileIOException(Unable to open directory C:/.../hist.csv)
(The csv file is present - but it's a file, not a directory.)
My questions:
1. How could it be that it work through the command line but not subprocess?
2. Why might it be trying to open a csv file as a directory, when it's not doing the same thing on the command line?

Maybe subprocess is not able to locate the file/directory..Are you sure the file is present and the path to the file does not contain any special character (e.g. ~/) ?
Otherwise try using argument shell=True
from the subprocess doc:
subprocess.call(args, *, stdin=None, stdout=None, stderr=None, shell=False)

Python stdin filename

I'm trying to get the filename thats given in the command line. For example:
python3 ritwc.py < DarkAndStormyNight.txt
I'm trying to get DarkAndStormyNight.txt
When I try fileinput.filename() I get back same with sys.stdin. Is this possible? I'm not looking for sys.argv[0] which returns the current script name.
Thanks!

In general it is not possible to obtain the filename in a platform-agnostic way. The other answers cover sensible alternatives like passing the name on the command-line.
On Linux, and some related systems, you can obtain the name of the file through the following trick:
import os
print(os.readlink('/proc/self/fd/0'))
/proc/ is a special filesystem on Linux that gives information about processes on the machine. self means the current running process (the one that opens the file). fd is a directory containing symbolic links for each open file descriptor in the process. 0 is the file descriptor number for stdin.

You can use ArgumentParser, which automattically gives you interface with commandline arguments, and even provides help, etc
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument('fname', metavar='FILE', help='file to process')
args = parser.parse_args()
with open(args.fname) as f:
#do stuff with f
Now you call python2 ritwc.py DarkAndStormyNight.txt. If you call python3 ritwc.py with no argument, it'll give an error saying it expected argument for FILE. You can also now call python3 ritwc.py -h and it will explain that a file to process is required.
PS here's a great intro in how to use it: http://docs.python.org/3.3/howto/argparse.html

In fact, as it seams that python cannot see that filename when the stdin is redirected from the console, you have an alternative:
Call your program like this:
python3 ritwc.py -i your_file.txt
and then add the following code to redirect the stdin from inside python, so that you have access to the filename through the variable "filename_in":
import sys
flag=0
for arg in sys.argv:
if flag:
filename_in = arg
break
if arg=="-i":
flag=1
sys.stdin = open(filename_in, 'r')
#the rest of your code...
If now you use the command:
print(sys.stdin.name)
you get your filename; however, when you do the same print command after redirecting stdin from the console you would got the result: <stdin>, which shall be an evidence that python can't see the filename in that way.

I don't think it's possible. As far as your python script is concerned it's writing to stdout. The fact that you are capturing what is written to stdout and writing it to file in your shell has nothing to do with the python script.