I want to call a subprocess in python using subprocess.call(), with the 'cwd' argument so that this particular subprocess is executed in a different directory. I don't want to use os.chdir() because for future processes later in the program I want to remain in the original directory from where the program was run.
BUT, I also want to run this particular subprocess on a set of files matching a glob pattern. So for example, I might want to do
subprocess.call(['ls'] + glob('*.txt'), cwd="/my/other/dir/")
But of course the glob command doesn't know to look in /my/other/dir, so it fails. How can I do this without using shell=True?
You could use the CWD in the glob pattern as well. Like glob.glob("/my/other/dir/*.txt"). It will expand with full match, like /my/other/dir/aa.txt. In case you do not want to pass the full path to the executable, cut it off.
CWD = "/my/other/dir/"
files = map(lambda x: x[len(CWD):], glob.glob(CWD + "*.txt"))
subprocess.call(['ls'] + files, cwd=CWD)
Or you could just change the directory back after the subprocess has finished.
Related
I am making a body tracking application where I want to run Open Pose if the user chooses to track their body movements. The OpenPose binary file can be run like so:
bin\OpenPoseDemo.exe --write_json 'path\to\dump\output'
So, in my Python script, I want to have a line of code that would run Open Pose, instead of having to ask the user to manually run OpenPose by opening a separate command line window. For that, I have tried:
import os
os.popen(r"C:\path\to\bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'")
But this gives the following error:
Error:
Could not create directory: 'C:\Users\Admin\Documents\Openpose\. Status error = -1. Does the parent folder exist and/or do you have writing access to that path?
Which I guess means that OpenPose can be opened only by going inside the openpose directory where the bin subdirectory resides. So, I wrote a shell script containing this line:
bin\OpenPoseDemo.exe --write_json 'C:\path\to\dump\output'
and saved it as run_openpose_binary.sh in the openpose directory (i.e., the same directory where bin is located).
I then tried to run this shell script from within my Python script like so:
import subprocess
subprocess.call(['sh', r'C:\path\to\openpose\run_openpose_binary.sh'])
and this gives the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I also tried the following:
os.popen(r"C:\path\to\openpose\run_openpose_binary.sh")
and
os.system(r"C:\path\to\openpose\run_openpose_binary.sh")
These do not produce any error, but instead just pop up a blank window and closes.
So, my question is, how do I run the OpenPoseDemo.exe from within my Python script?
For your last method, you're missing the return value from os.popen, which is a pipe. So, what you need is something like:
# untested as I don't have access to a Windows system
import os
with os.popen(r"/full/path/to/sh C:/path/to/openpose/run_openpose_binary.sh") as p:
# pipes work like files
output_of_command = p.read().strip() # this is a string
or, if you want to future-proof yourself, the alternative is:
# untested as I don't have access to a Windows system
popen = subprocess.Popen([r'/full/path/to/sh.exe', r'/full/path/to/run_openpose_binary.sh')], stdin=subprocess.PIPE, stdout=subprocess.PIPE,encoding='utf-8')
stdout, stderr = popen.communicate(input='')
Leave a comment if you have further difficulty.
I've had to fight this battle several times and I've found a solution. It's likely not the most elegant solution but it does work, and I'll explain it using an example of how to run OpenPose on a video.
You've got your path to the openpose download and your path to the video, and from there it's a 3-line solution. First, change the current working directory to that openpose folder, and then create your command, then call subprocess.run (I tried using subprocess.call and that did not work. I did not try shell=False but I have heard it's a safer way to do so. I'll leave that up to you.)
import os
import subprocess
openpose_path = "C:\\Users\\me\\Desktop\\openpose-1.7.0-binaries-win64-gpu-python3.7-flir-3d_recommended\\openpose\\"
video_path = "C:\\Users\\me\\Desktop\\myvideo.mp4"
os.chdir(openpose_path)
command = "".join(["bin\\OpenPoseDemo.exe", " -video ", video_path])
subprocess.run(command, shell=True)
How can i get the current directory to which I am in? like the use of
os.getcwd()
First, I presume you're not asking about a particular subprocess that exists simply to tell you the current working directory and do nothing else (Apducer's answer). If that were the case you could simply as os.getcwd() and forget the subprocess. You clearly already know that. So you must be dealing with some other (arbitrary?) subprocess.
Second, I presume you understand, via dr1fter's answer, that you have control over the working directory in which the subprocess starts. I suspect that's not enough for you.
Rather, I suspect you're thinking that the subprocess might, according to its own internal logic, have changed its working directory sometime since its launch, that you can't predict where it has ended up, and you want to be able to send some sort of signal to the subprocess at an arbitrary time, to interrogate it about where it's currently working. In general, this is only possible if the process has been specifically programmed with the logic that receives such a signal (through whatever route) and issues such a response. I think that's what SuperStew meant by the comment, "isn't that going to depend on the subprocess?"
I say "in general" because there are platform-specific approaches. For example, see:
windows batch command to determine working directory of a process
How do I print the current working directory of another user in linux?
by default, subprocesses you spawn inherit your PWD. you can however, specify the cwd argument to the subprocess.Popen c'tor to set a different initial PWD.
Unix (Linux, MacOS):
import subprocess
arguments = ['pwd']
directory = subprocess.check_output(arguments)
Windows:
import subprocess
arguments = ['cd']
directory = subprocess.check_output(arguments)
If you want to run in both types of OS, you'll have to check the machine OS:
import os
import subprocess
if os.name == 'nt': # Windows
arguments = ['cd']
else: # other (unix)
arguments = ['pwd']
directory = subprocess.check_output(arguments)
I have read way to many threads now and really lost.
Just trying to do something basic before I make it complicated.
so i have a script test.py
I want to call the script from within runme.py but without waiting so it will process the other chunk of code, but then when it gets to the end wait for test.py code to finish before continuing on.
I cant seem to figure out the correct syntax for the p = subprocess.Popen (I have tried so many)
and do I need the that to the test.py if its in the same directory?
here is what i have but cant get to work.
import subprocess
p = subprocess.Popen(['python test.py'])
#do some code
p.wait()
I cant seem to figure out the correct syntax for the p = subprocess.Popen (I have tried so many)
You want to pass it a list of arguments. The first argument is the program to run, python (although actually, you probably want sys.executable here); the second is the script that you want python to run. So:
p = subprocess.Popen(['python', 'test.py'])
and do I need the that to the test.py if its in the same directory?
This will work the same way as if you ran python test.py at the shell: it will just pass test.py as-is to python, and python will treat that as a path relative to the current working directory (CWD).
So, if test.py is in the CWD, this will just work.
If test.py is somewhere else, then you need to provide either an absolute path, or one relative to the CWD.
One common thing you want is that test.py is in not necessarily in the CWD, but instead it's in the same directory as the script/module that wants to launch it:
scriptpath = os.path.join(os.path.dirname(__file__), 'test.py')
… or in the same directory as the main script used to start your program:1
scriptpath = os.path.join(os.path.dirname(sys.argv[0]), 'test.py')
Either way, you just pass that as the argument:
p = subprocess.Popen(['python', scriptpath])
1. On some platforms, this may actually be a relative path. If you might have done an os.chdir since startup, it will now be wrong. If you need to handle that, you want to stash os.path.abspath(os.path.dirname(sys.argv[0])) in the main script at startup, then pass it down to other functions for them to use instead of calling dirname(argv[0]) themselves.
Idea
Basically, what my script does is checking C:/SOURCE for .txt files and add a timestamp to it. To replicate it you can basically make that folder and put some txt files in there. Then, it's supposed to run a .vbs file, which then runs a .bat files with some rclone commands which don't matter here. I did it like this because there wont be a CMD window opening when running the rclone command through the .vbs file.
Python code
import time, os, subprocess
while True:
print("Beginning checkup")
print("=================")
timestamp = time.strftime('%d_%m_%H_%M') # only underscores: no naming issues
the_dir = "C:/SOURCE"
for fname in os.listdir(the_dir):
if fname.lower().endswith(".txt"):
print("found " + fname)
time.sleep(0.1)
new_name = "{}-{}.txt".format(os.path.splitext(fname)[0], timestamp)
os.rename(os.path.join(the_dir, fname), os.path.join(the_dir, new_name))
time.sleep(0.5)
else:
subprocess.call(['cscript.exe', "copy.vbs"])
time.sleep(60)
VBScript code
Set WshShell = CreateObject("WScript.Shell" )
WshShell.Run Chr(34) & "copy.bat" & Chr(34), 0
Set WshShell = Nothing
The only important part for the Python script is below the very last else, where the subprocess.call() is supposed to run the .vbs file. What happens when running the script is it shows the first two lines that always come up when running CMD, but then nothing.
How could I fix that? I tried:
subprocess.call("cscript copy.vbs")
subprocess.call("cmd /c copy.vbs")
both with the same outcome, it doesn't do anything.
Anyone have an idea?
Why are you invoking a VBScript to invoke a batch script from Python? You should be able to simple run whatever the batch script is doing directly from your Python code. But even if you wanted to keep the batch script, something like this should do just fine without VBScript as an intermediary.
subprocess.call(['cmd', '/c', 'copy.bat'])
You may want to give the full path of the batch file, though, to avoid issues like the working directory not being what you think it is.
If your batch script resides in the same directory as the Python script, you can build the path with something like this:
import os
import subprocess
scriptdir = os.path.dirname(__file__)
batchfile = os.path.join(scriptdir, 'copy.bat')
subprocess.call(['cmd', '/c', os.path.realpath(batchfile)])
It seems there is no such an operation that could not be done using plain Python. Scan a directory, copy a file -- Python has it all in the standard library. See os.path and shutil modules.
Adding VB scripts and launching subprocesses make your code complex and difficult to debug.
I am trying to delete a file, then rename a file to the deleted file in python.
import sys
import subprocess
fileInput = sys.argv[1]
|
|
#code to create fileInput.tmp
|
|
ret=subprocess.Popen("rm "+fileInput,shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
print ret
ret1=subprocess.Popen("mv "+ fileInput+".tmp "+fileInput,shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
print ret1
Whats happening is sometimes (not always) both fileInput and fileInput.tmp are being deleted in "ret=" step and "ret1=" step doesn't execute.
Can someone suggest why is it happeing. This code is run on MacOSx
Ok, not answering the question directly but providing a better alternative.
Using subprocess here is a very bad idea. It is very easy here to end up doing horrible things; you'll at least need to escape the arguments passed to shell.
Instead, use what Python gives you:
os.rename(fileInput + '.tmp', fileInput)
This is the atomic move/rename operation. And you don't have to rm file before replacing it. Moreover, you usually don't do that because between the rm and mv calls there will be no file with that name. If you just use mv, the replacement will be atomic and some file will always be there.
That's all the short story, I think.
As a side note, os.rename() doesn't work across filesystems. So while using it for renaming a file is ok (the source and destination are in the same directory), if you're moving files across different directories you'd rather use:
import shutil
shutil.move(src, dst)
The first subprocess is not completing, do this:
p = subprocess.Popen("...")
p.communicate() # waits until the subprocess completes