I have the following program to 7-Zip a directory of files via python subprocess:
import subprocess
import os
appPath = r'C:\\Users\Person\Desktop\7-Zip'
zApp = r'7z.exe'
zAction = r'a'
zipFileName = r'C:\\Users\Person\Desktop\test.zip'
zPass = r'password'
zAnswer = '-y'
zDir = r'C:\\Users\Person\Desktop\test'
progDir = os.path.join(appPath, zApp)
cmd = [zApp, zAction, zipFileName, zPass, zAnswer, zDir]
subprocess.Popen(cmd, executable=progDir, stderr=subprocess.STDOUT, stdout=subprocess.PIPE)
However, I keep running into the following error:
FileNotFoundError: [WinError 2] The system cannot find the file specified
I am using Python 3.6 on a windows machine.
I would presume it is because you don't have C:\\Users\Person\Desktop\7-Zip in the PATH environment variable for either the current user or the machine.
In your example you already seem to be aware of this...
Change:
cmd = [zApp, zAction, zipFileName, zPass, zAnswer, zDir]
To:
cmd = [progDir, zAction, zipFileName, zPass, zAnswer, zDir]
Or else edit the PATH environment variable for the user or machine and put the path to the 7zip executable on it so that it can be called via simply 7z
Related
Python 3.10.6
Windows 10
I have a python function that executes a DXL script using subsystem.run() or os.system() (whichever works best I guess). The problem is that when I run a custom command using python it does not work, but when I paste the same command in the command prompt, it works. I should also clarify that command prompt is not the ms store windows terminal (cannot run ibm doors commands there for some reason). It is the OG prompt
I need to use both python and IBM Doors for the solution.
Here is a summer version of my code (Obviously, the access values are not real):
#staticmethod
def run_dxl_importRTF():
dquotes = chr(0x22) # ASCII --> "
module_name = "TEST_TEMP"
script_path = "importRTF.dxl"
script_do_nothing_path = "doNothing.dxl"
user = "user"
password = "pass"
database_config = "11111#11.11.1111.0"
doors_path = dquotes + r"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe" + dquotes
file_name = "LIBC_String.rtf"
# Based On:
# "C:\Program Files\IBM\Rational\DOORS\9.7\\bin\doors.exe" -dxl "string pModuleName = \"%~1\";string pFilename = \"%~2\";#include <importRTF.dxl>" -f "%TEMP%" -b "doNothing.dxl" -d 11111#11.11.1111.0 -user USER -password PASSWORD
script_arguments = f"{dquotes}string pModuleName=\{dquotes}{module_name}\{dquotes};string pFileName=\{dquotes}{file_name}\{dquotes};#include <{script_path}>{dquotes}"
command = [doors_path, "-dxl", script_arguments, "-f", "%TEMP%", "-b", script_do_nothing_path, '-d', database_config, '-user', user, '-password', password]
res = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(f"COMMAND:\n{' '.join(res.args)}")
print(f"STDERR: {repr(res.stderr)}")
print(f'STDOUT: {res.stdout}')
print(f'RETURN CODE: {res.returncode}')
return
PYTHON SCRIPT OUTPUT:
COMMAND:
"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe" -dxl "string pModuleName=\"TEST_TEMP\";string pFileName=\"LIBC_String.rtf\";#include <importRTF.dxl>" -f %TEMP% -b doNothing.dxl -d 11111#11.11.1111.0 -user USER_TEMP -password PASS_TEMP
STDERR: 'The system cannot find the path specified.\n'
STDOUT:
RETURN CODE: 1
When I run the same command in the command prompt, it works (dxl script is compiled).
I identified the problem which is the script_argument variable. Meaning that, when I try to just enter the IBM Doors server without compiling a DXL script, it works on python and the command prompt.
The python script needs to be dynamic meaning that all of the initial declared variables can change value and have a path string in it. I am also trying to avoid .bat files. They also did not work with dynamic path values
Thanks for your time
I tried:
Changing CurrentDirectory (cwd) to IBM Doors
os.system()
Multiple workarounds
Tried IBM Doors path without double quotes (it doesnt work because of the whitespaces)
.bat files
When calling subprocess.run with a command list and shell=True, python will expand the command list to a string, adding more quoting along the way. The details are OS dependent (on Windows, you always have to expand the list to a command) but you can see the result via the subprocess.list2cmdline() function.
Your problem is these extra escapes. Instead of using a list, build a shell command string that already contains the escaping you want. You can also use ' for quoting strings so that internal " needed for shell quoting can be entered literally.
Putting it all together (and likely messing something up here), you would get
#staticmethod
def run_dxl_importRTF():
module_name = "TEST_TEMP"
script_path = "importRTF.dxl"
script_do_nothing_path = "doNothing.dxl"
user = "user"
password = "pass"
database_config = "11111#11.11.1111.0"
doors_path = r"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe"
file_name = "LIBC_String.rtf"
script_arguments = (rf'string pModuleName=\"{module_name}\";'
'string pFileName=\"{file_name}\";'
'#include <{script_path}>')
command = (f'"{doors_path}" -dxl "{script_arguments}" -f "%TEMP%"'
' -b "{script_do_nothing_path}" -d {database_config}'
' -user {user} -password {pass}')
res = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(f"COMMAND:\n{' '.join(res.args)}")
print(f"STDERR: {repr(res.stderr)}")
print(f'STDOUT: {res.stdout}')
print(f'RETURN CODE: {res.returncode}')
I want to execute a script inside a subdirectory/superdirectory (I need to be inside this sub/super-directory first). I can't get subprocess to enter my subdirectory:
tducin#localhost:~/Projekty/tests/ve$ python
Python 2.7.4 (default, Sep 26 2013, 03:20:26)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> import os
>>> os.getcwd()
'/home/tducin/Projekty/tests/ve'
>>> subprocess.call(['cd ..'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Python throws OSError and I don't know why. It doesn't matter whether I try to go into an existing subdir or go one directory up (as above) - I always end up with the same error.
What your code tries to do is call a program named cd ... What you want is call a command named cd.
But cd is a shell internal. So you can only call it as
subprocess.call('cd ..', shell=True) # pointless code! See text below.
But it is pointless to do so. As no process can change another process's working directory (again, at least on a UNIX-like OS, but as well on Windows), this call will have the subshell change its dir and exit immediately.
What you want can be achieved with os.chdir() or with the subprocess named parameter cwd which changes the working directory immediately before executing a subprocess.
For example, to execute ls in the root directory, you either can do
wd = os.getcwd()
os.chdir("/")
subprocess.Popen("ls")
os.chdir(wd)
or simply
subprocess.Popen("ls", cwd="/")
To run your_command as a subprocess in a different directory, pass cwd parameter, as suggested in #wim's answer:
import subprocess
subprocess.check_call(['your_command', 'arg 1', 'arg 2'], cwd=working_dir)
A child process can't change its parent's working directory (normally). Running cd .. in a child shell process using subprocess won't change your parent Python script's working directory i.e., the code example in #glglgl's answer is wrong. cd is a shell builtin (not a separate executable), it can change the directory only in the same process.
subprocess.call and other methods in the subprocess module have a cwd parameter.
This parameter determines the working directory where you want to execute your process.
So you can do something like this:
subprocess.call('ls', shell=True, cwd='path/to/wanted/dir/')
Check out docs subprocess.popen-constructor
You want to use an absolute path to the executable, and use the cwd kwarg of Popen to set the working directory. See the docs.
If cwd is not None, the child’s current directory will be changed to
cwd before it is executed. Note that this directory is not considered
when searching the executable, so you can’t specify the program’s path
relative to cwd.
I guess these days you would do:
import subprocess
subprocess.run(["pwd"], cwd="sub-dir")
Another option based on this answer: https://stackoverflow.com/a/29269316/451710
This allows you to execute multiple commands (e.g cd) in the same process.
import subprocess
commands = '''
pwd
cd some-directory
pwd
cd another-directory
pwd
'''
process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = process.communicate(commands.encode('utf-8'))
print(out.decode('utf-8'))
just use os.chdir
Example:
>>> import os
>>> import subprocess
>>> # Lets Just Say WE want To List The User Folders
>>> os.chdir("/home/")
>>> subprocess.run("ls")
user1 user2 user3 user4
If you want to have cd functionality (assuming shell=True) and still want to change the directory in terms of the Python script, this code will allow 'cd' commands to work.
import subprocess
import os
def cd(cmd):
#cmd is expected to be something like "cd [place]"
cmd = cmd + " && pwd" # add the pwd command to run after, this will get our directory after running cd
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) # run our new command
out = p.stdout.read()
err = p.stderr.read()
# read our output
if out != "":
print(out)
os.chdir(out[0:len(out) - 1]) # if we did get a directory, go to there while ignoring the newline
if err != "":
print(err) # if that directory doesn't exist, bash/sh/whatever env will complain for us, so we can just use that
return
If you need to change directory, run a command and get the std output as well:
import os
import logging as log
from subprocess import check_output, CalledProcessError, STDOUT
log.basicConfig(level=log.DEBUG)
def cmd_std_output(cd_dir_path, cmd):
cmd_to_list = cmd.split(" ")
try:
if cd_dir_path:
os.chdir(os.path.abspath(cd_dir_path))
output = check_output(cmd_to_list, stderr=STDOUT).decode()
return output
except CalledProcessError as e:
log.error('e: {}'.format(e))
def get_last_commit_cc_cluster():
cd_dir_path = "/repos/cc_manager/cc_cluster"
cmd = "git log --name-status HEAD^..HEAD --date=iso"
result = cmd_std_output(cd_dir_path, cmd)
return result
log.debug("Output: {}".format(get_last_commit_cc_cluster()))
Output: "commit 3b3daaaaaaaa2bb0fc4f1953af149fa3921e\nAuthor: user1<user1#email.com>\nDate: 2020-04-23 09:58:49 +0200\n\n
I want to execute a script inside a subdirectory/superdirectory (I need to be inside this sub/super-directory first). I can't get subprocess to enter my subdirectory:
tducin#localhost:~/Projekty/tests/ve$ python
Python 2.7.4 (default, Sep 26 2013, 03:20:26)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> import os
>>> os.getcwd()
'/home/tducin/Projekty/tests/ve'
>>> subprocess.call(['cd ..'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Python throws OSError and I don't know why. It doesn't matter whether I try to go into an existing subdir or go one directory up (as above) - I always end up with the same error.
What your code tries to do is call a program named cd ... What you want is call a command named cd.
But cd is a shell internal. So you can only call it as
subprocess.call('cd ..', shell=True) # pointless code! See text below.
But it is pointless to do so. As no process can change another process's working directory (again, at least on a UNIX-like OS, but as well on Windows), this call will have the subshell change its dir and exit immediately.
What you want can be achieved with os.chdir() or with the subprocess named parameter cwd which changes the working directory immediately before executing a subprocess.
For example, to execute ls in the root directory, you either can do
wd = os.getcwd()
os.chdir("/")
subprocess.Popen("ls")
os.chdir(wd)
or simply
subprocess.Popen("ls", cwd="/")
To run your_command as a subprocess in a different directory, pass cwd parameter, as suggested in #wim's answer:
import subprocess
subprocess.check_call(['your_command', 'arg 1', 'arg 2'], cwd=working_dir)
A child process can't change its parent's working directory (normally). Running cd .. in a child shell process using subprocess won't change your parent Python script's working directory i.e., the code example in #glglgl's answer is wrong. cd is a shell builtin (not a separate executable), it can change the directory only in the same process.
subprocess.call and other methods in the subprocess module have a cwd parameter.
This parameter determines the working directory where you want to execute your process.
So you can do something like this:
subprocess.call('ls', shell=True, cwd='path/to/wanted/dir/')
Check out docs subprocess.popen-constructor
You want to use an absolute path to the executable, and use the cwd kwarg of Popen to set the working directory. See the docs.
If cwd is not None, the child’s current directory will be changed to
cwd before it is executed. Note that this directory is not considered
when searching the executable, so you can’t specify the program’s path
relative to cwd.
I guess these days you would do:
import subprocess
subprocess.run(["pwd"], cwd="sub-dir")
Another option based on this answer: https://stackoverflow.com/a/29269316/451710
This allows you to execute multiple commands (e.g cd) in the same process.
import subprocess
commands = '''
pwd
cd some-directory
pwd
cd another-directory
pwd
'''
process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = process.communicate(commands.encode('utf-8'))
print(out.decode('utf-8'))
just use os.chdir
Example:
>>> import os
>>> import subprocess
>>> # Lets Just Say WE want To List The User Folders
>>> os.chdir("/home/")
>>> subprocess.run("ls")
user1 user2 user3 user4
If you want to have cd functionality (assuming shell=True) and still want to change the directory in terms of the Python script, this code will allow 'cd' commands to work.
import subprocess
import os
def cd(cmd):
#cmd is expected to be something like "cd [place]"
cmd = cmd + " && pwd" # add the pwd command to run after, this will get our directory after running cd
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) # run our new command
out = p.stdout.read()
err = p.stderr.read()
# read our output
if out != "":
print(out)
os.chdir(out[0:len(out) - 1]) # if we did get a directory, go to there while ignoring the newline
if err != "":
print(err) # if that directory doesn't exist, bash/sh/whatever env will complain for us, so we can just use that
return
If you need to change directory, run a command and get the std output as well:
import os
import logging as log
from subprocess import check_output, CalledProcessError, STDOUT
log.basicConfig(level=log.DEBUG)
def cmd_std_output(cd_dir_path, cmd):
cmd_to_list = cmd.split(" ")
try:
if cd_dir_path:
os.chdir(os.path.abspath(cd_dir_path))
output = check_output(cmd_to_list, stderr=STDOUT).decode()
return output
except CalledProcessError as e:
log.error('e: {}'.format(e))
def get_last_commit_cc_cluster():
cd_dir_path = "/repos/cc_manager/cc_cluster"
cmd = "git log --name-status HEAD^..HEAD --date=iso"
result = cmd_std_output(cd_dir_path, cmd)
return result
log.debug("Output: {}".format(get_last_commit_cc_cluster()))
Output: "commit 3b3daaaaaaaa2bb0fc4f1953af149fa3921e\nAuthor: user1<user1#email.com>\nDate: 2020-04-23 09:58:49 +0200\n\n
I made a small script in sublime that will extract commands from a json file that is on the user's computer and then it will open the terminal and run the settings/command. This works, except that it doesn't really open up the terminal. It only runs the command (and it works, as in my case it will run gcc to compile a simple C file), and pipes to STDOUT without opening up the terminal.
import json
import subprocess
import sublime_plugin
class CompilerCommand(sublime_plugin.TextCommand):
def get_dir(self, fullpath):
path = fullpath.split("\\")
path.pop()
path = "\\".join(path)
return path
def get_settings(self, path):
_settings_path = path + "\\compiler_settings.json"
return json.loads(open(_settings_path).read())
def run(self, edit):
_path = self.get_dir(self.view.file_name())
_settings = self.get_settings(_path)
_driver = _path.split("\\")[0]
_command = _driver + " && cd " + _path + " && " + _settings["compile"] + " && " + _settings["exec"]
proc = subprocess.Popen(_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
I'm not sure if using subprocess.Popen is the right way to go about it as I'm new to Python.
So to re-iterate; I want it to open up the terminal, run the command, and have the terminal stay open until the user presses ENTER or something. I'm running Windows 7 and Python 3, if that matters.
subprocess.Popen simply creates a subprocess with the given command. It is in no way related to opening a terminal window or any other windows for that matter.
You'll have to look into your platform specific UI automation solutions in order to achieve what you want. Or see if maybe the Sublime plugins mechanism can already do that.
NOTES:
Also, you should be using os.path.join/os.path.split/os.path.sep etc for your path operations—Sublime also runs on OS X for example, and OS X does not use backslashes. Also, file handles need to be closed, so use:
with open(...) as f:
return json.load(f) # also not that there is no nead to f.read()+json.loads()
# if you can just json.load() on the file handle
Furthermore, strings should usually be built using string interpolation:
_command = "{} && cd {} && {} && {}".format(_driver, _path, _settings["compile"], _settings["exec"])
...and, you should not be prefixing your local variables with _—it doesn't look nice and serves no purpose in Python either; and while we're at it, I might as well use the chance to recommend you to read PEP8: http://www.python.org/dev/peps/pep-0008/.
I would like to know how to execute this java process using the windows command line, from inside Python 2.7 on Windows 8.
I thought I had already solved this problem, but I recently changed computers from Windows 7 to Windows 8 and my code stopped working. I have confirmed that the windows command used in the script below executes properly when run directly from cmd.exe
import os
import subprocess
def FileProcess(inFile):
#Create the startup info so the java program runs in the background (for windows computers)
startupinfo = None
if os.name == 'nt':
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
#Execute Stanford Core NLP from the command line
print inFile
cmd = ['java', '-Xmx1g','-cp', 'stanford-corenlp-1.3.5.jar;stanford-corenlp-1.3.5-models.jar;xom.jar;joda-time.jar', 'edu.stanford.nlp.pipeline.StanfordCoreNLP', '-annotators', 'tokenize,ssplit,pos,parse', '-file', inFile]
output = subprocess.call(cmd, startupinfo=startupinfo)
print inFile[(str(inFile).rfind('\\'))+1:] + '.xml'
outFile = file(inFile[(str(inFile).rfind('\\'))+1:] + '.xml')
FileProcess("C:\\NSF_Stuff\\ErrorPropagationPaper\\RandomTuftsPlain\\PreprocessedTufts8199PLAIN.txt")
When this code is executed, I receive the error message that the output file does not exist. The java process I am executing should output an xml file when it is done.
It is my belief that for some reason subprocess.call is never successfully executing the command. I have tried using subprocesss.popen for the same task and I get the same results.
EDIT: I have changed my code so that I can capture error messages and I think I am beginning to understand the problem.
I changed my code to
import os
import subprocess
def FileProcess(inFile):
#Create the startup info so the java program runs in the background (for windows computers)
startupinfo = None
if os.name == 'nt':
startupinfo = subprocess.STARTUPINFO()
startupinfo.dwFlags |= subprocess.STARTF_USESHOWWINDOW
#Execute Stanford Core NLP from the command line
print inFile
cmd = ['java', '-Xmx1g','-cp', 'stanford-corenlp-1.3.5.jar;stanford-corenlp-1.3.5-models.jar;xom.jar;joda-time.jar', 'edu.stanford.nlp.pipeline.StanfordCoreNLP', '-annotators', 'tokenize,ssplit,pos,parse', '-file', inFile]
proc = subprocess.Popen(cmd, stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=True)
print proc
stdoutdata, stderrdata = proc.communicate()
print stdoutdata
print stderrdata
outFile = file(inFile[(str(inFile).rfind('\\'))+1:] + '.xml')
FileProcess("C:\\NSF_Stuff\\ErrorPropagationPaper\\RandomTuftsPlain\\PreprocessedTufts8199PLAIN.txt")
stdoutdata contains the message "'java' is not recognized as an internal or external command, operable program or batch file."
Now this is a very bizarre message because java is definitely a recognized command when I run it from the cmd.exe . There is some issue here where executing the command from python is messing with my system environment variables such that java is no longer recognized as a command.
I was able to solve my problem by adding the location of java to my PATH variable. Apparently java wasn't in my path variable. I didn't even bother checking this originally because I was having no problems executing java commands from the windows command line. I'm guessing that commands executed directly from cmd.exe use a different environment variable to find the java executable than commands executed indirectly from the subprocess module.
By trying your code it prints out PreprocessedTufts8199PLAIN.txt.xml file name. I'm not sure if the .txt.xml extension was the desired result. If your file has only .xml extension, then you're not stripping away the original .txt header.
Try to change this line:
outFile = file(inFile[(str(inFile).rfind('\\'))+1:] + '.xml')
Into this code:
fnameext = inFile[(str(inFile).rfind('\\'))+1:]
fname,fext = os.path.splitext(fnameext)
xmlfname = fname + '.xml'
xmlfpath = os.path.join(".", xmlfname)
print "xmlfname:", xmlfname, " xmlfpath:", xmlfpath
print "current working directory:", os.getcwd()
outFile = open(xmlfpath, "r")
Answer for extension stripping.