Develop Reference

Command works on Command Prompt but it does not work when called with subprocess.run() or os.system() in python

Python 3.10.6
Windows 10
I have a python function that executes a DXL script using subsystem.run() or os.system() (whichever works best I guess). The problem is that when I run a custom command using python it does not work, but when I paste the same command in the command prompt, it works. I should also clarify that command prompt is not the ms store windows terminal (cannot run ibm doors commands there for some reason). It is the OG prompt
I need to use both python and IBM Doors for the solution.
Here is a summer version of my code (Obviously, the access values are not real):
#staticmethod
def run_dxl_importRTF():
dquotes = chr(0x22) # ASCII --> "
module_name = "TEST_TEMP"
script_path = "importRTF.dxl"
script_do_nothing_path = "doNothing.dxl"
user = "user"
password = "pass"
database_config = "11111#11.11.1111.0"
doors_path = dquotes + r"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe" + dquotes
file_name = "LIBC_String.rtf"
# Based On:
# "C:\Program Files\IBM\Rational\DOORS\9.7\\bin\doors.exe" -dxl "string pModuleName = \"%~1\";string pFilename = \"%~2\";#include <importRTF.dxl>" -f "%TEMP%" -b "doNothing.dxl" -d 11111#11.11.1111.0 -user USER -password PASSWORD
script_arguments = f"{dquotes}string pModuleName=\{dquotes}{module_name}\{dquotes};string pFileName=\{dquotes}{file_name}\{dquotes};#include <{script_path}>{dquotes}"
command = [doors_path, "-dxl", script_arguments, "-f", "%TEMP%", "-b", script_do_nothing_path, '-d', database_config, '-user', user, '-password', password]
res = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(f"COMMAND:\n{' '.join(res.args)}")
print(f"STDERR: {repr(res.stderr)}")
print(f'STDOUT: {res.stdout}')
print(f'RETURN CODE: {res.returncode}')
return
PYTHON SCRIPT OUTPUT:
COMMAND:
"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe" -dxl "string pModuleName=\"TEST_TEMP\";string pFileName=\"LIBC_String.rtf\";#include <importRTF.dxl>" -f %TEMP% -b doNothing.dxl -d 11111#11.11.1111.0 -user USER_TEMP -password PASS_TEMP
STDERR: 'The system cannot find the path specified.\n'
STDOUT:
RETURN CODE: 1
When I run the same command in the command prompt, it works (dxl script is compiled).
I identified the problem which is the script_argument variable. Meaning that, when I try to just enter the IBM Doors server without compiling a DXL script, it works on python and the command prompt.
The python script needs to be dynamic meaning that all of the initial declared variables can change value and have a path string in it. I am also trying to avoid .bat files. They also did not work with dynamic path values
Thanks for your time
I tried:
Changing CurrentDirectory (cwd) to IBM Doors
os.system()
Multiple workarounds
Tried IBM Doors path without double quotes (it doesnt work because of the whitespaces)
.bat files

When calling subprocess.run with a command list and shell=True, python will expand the command list to a string, adding more quoting along the way. The details are OS dependent (on Windows, you always have to expand the list to a command) but you can see the result via the subprocess.list2cmdline() function.
Your problem is these extra escapes. Instead of using a list, build a shell command string that already contains the escaping you want. You can also use ' for quoting strings so that internal " needed for shell quoting can be entered literally.
Putting it all together (and likely messing something up here), you would get
#staticmethod
def run_dxl_importRTF():
module_name = "TEST_TEMP"
script_path = "importRTF.dxl"
script_do_nothing_path = "doNothing.dxl"
user = "user"
password = "pass"
database_config = "11111#11.11.1111.0"
doors_path = r"C:\Program Files\IBM\Rational\DOORS\9.7\bin\doors.exe"
file_name = "LIBC_String.rtf"
script_arguments = (rf'string pModuleName=\"{module_name}\";'
'string pFileName=\"{file_name}\";'
'#include <{script_path}>')
command = (f'"{doors_path}" -dxl "{script_arguments}" -f "%TEMP%"'
' -b "{script_do_nothing_path}" -d {database_config}'
' -user {user} -password {pass}')
res = subprocess.run(command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, text=True)
print(f"COMMAND:\n{' '.join(res.args)}")
print(f"STDERR: {repr(res.stderr)}")
print(f'STDOUT: {res.stdout}')
print(f'RETURN CODE: {res.returncode}')

Running a C executable inside a python program

I have written a C code where I have converted one file format to another file format. To run my C code, I have taken one command line argument : filestem.
I executed that code using : ./executable_file filestem > outputfile
Where I have got my desired output inside outputfile
Now I want to take that executable and run within a python code.
I am trying like :
import subprocess
import sys
filestem = sys.argv[1];
subprocess.run(['/home/dev/executable_file', filestem , 'outputfile'])
But it is unable to create the outputfile. I think some thing should be added to solve the > issue. But unable to figure out. Please help.

subprocess.run has optional stdout argument, you might give it file handle, so in your case something like
import subprocess
import sys
filestem = sys.argv[1]
with open('outputfile','wb') as f:
subprocess.run(['/home/dev/executable_file', filestem],stdout=f)
should work. I do not have ability to test it so please run it and write if it does work as intended

You have several options:
NOTE - Tested in CentOS 7, using Python 2.7
1. Try pexpect:
"""Usage: executable_file argument ("ex. stack.py -lh")"""
import pexpect
filestem = sys.argv[1]
# Using ls -lh >> outputfile as an example
cmd = "ls {0} >> outputfile".format(filestem)
command_output, exitstatus = pexpect.run("/usr/bin/bash -c '{0}'".format(cmd), withexitstatus=True)
if exitstatus == 0:
print(command_output)
else:
print("Houston, we've had a problem.")
2. Run subprocess with shell=true (Not recommended):
"""Usage: executable_file argument ("ex. stack.py -lh")"""
import sys
import subprocess
filestem = sys.argv[1]
# Using ls -lh >> outputfile as an example
cmd = "ls {0} >> outputfile".format(filestem)
result = subprocess.check_output(shlex.split(cmd), shell=True) # or subprocess.call(cmd, shell=True)
print(result)
It works, but python.org frowns upon this, due to the chance of a shell injection: see "Security Considerations" in the subprocess documentation.
3. If you must use subprocess, run each command separately and take the SDTOUT of the previous command and pipe it into the STDIN of the next command:
p = subprocess.Popen(cmd, stdin=PIPE, stdout=PIPE)
stdout_data, stderr_data = p.communicate()
p = subprocess.Popen(cmd, stdin=stdout_data, stdout=PIPE)
etc...
Good luck with your code!

Responding to an input when executing a python script through SSH

I am currently using a windows machine and trying to SSH to an Ubuntu Server, using PKI. I need to run the python script test.py using the sudo command. The script contains an input that will ask me for a number after running test.py. I have tried putting for a number after the sudo command but it did not work.
import subprocess
command = "ssh -t john#x.x.x.x echo 'john' | sudo -S python3.7 ./Desktop/test.py"
command = command.split()
out = subprocess.check_output(["scp", "test.py", "john#x.x.x.x:./Desktop"])
run_script = subprocess.run(command, stdout = subprocess.PIPE, stderr = subprocess.PIPE)
Example of test.py:
import subprocess
choice = input("Choose a number: ")
if choice == 1:
result = subprocess.run(['cat' '/etc/passwd'], stdout = subprocess.PIPE, stderr = subprocess.PIPE)
print(result.stdout.decode('utf-8'))
How can I respond to this input and where should I put it in my code?

If your remote command's standard input is connected to your Python script, simply use that.
run_script = subprocess.run(command,
stdout = subprocess.PIPE, stderr = subprocess.PIPE,
input="1\n", text=True)
I'm guessing you'll want to capture the output with capture=True, and probably also add check=True to the keyword arguments as well.
The addition of text=True saves you from having to encnde your input into bytes, and similarly from having to decode the output.

from Python run PS command with argument (quotation and slashes)

I'd like to change my current password from Python script using PowerShell. For this I need to run PowerShell command from the code bellow. What would be the best way to do that?
import datetime
#generated password is always changing. There might be different quotations and slashes so it's tricky to put it into PowerShell command bellow
generated_password=r'/#\n\9{S;-l2H~'
#password to set is changing depending on date. Let's say today it has become 'xxx180220'
password_to_set="xxx"+datetime.datetime.now().strftime("%y%m%d")
#I'd like to run this PS command to change my generated password to value stored in password_to_set variable. How do I do it?
PowerShell.exe -ExecutionPolicy ByPass ([adsi]'WinNT://valueaddco/wadmin,user').ChangePassword('/#\n\9{S;-l2H~','xxx180220')

I've created new1.ps file:
echo "changing password from $($args[0]) to $($args[1])"
([adsi]'WinNT://valueaddco/wadmingolyaa1,user').ChangePassword($($args[0]), $($args[1]))
It could be launched with Python code:
from subprocess import Popen, PIPE
cmd = ['powershell.exe', '-ExecutionPolicy', 'ByPass', '-File', 'new.ps1', generated_password, password_to_set]
proc= Popen(cmd, stdout=PIPE, stderr=PIPE)
while True:
line = proc.stdout.readline()
if line != b'':
print(line.strip())
else:
break

Switch user before executing a python script

I am trying to use su linux command in a python script along with getpass and subprocess python standard libraries to switch to a new user before executing my script. I am using this function:
import subprocess
import getpass
def ask_passwd():
pw = getpass.getpass(prompt='Please insert the passwd to use the auto api:') # prompt the user for a passwd
print pw
command = "su new_user"
p = subprocess.Popen(command.split(), stdin=subprocess.PIPE)
p.communicate(pw) # communicate the passwd
def main():
# code here
if __name__ == '__main__':
ask_passwd()
main()
When executing the script the main code works but not the su command. Here what I get back:
Please insert the passwd to use the auto api:
pass
su: must be run from a terminal
Any help? thanks

you may use subprocess.popen. There is an example
import subprocess
sudo_password = 'yourpassword'
command = '-i -u youruser \n id'
command = command.split()
cmd1 = subprocess.Popen(['echo', sudo_password], stdout=subprocess.PIPE)
cmd2 = subprocess.Popen(['sudo', '-S'] + command, stdin=cmd1.stdout, stdout=subprocess.PIPE)
output = cmd2.stdout.read()
print output