Develop Reference

Writing Python Script from Batch Script is not working for one command

I'm trying to convert batch script into python script.
This is batch script, which is calling Klockwork exe on the project specified building it.
%KwPath%\Kwinject -o kwinjectmp.out msbuild %BaseProjPath%/CodingGuide.vcxproj /t:Rebuild /p:Configuration="Release" /p:Platform="x64" /p:CLToolExe=cl.exe /p:CLToolPath=%VSBinPath%
I have write equivalent python script for it.
args = KwPath + '\\Kwinject.exe sync -o ' + 'kwinjectmp.out' + 'msbuild ' + BaseProject + '\\' + ProjectFolder + '\\' + ProjectName + '/t:Rebuild /p:Configuration="Release" /p:Platform="x64" /p:CLToolExe=cl.exe /p:CLToolPath=' + VSBinPath
print(args)
subprocess.call(args, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
Where I have declared BaseProject, VSBinPath, KwPath correctly. But the exectuion is not happening as it is happening in BatchScript, Basically script is not giving any output/working.

spaces between arguments may be absent because of your usage. try this:
import os
path1 = os.path.join(KwPath, 'Kwinject.exe')
path2 = os.path.join(BaseProject, ProjectFolder, ProjectName)
subprocess.call([
path1,
'sync',
'-o', 'kwinjectmp.out',
'msbuild',
path2,
'/t:Rebuild',
'/p:Configuration="Release"',
'/p:Platform="x64"',
'/p:CLToolExe=cl.exe',
'/p:CLToolPath=' + VSBinPath
],
shell=True,
stdout=subprocess.PIPE,
stderr=subprocess.STDOUT
)

Calling R script from Python does not save log file in version 4

I am calling from python an R script that is very simple called R Code.R:
args <- commandArgs(TRUE)
print(args)
source(args[1])
setwd("<YOUR PATH>")
output <- head(mtcars, n = n)
write.table(output, "output.txt")
using the following script:
import subprocess
pth = "<YOUR PATH>"
subprocess.call(" ".join(["C:/R/R-3.6.0/bin/x64/R.exe", "-f", '"' + pth + '/R Code.R"', "--args",
'"' + pth + '/arguments.txt"',"1>", '"' + pth + '/log.txt"', "2>&1"]))
subprocess.call(" ".join(["C:/R/R-4.0.3/bin/x64/R.exe", "-f", '"' + pth + '/R Code.R"', "--args",
'"' + pth + '/arguments.txt"',"1>", '"' + pth + '/log.txt"', "2>&1"]))
where arguments.txt contain:
n <- 10
The problem is that when I am using R-4.0.3 the log.txt file is not generating and I need to dump a log file because it is automatically looking for it in a posterior process I have.
When I am executing in CMD (Windows) the following command:
C:/R/R-4.0.3/bin/x64/R.exe -f "<YOUR PATH>/R Code.R" --args "<YOUR PATH>/arguments.txt" 1> "<YOUR PATH>/log.txt" 2>&1'
It does work perfectly, it is only when embedded in another software.
Also, I have tried without white space in the name and calling the scripts from root folder without having to specify the path.
Any idea of why it doesn't work for R-4.* or even better, how to solve it?
Thank you!
PD: Thank you, Martin, for your tips and for making me formulate a better question

Rhelp people got this solved, thank you, Duncan Murdoch!
Solution 1:
import os
pth = "<YOUR PATH>"
os.system(" ".join(["C:/R/R-4.0.3/bin/x64/R.exe", "-f", '"' + pth + '/RCode.R"', "--args",
'"' + pth + '/arguments.txt"',"1>", '"' + pth + '/log.txt"']))
Solution 2:
import subprocess
pth = "<YOUR PATH>"
subprocess.call(" ".join(["1>", '"' + pth + '/log.txt"', "2>&1",
"C:/R/R-4.0.3/bin/x64/R.exe", "-f", '"' + pth + '/RCode.R"', "--args",
'"' + pth + '/arguments.txt"']), shell = True)

well, in one case (3.6.0) you use R.exe , in the other (4.0.3) Rscript.exe.
Both R and Rscript have existed for a long time, and they have always had slightly different behavior.
You really should not confuse them with each other (even though, on Windows, I see, they look like the same file .. they will not behave the same).
Ok, now you use R.exe for both.
Just to find out more / see more where the problem may happen, can you try
all of
using a minimal reproducible example, i.e. one we can use directly ourselves, i.e., not using "<YOUR PATH>" (nor setwd(.))
not using file names with a ' ' (space), i.e., e.g., use code.R
calling this from a "terminal"/shell instead of as python subprocess ?
Last but not least: Yes, for R 4.0.0, a completely updated toolset ("brandnew toolchain", e.g. much newer C compiler) was used to build R for windows, "Rtools 4.0" or rtools40: https://cran.r-project.org/bin/windows/Rtools/ . So changes are expected but should typically only have been to the better, not the worse ..

Convert .xls to .xlsb Running PowerShell Script from Python, ERROR: "You cannot call a method on a null-valued expression"

I am attempting to run PowerShell script from python to convert .xls files to .xlsb. by looping through a list of file names. I am encountering a PowerShell error "You cannot call a method on a null-valued expression" for command 3 (i.e. cmd3), and I am unsure why (this is my first time with python and running PowerShell script in general). The error is encountered when trying to open the workbook, but when the command is run in PowerShell directly, it seems to work fine.
Code:
import logging, os, shutil, itertools, time, pyxlsb, subprocess
# convert .xls to .xlsb and / transfer new terminology files
for i in itertools.islice(FileList, 0, 6, None):
# define extension
ext = '.xls'
# define file path
psPath = f'{downdir}' + f'\{i}'
# define ps scripts
def run(cmd):
completed = subprocess.run(["powershell", "-Command", cmd], capture_output=True)
return completed
# ps script: open workbook
cmd1 = "$xlExcel12 = 50"
cmd2 = "$Excel = New-Object -Com Excel.Application"
cmd3 = f"$WorkBook = $Excel.Workbooks.Open('{psPath}{ext}')"
cmd4 = f"$WorkBook.SaveAs('{psPath}{ext}',$xlExcel12,[Type]::Missing,
[Type]::Missing,$false,$false,2)"
cmd5 = "$Excel.Quit()"
# ps script: delete.xls files
cmd6 = f"Remove-Item '{psPath}{ext}'"
run(cmd1)
run(cmd2)
run(cmd3)
# change extension
ext = '.xlsb'
run(cmd4)
run(cmd5)
run(cmd6)
# copy .xlsb files to terminology folder
shutil.copy(i + ext, termdir)
Error:
Out[79]: CompletedProcess(args=['powershell', '-Command', "$WorkBook = > > $Excel.Workbooks.Open('C:\Users\Username\Downloads\SEND Terminology.xls')"], returncode=1, stdout=b'', stderr=b"You cannot call a method on a null-valued expression.\r\nAt line:1 char:1\r\n+ $WorkBook = $Excel.Workbooks.Open('C:\Username\User\Downloads\SEND Ter ...\r\n+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~\r\n + CategoryInfo : InvalidOperation: (:) [], RuntimeException\r\n + FullyQualifiedErrorId : InvokeMethodOnNull\r\n \r\n")
Any input would be helpful.
Thank you!

Problem
As commenter vonPryz correctly stated, the Powershell commands run in separate processes. The memory spaces of processes are isolated from each other, and will be cleared when a process ends.
When you run the commands in separate Powershell processes, the cmd3, cmd4 and cmd5 won't have the variable $Excel available. Powershell defaults to a $null value for non-existing variables, hence the error message "You cannot call a method on a null-valued expression". The same happens for variable $xlExcel12. These variables only exists as long as the processes that created them were running and would only be visible within these processes, even if you managed to create two processes in parallel.
Solution
Commands cmd1..5 need to be run in the same Powershell process, so each command will be able to "see" the variables created by previous commands:
run( cmd1 + ';' + cmd2 + ';' + cmd3 + ';' + cmd4 + ';' + cmd5 )
You will need to change cmd4 to use another variable for the extension that will be used for saving, e. g. extSave
cmd4 = f"$WorkBook.SaveAs('{psPath}{extSave}',$xlExcel12,[Type]::Missing,
[Type]::Missing,$false,$false,2)"
The cmd6 is completely independent, because it does not depend on Powershell variables. It only depends on python variables, which are resolved before the process starts, so it could still be run in a separate process.

How to translate command-line into python code

I am currently working with python and I have some real trouble with the syntax.
I wanted to include a Cura command-line into my script but I do not know how to actually parse the line so that the command will be executed out of my Blender Script.
My code is the following and I really do not know if this is even valid:
def main(context):
blend_file_path = bpy.data.filepath
directory = os.path.dirname(blend_file_path)
target_file = os.path.join(directory,bpy.path.basename(bpy.context.blend_data.filepath) +'.stl')
bpy.ops.export_mesh.stl(filepath=target_file)
cevar = 'CuraEngine slice -j "C:\\Programme\\Ultimaker Cura 4.5\\resources\\definitions\\fdmprinter.def.json"' +' -l ' + directory + bpy.path.basename(bpy.context.blend_data.filepath) +'.stl +o '+ directory + bpy.path.basename(bpy.context.blend_data.filepath) +'.gcode'
os.system(cevar)

Use this code
import subprocess
cmd_args = [
"CuraEngine",
"slice",
"-j",
"C:\\Programme\\Ultimaker Cura 4.5\\resources\\definitions\\fdmprinter.def.json",
"-l",
f"{os.path.join(directory,bpy.path.basename(bpy.context.blend_data.filepath))}.stl",
"+o",
f"{os.path.join(directory, bpy.path.basename(bpy.context.blend_data.filepath))}.gcode",
]
output = subprocess.Popen(cmd_args, stdout=subprocess.PIPE).communicate()[0]

python 3 open terminal and run program

I made a small script in sublime that will extract commands from a json file that is on the user's computer and then it will open the terminal and run the settings/command. This works, except that it doesn't really open up the terminal. It only runs the command (and it works, as in my case it will run gcc to compile a simple C file), and pipes to STDOUT without opening up the terminal.
import json
import subprocess
import sublime_plugin
class CompilerCommand(sublime_plugin.TextCommand):
def get_dir(self, fullpath):
path = fullpath.split("\\")
path.pop()
path = "\\".join(path)
return path
def get_settings(self, path):
_settings_path = path + "\\compiler_settings.json"
return json.loads(open(_settings_path).read())
def run(self, edit):
_path = self.get_dir(self.view.file_name())
_settings = self.get_settings(_path)
_driver = _path.split("\\")[0]
_command = _driver + " && cd " + _path + " && " + _settings["compile"] + " && " + _settings["exec"]
proc = subprocess.Popen(_command, shell=True, stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
I'm not sure if using subprocess.Popen is the right way to go about it as I'm new to Python.
So to re-iterate; I want it to open up the terminal, run the command, and have the terminal stay open until the user presses ENTER or something. I'm running Windows 7 and Python 3, if that matters.

subprocess.Popen simply creates a subprocess with the given command. It is in no way related to opening a terminal window or any other windows for that matter.
You'll have to look into your platform specific UI automation solutions in order to achieve what you want. Or see if maybe the Sublime plugins mechanism can already do that.
NOTES:
Also, you should be using os.path.join/os.path.split/os.path.sep etc for your path operations—Sublime also runs on OS X for example, and OS X does not use backslashes. Also, file handles need to be closed, so use:
with open(...) as f:
return json.load(f) # also not that there is no nead to f.read()+json.loads()
# if you can just json.load() on the file handle
Furthermore, strings should usually be built using string interpolation:
_command = "{} && cd {} && {} && {}".format(_driver, _path, _settings["compile"], _settings["exec"])
...and, you should not be prefixing your local variables with _—it doesn't look nice and serves no purpose in Python either; and while we're at it, I might as well use the chance to recommend you to read PEP8: http://www.python.org/dev/peps/pep-0008/.