I am trying to launch a python subprocess from excel using PYXLL, but it seems to have trouble launching the cmd window and running commands.
Below is a sample of what I am trying to run:
#xl_macro()
def test():
if 1 == 1:
xlcAlert("Next line nothing happens") #Popup appears
p = subprocess.Popen(r'start cmd /k', shell=True, creationflags=subprocess.CREATE_NEW_CONSOLE, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
xlcAlert("{}".format(p.pid)) #p was never launched
I am trying to capture values from excel and pass them in a subprocess. This works when executing in my IDE: data is read from excel and then subprocess launches window. However, once adding the decorator to have it run as macro in EXCEL, the script will just stop once subprocess.Popen line is reached. Is there any way to launch a subprocess from pyxll?
After investigation, and thanks to Charles Duffy, Microsoft Office SandBoxing kills the shell subprocess. This has been implemented for security reasons in latest versions.
The simple solution is to run subprocess with shell=False and pass the args in a list:
p1 = subprocess.Popen(cmdlist, shell=False)
The Sandboxing will not terminate the process - python window will open while script is running.
Related
I have been trying to debug a massive PyTorch model in VScode.
The starting point of the code first processes a configuration file, and then runs a subprocess containing the cofigs.
The issue is that after calling subprocess.call functions, the VScode, the code is executed in an external sub-process, which does not allow to use breakpoint.
example code:
def main():
args = parse_args()
cmd = construct_cmd(args)
subprocess.call(cmd, shell=True)
where cmd is the string command to be executed, but after this line, all breakpoints are ignored (probably because a sub-process runs this command.
Any solutions how to solve this?
I am trying to run the following simple command in Python which returns a directory associated with the Node.js
import subprocess
import shutil
cmd = [shutil.which("npm"), 'bin']
subprocess.run(cmd, stdout=subprocess.PIPE)
I've tested this on two different systems, on my Mac it runs almost instantly whereas on my Windows 10 system it hangs for a long time even though running the command npm bin in the terminal completes immediately. Running other commands, say cmd = [shutil.which("condo"), "info"] works fine. If I don't specify the stdout argument it will also work OK. I would like to figure why it is hanging and how to fix that.
UPDATE
Also tried
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
p.communicate()
On Windows 10 under Python 3.6.10, same result as running subprocess.run().
I am pulling my hair out here. I am spawning a process which I need the feedback from in Python.
When I run the command in the cmd window it runs fine, but when I try to run it via Python the terminal hangs.
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Where startcmd is a string which when printed in the Python console looks like this:
"C:/Program Files/GRASS GIS 7.2.1/grass72.bat" --version
If I copy and paste this into a Windows cmd, it shows the version information and returns control to the command prompt about a second later, but in Python it freezes up.
I should point out, if I replace the startcmd string with something like "dir" or even "python --version", it works fine!
Additional: I have tried shell=True, this has the same result.
Additional: I have tried sending the cmd and arguments through as an array as suggested in an answer below given that shell=False, but this also hangs the same.
Additional: I have added the GRASS path to the system PATH, so that now I can simply call grass72 --version in the cmd window to get a result, however this also still freezes in Python but works fine in cmd.
Additional: I have created a basic .bat file to test if .bat files run ok via Python, here is what I created:
#echo off
title Test Batch Script
echo I should see this message
This runs fine both in cmd, and in Python.
Problem found but not solved!
So, I'm running the script which spawns the process using subprocess.Popen using Python 3.6. The .bat file which is spawned launches a Python script using a version of Python (based on 2.7) which comes shipped with GRASS:
%GRASS_PYTHON% "\BLAH\BLAH\grass72.py"
What is interesting, is that if I launch the subprocess.Popen script with Python 2.7, it works fine. Ahah, you may think, solved! But this doesn't solve my problem - because I really need Python 3.6 to be launching the process, also why does it matter what version of Python launches the batch file? The new Python script which is spawned is launched with Python 2.7 anyway.
Since I started re-directing stdout I can see that there is an error when I use Python 3.6 to launch the process:
File "C:\ProgramData\Anaconda3\lib\site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Notice its reverting to Anaconda3! Even though it is launched using python.exe from 2.7!
I experienced the same issue with Python 3.6 and 3.7 on Windows hanging for subprocess calls:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Upon closer investigation I noticed this occurs only if the process writes more than about 4 KB (4096 bytes) of output which might explain why your short script does not reproduce this.
A workaround I found is using tempfile in the standard library:
# Write to a temporary file because pipe redirection seems broken
with tempfile.NamedTemporaryFile(mode="w+") as tmp_out,
tempfile.NamedTemporaryFile(mode="w+") as tmp_err:
p = subprocess.Popen(startcmd, stdout=tmp_out, stderr=tmp_err,
universal_newlines=True)
# `run` waits for command to complete, `Popen` continues Python program
while p.poll() is None:
time.sleep(.1)
# Cursor is after the last write call, reset to read output
tmp_out.seek(0)
tmp_err.seek(0)
out = tmp_out.read()
err = tmp_err.read()
You don't specify shell=True in your arguments to Popen. The recommended usage in that case is to specify a sequence of arguments instead of a string. So you should set startcmd equal to ["C:/Program Files/GRASS GIS 7.2.1/grass72.bat", "--version"].
Try this:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
In Python, external processes can be started easily using the subprocess module. For instance on Windows:
command = 'external_app'
my_process = subprocess.Popen(
command,
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
shell=True,
stderr=subprocess.STDOUT,
stdout=subprocess.PIPE,
universal_newlines=True)
To kill the process, we can run:
os.kill(my_process.pid, signal.CTRL_BREAK_EVENT)
This works fine using the command-line interpreter of Python (python.exe). But if I like to start and stop processes from within a graphical Python application without a command-line window using pythonw.exe the problem occurs that I can’t stop the process with os.kill anymore.
How can I kill an external process on Windows with pythonw.exe?
Is it possible to run a python script from another python script without wating for termination.
Parent process will terminate immediately after creation of child process.
I tried:
subprocess.Popen([sys.executable, "main.py"], stdout=subprocess.PIPE, stderr=subprocess.PIPE, stdin=subprocess.PIPE)
and also:
os.system(...)
If you know that the other Python script has a main method, you could simply in you code call that other script:
import main
...
exit(main.main())
But here the other script executes in the context of calling script. If you want to avoid it, you could use the os.exec... functions, by launching a new Python interpretor:
import os
...
os.execl(sys.executable, "python", 'main.py')
The exec family functions will replace (under Unix-Linux) the current Python interpretor with a new one.
You can just add & to start script in background:
import os
os.system('/path/to/script.sh &')
exit()
In this case launched shell script will continue working even after main Python script exits.
But keep in mind that it can cause zombie processes appearance in our system.