Develop Reference

Trying to launch subprocess from macro

I am trying to launch a python subprocess from excel using PYXLL, but it seems to have trouble launching the cmd window and running commands.
Below is a sample of what I am trying to run:
#xl_macro()
def test():
if 1 == 1:
xlcAlert("Next line nothing happens") #Popup appears
p = subprocess.Popen(r'start cmd /k', shell=True, creationflags=subprocess.CREATE_NEW_CONSOLE, stdout=subprocess.PIPE,
stderr=subprocess.STDOUT)
xlcAlert("{}".format(p.pid)) #p was never launched
I am trying to capture values from excel and pass them in a subprocess. This works when executing in my IDE: data is read from excel and then subprocess launches window. However, once adding the decorator to have it run as macro in EXCEL, the script will just stop once subprocess.Popen line is reached. Is there any way to launch a subprocess from pyxll?

After investigation, and thanks to Charles Duffy, Microsoft Office SandBoxing kills the shell subprocess. This has been implemented for security reasons in latest versions.
The simple solution is to run subprocess with shell=False and pass the args in a list:
p1 = subprocess.Popen(cmdlist, shell=False)
The Sandboxing will not terminate the process - python window will open while script is running.

Python subprocess hangs when using stdout=subprocess.PIPE

I am trying to run the following simple command in Python which returns a directory associated with the Node.js
import subprocess
import shutil
cmd = [shutil.which("npm"), 'bin']
subprocess.run(cmd, stdout=subprocess.PIPE)
I've tested this on two different systems, on my Mac it runs almost instantly whereas on my Windows 10 system it hangs for a long time even though running the command npm bin in the terminal completes immediately. Running other commands, say cmd = [shutil.which("condo"), "info"] works fine. If I don't specify the stdout argument it will also work OK. I would like to figure why it is hanging and how to fix that.
UPDATE
Also tried
p = subprocess.Popen(cmd, stdout=subprocess.PIPE)
p.communicate()
On Windows 10 under Python 3.6.10, same result as running subprocess.run().

Python Popen hangs, yet same command in Windows cmd window runs fine

I am pulling my hair out here. I am spawning a process which I need the feedback from in Python.
When I run the command in the cmd window it runs fine, but when I try to run it via Python the terminal hangs.
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Where startcmd is a string which when printed in the Python console looks like this:
"C:/Program Files/GRASS GIS 7.2.1/grass72.bat" --version
If I copy and paste this into a Windows cmd, it shows the version information and returns control to the command prompt about a second later, but in Python it freezes up.
I should point out, if I replace the startcmd string with something like "dir" or even "python --version", it works fine!
Additional: I have tried shell=True, this has the same result.
Additional: I have tried sending the cmd and arguments through as an array as suggested in an answer below given that shell=False, but this also hangs the same.
Additional: I have added the GRASS path to the system PATH, so that now I can simply call grass72 --version in the cmd window to get a result, however this also still freezes in Python but works fine in cmd.
Additional: I have created a basic .bat file to test if .bat files run ok via Python, here is what I created:
#echo off
title Test Batch Script
echo I should see this message
This runs fine both in cmd, and in Python.
Problem found but not solved!
So, I'm running the script which spawns the process using subprocess.Popen using Python 3.6. The .bat file which is spawned launches a Python script using a version of Python (based on 2.7) which comes shipped with GRASS:
%GRASS_PYTHON% "\BLAH\BLAH\grass72.py"
What is interesting, is that if I launch the subprocess.Popen script with Python 2.7, it works fine. Ahah, you may think, solved! But this doesn't solve my problem - because I really need Python 3.6 to be launching the process, also why does it matter what version of Python launches the batch file? The new Python script which is spawned is launched with Python 2.7 anyway.
Since I started re-directing stdout I can see that there is an error when I use Python 3.6 to launch the process:
File "C:\ProgramData\Anaconda3\lib\site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Notice its reverting to Anaconda3! Even though it is launched using python.exe from 2.7!

I experienced the same issue with Python 3.6 and 3.7 on Windows hanging for subprocess calls:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Upon closer investigation I noticed this occurs only if the process writes more than about 4 KB (4096 bytes) of output which might explain why your short script does not reproduce this.
A workaround I found is using tempfile in the standard library:
# Write to a temporary file because pipe redirection seems broken
with tempfile.NamedTemporaryFile(mode="w+") as tmp_out,
tempfile.NamedTemporaryFile(mode="w+") as tmp_err:
p = subprocess.Popen(startcmd, stdout=tmp_out, stderr=tmp_err,
universal_newlines=True)
# `run` waits for command to complete, `Popen` continues Python program
while p.poll() is None:
time.sleep(.1)
# Cursor is after the last write call, reset to read output
tmp_out.seek(0)
tmp_err.seek(0)
out = tmp_out.read()
err = tmp_err.read()

You don't specify shell=True in your arguments to Popen. The recommended usage in that case is to specify a sequence of arguments instead of a string. So you should set startcmd equal to ["C:/Program Files/GRASS GIS 7.2.1/grass72.bat", "--version"].

Try this:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

Kill external process on pythonw.exe

In Python, external processes can be started easily using the subprocess module. For instance on Windows:
command = 'external_app'
my_process = subprocess.Popen(
command,
creationflags=subprocess.CREATE_NEW_PROCESS_GROUP,
shell=True,
stderr=subprocess.STDOUT,
stdout=subprocess.PIPE,
universal_newlines=True)
To kill the process, we can run:
os.kill(my_process.pid, signal.CTRL_BREAK_EVENT)
This works fine using the command-line interpreter of Python (python.exe). But if I like to start and stop processes from within a graphical Python application without a command-line window using pythonw.exe the problem occurs that I can’t stop the process with os.kill anymore.
How can I kill an external process on Windows with pythonw.exe?

How to automate interactive console applications in Python?

I want to control running process/program by script in Python.
I have a program `linphonec´ (You can install: apt-get install linphonec).
My task is:
Run linphonec (I'm using subprocess at the moment)
When linphonec is running it has many commands to control this and I want to e.g use proxy list (command in linphonec).
Simple flow:
test#ubuntu$ > linphonec
linphonec > proxy list
How can I do this?

There are actually 2 ways to communicate:
Run your program with myprogram.py | linphonec to pass everything you print to linphonec
Use subprocess.Popen with subprocess.PIPE in constructor via keywrod-args for stdin (propably stdout and stderr, too) and then communicate for a single command or use stdin and stdout (stderr) as files
import subprocess
p=subprocess.Popen("linphonec",
stdin=subprocess.PIPE,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
universal_newlines=True) #this is for text communication
p.stdin.write("proxy list\n")
result_first_line=p.stdout.readline()