Develop Reference

Command not found error while trying to run imagemagick from within IDLE

I want to do some ImageMagick stuff through Python and I have ImageMagick installed on my Mac OS. When I run my code from within Python IDLE, it gives me /bin/sh: magick: command not found error while running the same program through terminal works fine. For example, when I run it like sh-3.2$ python3 imageMagicProblems.py from terminal, there is no issue.
Can someone help me understand why the problem happens when run through Python IDLE? See attached screenshot also.
Here is my code:
import subprocess
args = "magick -version"
p = subprocess.Popen(args, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
print (out.decode('utf-8').rstrip())
print (err.decode('utf-8').rstrip())

The problem is that your environment is different, specifically your PATH. In order to find the full path to your magick command, go in your normal Terminal where magick works and run:
type magick
and it will tell you something like:
magick is
/usr/local/bin/magick
Now in your Python subprocess command, use that same full path you found, e.g.
args = '/usr/local/bin/magick -version'

PyInstaller + subprocess.check_output + Windows Scheduled task

this is a topic for somebody who may face similar thing in future.
Once you try to schedule task in Windows using executable that was compiled with PyInstaller with option -noconsole and trying to use subprocess.check_output it does not work
Funny thing that if you call .exe directly then it works perfectly fine. However if you schedule a task or try to wrap execution into batch like:
C:\git\backend.exe -p C:\Users\settings.json
then it fails without any error code.
Only thing one can found is in tasks event log is error code: 2147942401

solution will be to explicitly specify Popen with stdin, out, err
p = subprocess.Popen(command, stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
output = p.stdout.read().decode('utf-8')
p.communicate()
I found similar topics but just want to explicitly mention that by default command call or .exe double click it works but not from scheduled task
Inspired from here:
Pyinstaller issue with subprocess.check_output
Pyinstaller subprocess.check_output error
pyinstaller on Windows with --noconsole simply won't work
PS
I always look for better solutions, if you have one, please suggest!

Python Popen hangs, yet same command in Windows cmd window runs fine

I am pulling my hair out here. I am spawning a process which I need the feedback from in Python.
When I run the command in the cmd window it runs fine, but when I try to run it via Python the terminal hangs.
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Where startcmd is a string which when printed in the Python console looks like this:
"C:/Program Files/GRASS GIS 7.2.1/grass72.bat" --version
If I copy and paste this into a Windows cmd, it shows the version information and returns control to the command prompt about a second later, but in Python it freezes up.
I should point out, if I replace the startcmd string with something like "dir" or even "python --version", it works fine!
Additional: I have tried shell=True, this has the same result.
Additional: I have tried sending the cmd and arguments through as an array as suggested in an answer below given that shell=False, but this also hangs the same.
Additional: I have added the GRASS path to the system PATH, so that now I can simply call grass72 --version in the cmd window to get a result, however this also still freezes in Python but works fine in cmd.
Additional: I have created a basic .bat file to test if .bat files run ok via Python, here is what I created:
#echo off
title Test Batch Script
echo I should see this message
This runs fine both in cmd, and in Python.
Problem found but not solved!
So, I'm running the script which spawns the process using subprocess.Popen using Python 3.6. The .bat file which is spawned launches a Python script using a version of Python (based on 2.7) which comes shipped with GRASS:
%GRASS_PYTHON% "\BLAH\BLAH\grass72.py"
What is interesting, is that if I launch the subprocess.Popen script with Python 2.7, it works fine. Ahah, you may think, solved! But this doesn't solve my problem - because I really need Python 3.6 to be launching the process, also why does it matter what version of Python launches the batch file? The new Python script which is spawned is launched with Python 2.7 anyway.
Since I started re-directing stdout I can see that there is an error when I use Python 3.6 to launch the process:
File "C:\ProgramData\Anaconda3\lib\site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Notice its reverting to Anaconda3! Even though it is launched using python.exe from 2.7!

I experienced the same issue with Python 3.6 and 3.7 on Windows hanging for subprocess calls:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Upon closer investigation I noticed this occurs only if the process writes more than about 4 KB (4096 bytes) of output which might explain why your short script does not reproduce this.
A workaround I found is using tempfile in the standard library:
# Write to a temporary file because pipe redirection seems broken
with tempfile.NamedTemporaryFile(mode="w+") as tmp_out,
tempfile.NamedTemporaryFile(mode="w+") as tmp_err:
p = subprocess.Popen(startcmd, stdout=tmp_out, stderr=tmp_err,
universal_newlines=True)
# `run` waits for command to complete, `Popen` continues Python program
while p.poll() is None:
time.sleep(.1)
# Cursor is after the last write call, reset to read output
tmp_out.seek(0)
tmp_err.seek(0)
out = tmp_out.read()
err = tmp_err.read()

You don't specify shell=True in your arguments to Popen. The recommended usage in that case is to specify a sequence of arguments instead of a string. So you should set startcmd equal to ["C:/Program Files/GRASS GIS 7.2.1/grass72.bat", "--version"].

Try this:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)

Why 'os.system' exits with return code 1?

I want to execute some adb commands from python script. But when i executed the following line
os.system('adb devices')
The cmd returns with 1 instead of 0. I also tried executing
os.popen('adb devices').read()
I am getting empty string. Please help me to solve this.
Note: I tried the same commands from command window and it was working fine. I also added the path of adb.exe to windows PATH environment variable.

According to Windows docs, you've got 1, because there was an error on your command.
Maybe use subprocess could be a better approach.
import subprocess
subprocess.check_output(
"adb devices",
stderr=subprocess.STDOUT,
shell=True)

Struggling with Python's subprocess.Popen running a Perl script

I have a Perl script that I need to run with Python, and I've been trying to use subprocess to do it, unsuccessfully. I'm able to get the command to run just fine on the command line, but subprocess isn't able to get it to work.
If I have a Perl script like this:
#!/usr/bin/perl
use strict;
use warnings;
my $name = shift;
print "Hello $name!\n";
I am able to successfully run the command on the command line like so
C:\current\path> perl test.pl world
>>>Hello world!
But when I try and invoke the same command with subprocess, I get this error
cmd = 'perl test.pl world'
pipe = subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE)
>>>"'perl' is not recognized as an internal or external command, operable program or batch file."
I've tried different combinations of creating cmd as a list
cmd = ['perl','test.pl','world']
and of giving subprocess the absolute path to Perl
cmd = ['C:\path\to\perl.exe','test.pl','world']
but nothing gets it to work. I'm able to get subprocess to play well without Perl
pipe = subprocess.Popen('dir',shell=True,stdout=subprocess.PIPE)
pipe.stdout.read()
>>>Volume in drive C is Windows7_OS....
I'm absolutely sure that Perl is in my PATH, because like I said I can invoke the command just fine in the command line. I've looked around at different posts that suggested to check os.environ["COMSPEC"], adding Perl to the subprocess env, and nothing has worked.
EDIT: I also can't get the command to work with any other subprocess methods: check_output() returns an empty byte string, and call() returns 1.
Any sort of fix or alternative solution would be immensely appreciated. I'm running Python 3.4 on Windows 7 64bit. I've tried 32- and 64-bit Python as well, to no avail.
UPDATE:
I've been able to get this to work on Mac, but with one difference: I can't have shell=True. Other than that, any subprocess function I want will work, including Popen, check_output, and call.
So I guess this is a Windows problem more than anything. I've tried not setting shell=True on my Windows machine, but I always get this error:
WindowsError: [Error 193] %1 is not a valid Win32 application
UPDATE 2:
I've also tried creating a .bat file that runs the command, but I get the same error as well I try to just call Perl.
pipe = subprocess.Popen('test.bat',shell=True,stdout=subprocess.PIPE)
>>>"'test.bat' is not recognized as an internal or external command, operable program or batch file."
Where test.bat has only one line:
echo Hello, World

you could try using os.system('command') instead. Apparently its not well regarded for some reason, or at least not as well regarded as subprocess though. Alternately, you could try some of the other subprocess methods, such as call, check_call, or check_output. I've had a similar problem with Popen in the past and switching to one of these methods helped.

I got the same error while trying to run a perl script from Python.
I managed to get it work by adding the full path of the perl.exe (with an extra escape character, as '\') to the command I was trying to run as below:
pipe = subprocess.Popen('C:\\Perl64\\bin\\perl myscrip.pl',shell=True,stdout=subprocess.PIPE)
pipe.stdout.read()

This works fine for me. I know it's been a while and things may have just fixed themselves.
I'm using activestate perl 5.8.9
and python 3.9.5 on Windows 10.
from subprocess import Popen, PIPE
proc = Popen(['perl', 'test.pl', 'world'], stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
if proc.returncode == 0:
print(stdout.decode())
else:
print(stderr.decode())