I have a Perl script that I need to run with Python, and I've been trying to use subprocess to do it, unsuccessfully. I'm able to get the command to run just fine on the command line, but subprocess isn't able to get it to work.
If I have a Perl script like this:
#!/usr/bin/perl
use strict;
use warnings;
my $name = shift;
print "Hello $name!\n";
I am able to successfully run the command on the command line like so
C:\current\path> perl test.pl world
>>>Hello world!
But when I try and invoke the same command with subprocess, I get this error
cmd = 'perl test.pl world'
pipe = subprocess.Popen(cmd,shell=True,stdout=subprocess.PIPE)
>>>"'perl' is not recognized as an internal or external command, operable program or batch file."
I've tried different combinations of creating cmd as a list
cmd = ['perl','test.pl','world']
and of giving subprocess the absolute path to Perl
cmd = ['C:\path\to\perl.exe','test.pl','world']
but nothing gets it to work. I'm able to get subprocess to play well without Perl
pipe = subprocess.Popen('dir',shell=True,stdout=subprocess.PIPE)
pipe.stdout.read()
>>>Volume in drive C is Windows7_OS....
I'm absolutely sure that Perl is in my PATH, because like I said I can invoke the command just fine in the command line. I've looked around at different posts that suggested to check os.environ["COMSPEC"], adding Perl to the subprocess env, and nothing has worked.
EDIT: I also can't get the command to work with any other subprocess methods: check_output() returns an empty byte string, and call() returns 1.
Any sort of fix or alternative solution would be immensely appreciated. I'm running Python 3.4 on Windows 7 64bit. I've tried 32- and 64-bit Python as well, to no avail.
UPDATE:
I've been able to get this to work on Mac, but with one difference: I can't have shell=True. Other than that, any subprocess function I want will work, including Popen, check_output, and call.
So I guess this is a Windows problem more than anything. I've tried not setting shell=True on my Windows machine, but I always get this error:
WindowsError: [Error 193] %1 is not a valid Win32 application
UPDATE 2:
I've also tried creating a .bat file that runs the command, but I get the same error as well I try to just call Perl.
pipe = subprocess.Popen('test.bat',shell=True,stdout=subprocess.PIPE)
>>>"'test.bat' is not recognized as an internal or external command, operable program or batch file."
Where test.bat has only one line:
echo Hello, World
you could try using os.system('command') instead. Apparently its not well regarded for some reason, or at least not as well regarded as subprocess though. Alternately, you could try some of the other subprocess methods, such as call, check_call, or check_output. I've had a similar problem with Popen in the past and switching to one of these methods helped.
I got the same error while trying to run a perl script from Python.
I managed to get it work by adding the full path of the perl.exe (with an extra escape character, as '\') to the command I was trying to run as below:
pipe = subprocess.Popen('C:\\Perl64\\bin\\perl myscrip.pl',shell=True,stdout=subprocess.PIPE)
pipe.stdout.read()
This works fine for me. I know it's been a while and things may have just fixed themselves.
I'm using activestate perl 5.8.9
and python 3.9.5 on Windows 10.
from subprocess import Popen, PIPE
proc = Popen(['perl', 'test.pl', 'world'], stdout=PIPE, stderr=PIPE)
stdout, stderr = proc.communicate()
if proc.returncode == 0:
print(stdout.decode())
else:
print(stderr.decode())
Related
I am trying to write a simple python script that will open the Windows cmd line, change to a specified directory, and then input the text: 'Test.exe -blah -blahblah etc...' in order to run my Test.exe executable with my specified parameters from the cmd line.
The code I have so far is the following:
import subprocess
subprocess.Popen(r'C:\Windows\System32\cmd.exe', cwd=r'C:\PythonTestScripts')
This code successfully launches the windows cmd.exe, and changes the directory to the specified cwd above, but I have no idea how to pass text into the cmd window from Python.
I have tried passing it as a string argument within the subprocess.Popen brackets, I have also tried assigning PIPE to the stdin and stdout with not much luck. I am familiar with simple coding from Uni, but I am not familiar with Python's syntax or scripting.
Any help is greatly appreciated.
One of your issues may be that cmd.exe doesn't exit once you've started it.
I was successful passing the /C argument to cmd.exe, which tell it to exit after processing the command.
This works:
from subprocess import Popen,PIPE
my_command = "dir *.log"
with Popen([r"C:\Windows\System32\cmd.exe","/C",my_command], stdout=PIPE, cwd=r'C:\Temp') as proc:
print(proc.stdout.read().decode().replace("\r\n","\n"))
I'm working on using the subprocess module to send shell commands from Python, specifically, ssh. Below is a barebones sample:
import subprocess
sp = subprocess.run(["ssh"], shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
print(f"stdout: {sp.stdout.decode()} \n\nstderr: {sp.stderr.decode()}")
This should return the ssh command help from stdout, and nothing from stderr. However, I get:
stdout:
stderr: 'ssh' is not recognized as an internal or external command,
operable program or batch file.
I've tried other commands, like echo and cd, and those work fine. I am also able to use ssh when manually typing the command into the shell, but it fails when I try to do it through subprocess. The directory C:\Windows\System32\OpenSSH does exist on my computer (and it contains ssh.exe), but for some strange reason I'm unable to cd to it using subprocess.
If it matters, subprocess is using the command prompt, cmd.exe, as it seems to be the default.
Any help is appreciated. Thanks!
-- Edits with tests from comments --
Using the absolute path C:/Windows/System32/OpenSSH/ssh.exe does not work, and gives The system cannot find the path specified via stderr. The OpenSSH folder doesn't seem to be visible to Python through subprocess
os.environ[PATH] contains both C:/Windows/System32/ and C:/Windows/System32/OpenSSH/
Running it with shell=False (either with the absolute path or just with ssh) raises an error in Python: FileNotFoundError: [WinError 2] The system cannot find the file specified
You say C:\Windows\System32\OpenSSH\ssh.exe exists, but that it's not found when running from Python. This is likely a result of having a 32 bit version of Python installed, rather than a 64 bit version.
If the path exists elsewhere, but not for Python, that would tend to implicate the file system redirector. Python is probably seeing C:\Windows\SysWOW64 when you tell it to look in C:\Windows\System32. I'd recommend uninstalling whatever Python you have, and explicitly installing a 64 bit version, so it isn't affected by the redirector, and sees the "real" System32.
I am pulling my hair out here. I am spawning a process which I need the feedback from in Python.
When I run the command in the cmd window it runs fine, but when I try to run it via Python the terminal hangs.
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Where startcmd is a string which when printed in the Python console looks like this:
"C:/Program Files/GRASS GIS 7.2.1/grass72.bat" --version
If I copy and paste this into a Windows cmd, it shows the version information and returns control to the command prompt about a second later, but in Python it freezes up.
I should point out, if I replace the startcmd string with something like "dir" or even "python --version", it works fine!
Additional: I have tried shell=True, this has the same result.
Additional: I have tried sending the cmd and arguments through as an array as suggested in an answer below given that shell=False, but this also hangs the same.
Additional: I have added the GRASS path to the system PATH, so that now I can simply call grass72 --version in the cmd window to get a result, however this also still freezes in Python but works fine in cmd.
Additional: I have created a basic .bat file to test if .bat files run ok via Python, here is what I created:
#echo off
title Test Batch Script
echo I should see this message
This runs fine both in cmd, and in Python.
Problem found but not solved!
So, I'm running the script which spawns the process using subprocess.Popen using Python 3.6. The .bat file which is spawned launches a Python script using a version of Python (based on 2.7) which comes shipped with GRASS:
%GRASS_PYTHON% "\BLAH\BLAH\grass72.py"
What is interesting, is that if I launch the subprocess.Popen script with Python 2.7, it works fine. Ahah, you may think, solved! But this doesn't solve my problem - because I really need Python 3.6 to be launching the process, also why does it matter what version of Python launches the batch file? The new Python script which is spawned is launched with Python 2.7 anyway.
Since I started re-directing stdout I can see that there is an error when I use Python 3.6 to launch the process:
File "C:\ProgramData\Anaconda3\lib\site.py", line 177
file=sys.stderr)
^
SyntaxError: invalid syntax
Notice its reverting to Anaconda3! Even though it is launched using python.exe from 2.7!
I experienced the same issue with Python 3.6 and 3.7 on Windows hanging for subprocess calls:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
(out, err) = p.communicate()
Upon closer investigation I noticed this occurs only if the process writes more than about 4 KB (4096 bytes) of output which might explain why your short script does not reproduce this.
A workaround I found is using tempfile in the standard library:
# Write to a temporary file because pipe redirection seems broken
with tempfile.NamedTemporaryFile(mode="w+") as tmp_out,
tempfile.NamedTemporaryFile(mode="w+") as tmp_err:
p = subprocess.Popen(startcmd, stdout=tmp_out, stderr=tmp_err,
universal_newlines=True)
# `run` waits for command to complete, `Popen` continues Python program
while p.poll() is None:
time.sleep(.1)
# Cursor is after the last write call, reset to read output
tmp_out.seek(0)
tmp_err.seek(0)
out = tmp_out.read()
err = tmp_err.read()
You don't specify shell=True in your arguments to Popen. The recommended usage in that case is to specify a sequence of arguments instead of a string. So you should set startcmd equal to ["C:/Program Files/GRASS GIS 7.2.1/grass72.bat", "--version"].
Try this:
p = subprocess.Popen(startcmd, stdout=subprocess.PIPE, stderr=subprocess.PIPE, shell=True)
I want to execute some adb commands from python script. But when i executed the following line
os.system('adb devices')
The cmd returns with 1 instead of 0. I also tried executing
os.popen('adb devices').read()
I am getting empty string. Please help me to solve this.
Note: I tried the same commands from command window and it was working fine. I also added the path of adb.exe to windows PATH environment variable.
According to Windows docs, you've got 1, because there was an error on your command.
Maybe use subprocess could be a better approach.
import subprocess
subprocess.check_output(
"adb devices",
stderr=subprocess.STDOUT,
shell=True)
I'm working in a windows environment (my laptop!) and I need a couple of scripts that run other programs, pretty much like a windows batch file.
how can I run a command from python such that the program when run, will replace the script? The program is interactive (for instance, unison) and keeps printing lines and asking for user input all the time.
So, just running a program and printing the output won't suffice. The program has to takeover the script's input/output, pretty mcuh like running the command from a .bat file.
I tried os.execl but it keeps telling me "invalid arguments", also, it doesn't find the program name (doesn't search the PATH variable); I have to give it the full path ..?!
basically, in a batch script I can write:
unison profile
how can I achieve the same effect in python?
EDIT:
I found out it can be done with os.system( ... ) and since I cannot accept my own answer, I'm closing the question.
EDIT: this was supposed to be a comment, but when I posted it I didn't have much points.
Thanks Claudiu, that's pretty much what I want, except for a little thing: I want the function to end when the program exits, but when I try it on unison, it doesn't return control to the python script, but to the windows command line environment
>>> os.execlp("unison")
C:\>Usage: unison [options]
or unison root1 root2 [options]
or unison profilename [options]
For a list of options, type "unison -help".
For a tutorial on basic usage, type "unison -doc tutorial".
For other documentation, type "unison -doc topics".
C:\>
C:\>
C:\>
how to get around this?
You should create a new processess using the subprocess module.
I'm not fluent in windows processes but its Popen function is cross-platform, and should be preffered to OS specific solutions.
EDIT: I maintain that you should prefer the Subprocess module to os.* OS specific functions, it is cross-platform and more pythonic (just google it). You can wait for the result easily, and cleanly:
import os
import subprocess
unison = os.path.join(os.path.curdir, "unison")
p = subprocess.Popen(unison)
p.wait()
I found out that os.system does what I want,
Thanks for all that tried to help.
os.system("dir")
runs the command just as if it was run from a batch file
import subprocess
proc = subprocess.Popen(['unison', 'profile'], stderr=subprocess.PIPE,
stdout=subprocess.PIPE, stdin=subprocess.PIPE)
proc.stdin.write('user input')
print proc.stdout.read()
This should help you get started. Please edit your question with more information if you want a more detailed answer!
os.execlp should work. This will search your path for the command. Don't give it any args if they're not necessary:
>>> import os
>>> os.execlp("cmd")
D:\Documents and Settings\Claudiu>Microsoft Windows XP [Version 5.1.2600]
(C) Copyright 1985-2001 Microsoft Corp.
D:\Documents and Settings\Claudiu>