How do you specify what directory the python module subprocess uses to execute commands? I want to run a C program and process it's output in Python in near realtime.
I want to excute the command (this runs my makefile for my C program:
make run_pci
In:
/home/ecorbett/hello_world_pthread
I also want to process the output from the C program in Python. I'm new to Python so example code would be greatly appreciated!
Thanks,
Edward
Use the cwd argument to Popen, call, check_call or check_output. E.g.
subprocess.check_output(["make", "run_pci"],
cwd="/home/ecorbett/hello_world_pthread")
Edit: ok, now I understand what you mean by "near realtime". Try
p = subprocess.Popen(["make", "run_pci"],
stdout=subprocess.PIPE,
cwd="/home/ecorbett/hello_world_pthread")
for ln in p.stdout:
# do processing
Read the documentation. You can specify it with the cwd argument, otherwise it uses the current directory of the parent (i.e., the script running the subprocess).
Related
I am using Ubuntu and I am running a command using the subprocess module. I am trying to find the maximum number of days a password can be use.
import subprocess
pass_max = subprocess.check_output('grep PASS_MAX_DAYS /etc/login.defs')
print(pass_max)
After running this code, I receive the error no such file or directory. How am I able to find the maximum number of days a password can be use?
check_output expects the command as a list:
subprocess.check_output(['grep', 'PASS_MAX_DAYS', '/etc/login.defs'])
Alternative is to pass shell=True, taking into account the security considerations
subprocess.check_output('grep PASS_MAX_DAYS /etc/login.defs', shell=True)
grep PASS_MAX_DAYS /etc/login.defs is being interpreted as a single executable, which can't be found. Use an array to pass an executable with arguments.
subprocess.check_output(['grep', 'PASS_MAX_DAYS', '/etc/login.defs'])
the argument of the function check_output has to be a list, so just add split at the end of your command string
import subprocess
pass_max = subprocess.check_output('grep PASS_MAX_DAYS /etc/login.defs'.split())
print(pass_max)
That should work
try this -
shell_output
By using the above you should be able to overcome all the challenges that otherwise appear with a subprocess module
Usage -
print(shell_output("your shell command here"))
So I use a program DSI Studio and I am doing a lot of repetitive tasks that I would like to automate. It has a command line interface that works for me with the command
dsi_studio --action=trk -source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --
seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track5.trk
it does exactly what I want and outputs a file.
However when I try
import subprocess
subprocess.call("dsi_studio --action=trk --source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --fa_threshold=0.00000 --turning_angle=70 --step_size=0.01 --smoothing=0 --min_length=0.0 --max_length=300.0 --initial_dir=0 --seed_plan=0 --interpolation=0 --thread_count=12 --seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track4.trk", shell=True)
I get back return code 1. The same thing happens if I use subprocess.run. I have dinked around with different permutations to no avail. the only thing I have managed to get a 0 return code from is
subprocess.call('cd /d G:\Programs\dsi_studio_64', shell=True)
which I attempted because that is the directory I need to be in for the command to work in cmd. but even after doing that it still doesn't work. I'm pretty novice at python and I have spent a few days reading questions like mine but when I try to implement their solutions by template matching I have had no luck.
Each subprocess call makes its own shell, so your cd isn't actually affecting your later call, which is then breaking because you're in the wrong directory. Try
os.chdir("G:\Programs\dsi_studio_64")
subprocess.call("dsi_studio --action=trk --source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --fa_threshold=0.00000 --turning_angle=70 --step_size=0.01 --smoothing=0 --min_length=0.0 --max_length=300.0 --initial_dir=0 --seed_plan=0 --interpolation=0 --thread_count=12 --seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track4.trk", shell=True)
you can also do it with the cwd argument to call(), like
subprocess.call("your long command", cwd="directory")
You can use subprocess.Popen.
#!/usr/bin/env python
# -*- coding: utf-8 -*-
import subprocess
def run_process(exe):
'Define a function for running commands and capturing stdout line by line'
p = subprocess.Popen(exe.split(), stdout=subprocess.PIPE, stderr=subprocess.STDOUT)
return iter(p.stdout.readline, b'')
if __name__ == '__main__':
for line in run_process("G:\Programs\dsi_studio_64\dsi_studio --action=trk --source=HarveyReg2.hdr.src.gz.odf4.f5rec.012fx.rdi.gqi.0.2.fib.gz --method=0 --fa_threshold=0.00000 --turning_angle=70 --step_size=0.01 --smoothing=0 --min_length=0.0 --max_length=300.0 --initial_dir=0 --seed_plan=0 --interpolation=0 --thread_count=12 --seed=leftprechiasm.nii.gz --roi=1.nii.gz --fiber_count=100 --output=track4.trk"):
print(line)
My perl script is at path:
a/perl/perlScript.pl
my python script is at path:
a/python/pythonScript.py
pythonScript.py gets an argument from stdin, and returns result to stdout. From perlScript.pl , I want to run pythonScript.py with the argument hi to stdin, and save the results in some variable. That's what I tried:
my $ret = `../python/pythonScript.py < hi`;
but I got the following error:
The system cannot find the path specified.
Can you explain the path can't be found?
The qx operator (backticks) starts a shell (sh), in which prog < input syntax expects a file named input from which it will read lines and feed them to the program prog. But you want the python script to receive on its STDIN the string hi instead, not lines of a file named hi.
One way is to directly do that, my $ret = qx(echo "hi" | python_script).
But I'd suggest to consider using modules for this. Here is a simple example with IPC::Run3
use warnings;
use strict;
use feature 'say';
use IPC::Run3;
my #cmd = ('program', 'arg1', 'arg2');
my $in = "hi";
run3 \#cmd, \$in, \my $out;
say "script's stdout: $out";
The program is the path to your script if it is executable, or perhaps python script.py. This will be run by system so the output is obtained once that completes, what is consistent with the attempt in the question. See documentation for module's operation.
This module is intended to be simple while "satisfy 99% of the need for using system, qx, and open3 [...]. For far more power and control see IPC::Run.
You're getting this error because you're using shell redirection instead of just passing an argument
../python/pythonScript.py < hi
tells your shell to read input from a file called hi in the current directory, rather than using it as an argument. What you mean to do is
my $ret = `../python/pythonScript.py hi`;
Which correctly executes your python script with the hi argument, and returns the result to the variable $ret.
The Some of the other answers assume that hi must be passed as a command line parameter to the Python script but the asker says it comes from stdin.
Thus:
my $ret = `echo "hi" | ../python/pythonScript.py`;
To launch your external script you can do
system "python ../python/pythonScript.py hi";
and then in your python script
import sys
def yourFct(a, b):
...
if __name__== "__main__":
yourFct(sys.argv[1])
you can have more informations on the python part here
I have a a file structure like the following (Windows):
D:\
dir_1\
batch_1.bat
dir_1a\
batch_2.bat
dir_2\
main.py
For the sake of this question, batch_1.bat simply calls batch_2.bat, and looks like:
cd dir_1a
start batch_2.bat %*
Opening batch_1.bat from a command prompt indeed opens batch_2.bat as it's supposed to, and from there on, everything is golden.
Now I want my Python file, D:\dir_2\main.py, to spawn a new process which starts batch_1.bat, which in turn should start batch_2.bat. So I figured the following Python code should work:
import subprocess
subprocess.Popen(['cd "D:/dir_1"', "start batch_1.bat"], shell=True)
This results in "The system cannot find the path specified" being printed to my Python console. (No error is raised, of course.) This is due to the first command. I get the same result even if I cut it down to:
subprocess.Popen(['cd "D:/"'], shell=True)
I also tried starting the batch file directly, like so:
subprocess.Popen("start D:/dir_1/batch_1.bat", shell=True)
For reasons that I don't entirely get, this seems to just open a windows command prompt, in dir_2.
If I forego the start part of this command, then my Python process is going to end up waiting for batch_1 to finish, which I don't want. But it does get a little further:
subprocess.Popen("D:/dir_1/batch_1.bat", shell=True)
This results in batch_1.bat successfully executing... in dir_2, the directory of the Python script, rather than the directory of batch_1.bat, which results in it not being able to find dir_1a\ and hence, batch_2.bat is not executed at all.
I am left highly confused. What am I doing wrong, and what should I be doing instead?
Your question is answered here: Python specify popen working directory via argument
In a nutshell, just pass an optional cwd argument to Popen:
subprocess.Popen(["batch_1.bat"], shell=True, cwd=r'd:\<your path>\dir1')
I know there are some topic on Stack Overflow about this. But none of these make any sense to me. I am new to both python and perl and trying my best to understand. I would like to run a perl script from a piece of python code.
executing the perl script in command prompt goes as following:
perl perlscript.pl input.bopt7 output.xml
I would like to run this command from my python code.
I have tried the following:
pipe = subprocess.Popen(["perlscript.pl" , "input.bopt7" , "output.xml"], stdout=subprocess.PIPE)
but this does not work. I get an error saying it is not a valid win32 ...
I need no input or output from this script. Just need to run it once.
You need to include the perl command itself when executing a perl script:
pipe = subprocess.Popen(["perl", "perlscript.pl" , "input.bopt7" , "output.xml"], stdout=subprocess.PIPE)
You did the same thing on the command line prompt; the Popen class cannot guess from the perlscript.pl file that you wanted to run this script with Perl. :-)
Did you try to add perl to Popen arguments (just as you do on the command line)?
pipe = subprocess.Popen(["perl", "perlscript.pl" , "input.bopt7" , "output.xml"], stdout=subprocess.PIPE)
In your example, Windows tries to execute "perlscript.pl" as a Win32 executable, since this is the first parameter you specified, and fails because it doesn't contain the proper binary header (since it is a text file).
The first argument should be perl.exe, if perl.exe is in your PATH; or the full path to the executable as all the rest are arguments to perl.exe.
Also make sure you put the full path for perlscript.pl and input.bopt7.