Sorry if this question is dumb. I am using python subprocess statement to call a .bat file in Ubuntu (Natty 11.04), however, I got error messages:
Traceback (most recent call last):
File "pfam_picloud.py", line 40, in <module>
a=subprocess.Popen(src2, shell=0)
File "/usr/lib/python2.7/subprocess.py", line 672, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1213, in _execute_child
raise child_exception
run this python file
$python pfam_picloud.py
Python code (pfam_picloud.py)
#!/usr/bin/python
#
met="wTest.dvf"
run="run_pfam.bat"
inp="pfam_input.PFA"
import os
import stat
import shutil
import subprocess
import string
import random
# Generate a random ID for file save
def id_generator(size=6, chars=string.ascii_uppercase + string.digits):
return ''.join(random.choice(chars) for x in range(size))
name_temp=id_generator()
cwd=os.getcwd()
src=cwd
src1=cwd+'/'+name_temp
if not os.path.exists(src1):
os.makedirs(src1)
else:
shutil.rmtree(src1)
os.makedirs(src1)
##
shutil.copy(src+"/"+run,src1)
shutil.copy(src+"/"+met,src1)
shutil.copy(cwd+"/pfam_pi.exe",src1)
shutil.copy(src+"/"+inp,src1)
#
src2=src1+"/run_pfam.bat"
os.chdir(src1)
a=subprocess.Popen(src2, shell=0)
a.wait()
bash file (run_pfam.bat)
#!/bin/sh
./pfam_pi.exe pfam_input.PFA
I can successfully run this bash file in Ubuntu. So I guess, I messed up something in my Python script. Could anyone give me some suggestions? Thanks for any inputs.
EDIT
the file pfam_pi.exe is a Linux executable. I compiled it in Ubuntu. Sorry for the confusion.
update
Well, I got different types of error now.
1. With #!/bin/sh, it said No such file or directory.
2. With /bin/sh, it said exec format error.
3. If I sent everything as arguments a=subprocess.Popen(['./pfam_pi.exe', 'inp', 'src1'], shell=0), it said end of line symbol error
Since feature requests to mark a comment as an answer remain declined, I copy the above solution here.
#Ellioh: Thanks for your comments. I found once I changed the shell=1, problem is solved. – tao.hong
Try running wine (you should have it installed) and pass pfam_pi.exe to it as a parameter. Maybe pfam_pi.exe is not a Linux executable. :-) Certainly, executable file extensions are not meaningful on Linux, but probably it really is a Windows program, otherwise I hardly can imagine it named pfam_pi.exe.
However, if it is a Linux executable, note subprocess.Popen accepts a list of args (the first element is the program itself), not a command line:
>>> import shlex, subprocess
>>> command_line = raw_input()
/bin/vikings -input eggs.txt -output "spam spam.txt" -cmd "echo '$MONEY'"
>>> args = shlex.split(command_line)
>>> print args
['/bin/vikings', '-input', 'eggs.txt', '-output', 'spam spam.txt', '-cmd', "echo '$MONEY'"]
>>> p = subprocess.Popen(args) # Success!
Related
I want to execute a script inside a subdirectory/superdirectory (I need to be inside this sub/super-directory first). I can't get subprocess to enter my subdirectory:
tducin#localhost:~/Projekty/tests/ve$ python
Python 2.7.4 (default, Sep 26 2013, 03:20:26)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> import os
>>> os.getcwd()
'/home/tducin/Projekty/tests/ve'
>>> subprocess.call(['cd ..'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Python throws OSError and I don't know why. It doesn't matter whether I try to go into an existing subdir or go one directory up (as above) - I always end up with the same error.
What your code tries to do is call a program named cd ... What you want is call a command named cd.
But cd is a shell internal. So you can only call it as
subprocess.call('cd ..', shell=True) # pointless code! See text below.
But it is pointless to do so. As no process can change another process's working directory (again, at least on a UNIX-like OS, but as well on Windows), this call will have the subshell change its dir and exit immediately.
What you want can be achieved with os.chdir() or with the subprocess named parameter cwd which changes the working directory immediately before executing a subprocess.
For example, to execute ls in the root directory, you either can do
wd = os.getcwd()
os.chdir("/")
subprocess.Popen("ls")
os.chdir(wd)
or simply
subprocess.Popen("ls", cwd="/")
To run your_command as a subprocess in a different directory, pass cwd parameter, as suggested in #wim's answer:
import subprocess
subprocess.check_call(['your_command', 'arg 1', 'arg 2'], cwd=working_dir)
A child process can't change its parent's working directory (normally). Running cd .. in a child shell process using subprocess won't change your parent Python script's working directory i.e., the code example in #glglgl's answer is wrong. cd is a shell builtin (not a separate executable), it can change the directory only in the same process.
subprocess.call and other methods in the subprocess module have a cwd parameter.
This parameter determines the working directory where you want to execute your process.
So you can do something like this:
subprocess.call('ls', shell=True, cwd='path/to/wanted/dir/')
Check out docs subprocess.popen-constructor
You want to use an absolute path to the executable, and use the cwd kwarg of Popen to set the working directory. See the docs.
If cwd is not None, the child’s current directory will be changed to
cwd before it is executed. Note that this directory is not considered
when searching the executable, so you can’t specify the program’s path
relative to cwd.
I guess these days you would do:
import subprocess
subprocess.run(["pwd"], cwd="sub-dir")
Another option based on this answer: https://stackoverflow.com/a/29269316/451710
This allows you to execute multiple commands (e.g cd) in the same process.
import subprocess
commands = '''
pwd
cd some-directory
pwd
cd another-directory
pwd
'''
process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = process.communicate(commands.encode('utf-8'))
print(out.decode('utf-8'))
just use os.chdir
Example:
>>> import os
>>> import subprocess
>>> # Lets Just Say WE want To List The User Folders
>>> os.chdir("/home/")
>>> subprocess.run("ls")
user1 user2 user3 user4
If you want to have cd functionality (assuming shell=True) and still want to change the directory in terms of the Python script, this code will allow 'cd' commands to work.
import subprocess
import os
def cd(cmd):
#cmd is expected to be something like "cd [place]"
cmd = cmd + " && pwd" # add the pwd command to run after, this will get our directory after running cd
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) # run our new command
out = p.stdout.read()
err = p.stderr.read()
# read our output
if out != "":
print(out)
os.chdir(out[0:len(out) - 1]) # if we did get a directory, go to there while ignoring the newline
if err != "":
print(err) # if that directory doesn't exist, bash/sh/whatever env will complain for us, so we can just use that
return
If you need to change directory, run a command and get the std output as well:
import os
import logging as log
from subprocess import check_output, CalledProcessError, STDOUT
log.basicConfig(level=log.DEBUG)
def cmd_std_output(cd_dir_path, cmd):
cmd_to_list = cmd.split(" ")
try:
if cd_dir_path:
os.chdir(os.path.abspath(cd_dir_path))
output = check_output(cmd_to_list, stderr=STDOUT).decode()
return output
except CalledProcessError as e:
log.error('e: {}'.format(e))
def get_last_commit_cc_cluster():
cd_dir_path = "/repos/cc_manager/cc_cluster"
cmd = "git log --name-status HEAD^..HEAD --date=iso"
result = cmd_std_output(cd_dir_path, cmd)
return result
log.debug("Output: {}".format(get_last_commit_cc_cluster()))
Output: "commit 3b3daaaaaaaa2bb0fc4f1953af149fa3921e\nAuthor: user1<user1#email.com>\nDate: 2020-04-23 09:58:49 +0200\n\n
I have an applescript that takes in two parameters on execution.
on run {targetBuddyPhone, targetMessage}
tell application "Messages"
set targetService to 1st service whose service type = iMessage
set targetBuddy to buddy targetBuddyPhone of targetService
send targetMessage to targetBuddy
end tell
end run
I then want this script to execute from within a python script. I know how to execute a applescript from python, but how do I also give it arguments? Here is the python script that I currently have written out.
#!/usr/bin/env python3
import subprocess
def run_applescript(script, *args):
p = subprocess.Popen(['arch', '-i386', 'osascript', '-e', script] +
[unicode(arg).encode('utf8') for arg in args],
stdout=subprocess.PIPE, stderr=subprocess.PIPE)
err = p.wait()
if err:
raise RuntimeError(err, p.stderr.read()[:-1].decode('utf8'))
return p.stdout.read()[:-1].decode('utf8')
The error I receive after trying to execute this code in the terminal is:
Traceback (most recent call last):
File "messageExecuter.py", line 14, in <module>
run_applescript("sendMessage.scpt",1111111111,"hello")
File "messageExecuter.py", line 11, in run_applescript
raise RuntimeError(err, p.stderr.read()[:-1].decode('utf8'))
RuntimeError: (1, u'arch: posix_spawnp: osascript: Bad CPU type in executable')
Clue is in the error message. Delete 'arch', '-i386' from arguments list, as osascript is 64-bit only.
This question already has answers here:
subprocess.Popen stdin read file
(3 answers)
Closed 6 years ago.
for a little project of my own, I am trying to write a program that prints out the contents of a file on the computers default printer.
I know theres alot of similar questions around, but none of them works on my pc (Linux mint 17.3)
here is one that I tried, it got the closest to what i needed:
from subprocess import Popen
from cStringIO import StringIO
# place the output in a file like object
sio = StringIO("test.txt")
# call the system's lpr command
p = Popen(["lpr"], stdin=sio, shell=True)
output = p.communicate()[0]
this gives me the following error:
Traceback (most recent call last):
File "/home/vandeventer/x.py", line 8, in <module>
p = Popen(["lpr"], stdin=sio, shell=True)
File "/usr/lib/python2.7/subprocess.py", line 702, in __init__
errread, errwrite), to_close = self._get_handles(stdin, stdout, stderr)
File "/usr/lib/python2.7/subprocess.py", line 1117, in _get_handles
p2cread = stdin.fileno()
AttributeError: 'cStringIO.StringI' object has no attribute 'fileno'
Doe anyone out there know hoe one could implement this in python? it really does not have to work on windows
Regards
Cid-El
you don't have to use StringIO for this. Just use the pipe feature of subprocess and write your data to p.stdin:
from subprocess import Popen
# call the system's lpr command
p = Popen(["lpr"], stdin=subprocess.PIPE, shell=True) # not sure you need shell=True for a simple command
p.stdin.write("test.txt")
output = p.communicate()[0]
as a bonus, this is Python 3 compliant (StringIO has been renamed since :))
BUT: that would just print a big white page with one line: test.txt. lpr reads standard input and prints it (that's still an interesting piece of code :))
To print the contents of your file you have to read it, and in that case it's even simpler since pipe & files work right away together:
from subprocess import Popen
with open("test.txt") as f:
# call the system's lpr command
p = Popen(["lpr"], stdin=f, shell=True) # not sure you need shell=True for a simple command
output = p.communicate()[0]
I want to execute a script inside a subdirectory/superdirectory (I need to be inside this sub/super-directory first). I can't get subprocess to enter my subdirectory:
tducin#localhost:~/Projekty/tests/ve$ python
Python 2.7.4 (default, Sep 26 2013, 03:20:26)
[GCC 4.7.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import subprocess
>>> import os
>>> os.getcwd()
'/home/tducin/Projekty/tests/ve'
>>> subprocess.call(['cd ..'])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1308, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
Python throws OSError and I don't know why. It doesn't matter whether I try to go into an existing subdir or go one directory up (as above) - I always end up with the same error.
What your code tries to do is call a program named cd ... What you want is call a command named cd.
But cd is a shell internal. So you can only call it as
subprocess.call('cd ..', shell=True) # pointless code! See text below.
But it is pointless to do so. As no process can change another process's working directory (again, at least on a UNIX-like OS, but as well on Windows), this call will have the subshell change its dir and exit immediately.
What you want can be achieved with os.chdir() or with the subprocess named parameter cwd which changes the working directory immediately before executing a subprocess.
For example, to execute ls in the root directory, you either can do
wd = os.getcwd()
os.chdir("/")
subprocess.Popen("ls")
os.chdir(wd)
or simply
subprocess.Popen("ls", cwd="/")
To run your_command as a subprocess in a different directory, pass cwd parameter, as suggested in #wim's answer:
import subprocess
subprocess.check_call(['your_command', 'arg 1', 'arg 2'], cwd=working_dir)
A child process can't change its parent's working directory (normally). Running cd .. in a child shell process using subprocess won't change your parent Python script's working directory i.e., the code example in #glglgl's answer is wrong. cd is a shell builtin (not a separate executable), it can change the directory only in the same process.
subprocess.call and other methods in the subprocess module have a cwd parameter.
This parameter determines the working directory where you want to execute your process.
So you can do something like this:
subprocess.call('ls', shell=True, cwd='path/to/wanted/dir/')
Check out docs subprocess.popen-constructor
You want to use an absolute path to the executable, and use the cwd kwarg of Popen to set the working directory. See the docs.
If cwd is not None, the child’s current directory will be changed to
cwd before it is executed. Note that this directory is not considered
when searching the executable, so you can’t specify the program’s path
relative to cwd.
I guess these days you would do:
import subprocess
subprocess.run(["pwd"], cwd="sub-dir")
Another option based on this answer: https://stackoverflow.com/a/29269316/451710
This allows you to execute multiple commands (e.g cd) in the same process.
import subprocess
commands = '''
pwd
cd some-directory
pwd
cd another-directory
pwd
'''
process = subprocess.Popen('/bin/bash', stdin=subprocess.PIPE, stdout=subprocess.PIPE)
out, err = process.communicate(commands.encode('utf-8'))
print(out.decode('utf-8'))
just use os.chdir
Example:
>>> import os
>>> import subprocess
>>> # Lets Just Say WE want To List The User Folders
>>> os.chdir("/home/")
>>> subprocess.run("ls")
user1 user2 user3 user4
If you want to have cd functionality (assuming shell=True) and still want to change the directory in terms of the Python script, this code will allow 'cd' commands to work.
import subprocess
import os
def cd(cmd):
#cmd is expected to be something like "cd [place]"
cmd = cmd + " && pwd" # add the pwd command to run after, this will get our directory after running cd
p = subprocess.Popen(cmd, shell=True, stdout=subprocess.PIPE, stderr=subprocess.PIPE, universal_newlines=True) # run our new command
out = p.stdout.read()
err = p.stderr.read()
# read our output
if out != "":
print(out)
os.chdir(out[0:len(out) - 1]) # if we did get a directory, go to there while ignoring the newline
if err != "":
print(err) # if that directory doesn't exist, bash/sh/whatever env will complain for us, so we can just use that
return
If you need to change directory, run a command and get the std output as well:
import os
import logging as log
from subprocess import check_output, CalledProcessError, STDOUT
log.basicConfig(level=log.DEBUG)
def cmd_std_output(cd_dir_path, cmd):
cmd_to_list = cmd.split(" ")
try:
if cd_dir_path:
os.chdir(os.path.abspath(cd_dir_path))
output = check_output(cmd_to_list, stderr=STDOUT).decode()
return output
except CalledProcessError as e:
log.error('e: {}'.format(e))
def get_last_commit_cc_cluster():
cd_dir_path = "/repos/cc_manager/cc_cluster"
cmd = "git log --name-status HEAD^..HEAD --date=iso"
result = cmd_std_output(cd_dir_path, cmd)
return result
log.debug("Output: {}".format(get_last_commit_cc_cluster()))
Output: "commit 3b3daaaaaaaa2bb0fc4f1953af149fa3921e\nAuthor: user1<user1#email.com>\nDate: 2020-04-23 09:58:49 +0200\n\n
I am trying to use bash functions inside my python script to allow me to locate a specific directory and then grep a given file inside the directory. The catch is that I only have part of the directory name, so I need to use the bash function find to get the rest of the directory name (names are unique and will only ever return one folder)
The code I have so far is as follows:
def get_tag(part_of_foldername):
import subprocess
import os
p1 = subprocess.Popen(["find", "/path/to/directory", "-maxdepth", "1", "-name", "%s.*" % part_of_foldername, "-type", "d"], stdout=subprocess.PIPE)
directory = p1.communicate()[0].strip('\n')
os.chdir(directory)
p2 = subprocess.Popen(["grep", "STUFF_", ".hgtags"], stdout=subprocess.PIPE)
tag = p2.comminucate()[0].strip('\n')
return tag
Here is what's really strange. This code works when you enter it line by line into interactive, but not when it's run thru a script. It also works when you import the script file into interactive and call the function, but not when it's called by the main function. The traceback I get from running the script straight is as follows:
Traceback (most recent call last):
File "./integration.py", line 64, in <module>
main()
File "./integration.py", line 48, in main
tag = get_tag(folder)
File "./integration.py", line 9, in get_date
os.chdir(directory)
OSError: [Errno 2] No such file or directory: ''
And it's called in the main function like this:
if block_dict[block][0]=='0':
tag = get_tag(folder)
with "folder" being previously defined as a string.
Please note we use python 2.6 so I can't use the module check_output unfortunately.
Have you tried using the glob module as opposed to find?
import glob
glob.glob("/path/to/directory/*/SomeDir/path/*")
You can look past multiple dirctories using **:
glob.glob("/path/**/SomeDir/path/*")
and that would match /path/to/your/SomeDir/path/file.
evidently p1.communicate()[0].strip('\n') is returning an empty string. are you really using the hardcoded value "/path/to/directory" as in your example?
Check the result of p1.communicate()[0]. It maybe empty string.
. in "%s.*" % part_of_foldername seems to be the cause.
UPDATE
Found typo: comminucate -> comminucate
def get_tag(part_of_foldername):
p1 = subprocess.Popen(["find", "/path/to/directory", "-maxdepth", "1", "-name", "*%s*" % part_of_foldername, "-type", "d"], stdout=subprocess.PIPE)
out, err = p1.communicate()
directory = out.split('\n')[0]
p1.wait()
if directory:
os.chdir(directory)
p2 = subprocess.Popen(["grep", "STUFF_", ".hgtags"], stdout=subprocess.PIPE)
out, err = p2.communicate()
p2.wait()
return out.rstrip('\n')