This question already has answers here:
Why can't I change directories using "cd" in a script?
(33 answers)
Closed 6 years ago.
I am trying to execute some shell commands using python :
The command is cd /home/n1603031f/Desktop/parsec/wd/
It works fine through the shell, but when executed through python it does not work :
path_to_wd = "/home/n1603031f/Desktop/parsec/wd/"
call(["cd",path_to_wd])
Error :
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 522, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 710, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
I need this command to work as the original command I want to execute is :
cd ./parsec/wd/ && tar -cf ../abcd.tar *
which works correctly only when you change directories to not create the top-level folders in the .tar file
Even if you had the right call to change directories it wouldn't accomplish what you want because each subprocess.call creates a separate process.
What you really need is the cwd argument to subprocess.Popen to say what directory you want to work in. Additionally you need to use os.listdir since the subprocess call won't go through a shell to expand the * glob. This is the right way to do what you're trying to do:
d = './parsec/wd'
subprocess.Popen(['tar', '-cf', '../abcd.tar'] + os.listdir(d), cwd=d).wait()
However, os.listdir will list hidden files as well, if you want to you can filter them out beforehand:
files = [f for f in os.listdir(d) if not f.startswith('.')]
If you really need to (and you don't,) you can use shell=True to get this to work with *. Though unless you're working with trusted input shell=True is widely considered a security vulnerability.
subprocess.Popen('tar -cf ../abcd.tar *', shell=True, cwd='./parsec/wd').wait()
If you need your python process to change it's current working directory, use
os.chdir('./parsec/wd')
Related
This question already has answers here:
Calling the "source" command from subprocess.Popen
(9 answers)
Closed 3 years ago.
In python2.7 we can execute external linux commands using subprocess package.
import subprocess
subprocess.call(["ls", "-l"]) // or
subprocess.call("ls -l".split())
both works. I have a file test.sh in current work directory which contains just
date
so I tried
>>> subprocess.call("pwd".split())
/home/ckim
0
>>> subprocess.call("cat test.sh".split())
date
0
>>> subprocess.call("source test.sh".split())
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/usr/lib/python2.7/subprocess.py", line 523, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib/python2.7/subprocess.py", line 1343, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
What's wrong?
ADD(ANSWER) : This question and answer has enough information (but I'll leave my question here..) Calling the "source" command from subprocess.Popen
source is a shell builtin command. Python's subprocess module is provided to spawn new process, not running a proper Bourne shell (or zsh or ksh or etc.). You cannot access shel builtins from subprocess.call.
To determine if you can or not run a specific command with subprocess module, you may want to use which to get information about the command you need to use:
user#machine: ~
$ which source [7:41:38]
source: shell built-in command
user#machine: ~
$ which cat [7:41:42]
/bin/cat
user#machine: ~
$ which ls [7:41:47]
ls: aliased to ls --color=tty
This question already has answers here:
subprocess.call
(3 answers)
Closed 4 years ago.
I wanted to create some folders with subfolders and have this python3 script that would read a csv file and create directories for me.
from subprocess import call
import csv
def csv_dict_reader(file_obj):
"""
Read a CSV file using csv.DictReader
"""
reader = csv.DictReader(file_obj, delimiter=',')
for line in reader:
sanitized_name = line["course"].replace(" ", "-").lower()
command = 'mkdir -p ' + sanitized_name + '/week_{1..'+ line["week"] +'}/{slides,exercise,assignment}'
call(command)
if __name__ == "__main__":
with open("courses.csv") as f_obj:
csv_dict_reader(f_obj)
I was expecting that it executes commands of this sort
mkdir -p algorithmic-toolbox/week_{1..6}/{slides,exercise,assignment}
which works fine from the terminal and creates a folder called algorithmic-toolbox with 6 folders from week1 to week6 and then each folder contains slides, exercises and assignment folders.
A sample csv would be like this:
course,week
Algorithmic Toolbox,6
Data Structures,4
Algorithms on Graphs,5
I am getting this error when I run the python script.
Traceback (most recent call last):
File "course-folder.py", line 18, in <module>
csv_dict_reader(f_obj)
File "course-folder.py", line 13, in csv_dict_reader
call(command)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 267, in call
with Popen(*popenargs, **kwargs) as p:
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 709, in __init__
restore_signals, start_new_session)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/subprocess.py", line 1344, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'mkdir -p algorithmic-toolbox/week_{1..6}/{slides,exercise,assignment}': 'mkdir -p algorithmic-toolbox/week_{1..6}/{slides,exercise,assignment}'
Not sure why it says FilenotFoundError when I am creating directories with -p which is supposed to create the parent directories if they don't exist.
mkdir -p algorithmic-toolbox/week_{1..6}/{slides,exercise,assignment}
I believe parameter expansion used in the above command will not work without passing the shell=True argument to subprocess.call. This way Python will execute the command in an intermediate shell process.
If you want your script to be platform independent you could instead use os.mkdir and handle the logic in Python.
This question already has answers here:
"OSError: [Errno 2] No such file or directory" while using python subprocess with command and arguments
(3 answers)
Closed 4 years ago.
If I run this
test -d a
in Linux command line, it returns nothing and prints no error even if directory a doesn't exist, because the result is returned via exit code.
But if I want to grab this exit code with Python, I get
import subprocess
subprocess.call('test -d a')
Traceback (most recent call last):
File "<input>", line 1, in <module>
File "/opt/anaconda3/envs/python27/lib/python2.7/subprocess.py", line 168, in call
return Popen(*popenargs, **kwargs).wait()
File "/opt/anaconda3/envs/python27/lib/python2.7/subprocess.py", line 390, in __init__
errread, errwrite)
File "/opt/anaconda3/envs/python27/lib/python2.7/subprocess.py", line 1025, in _execute_child
raise child_exception
OSError: [Errno 2] No such file or directory
How to overcome this? Changing command text is not allowed, anly python caller code should change. It shoudl return 0 or 1 depending on directory existence as described in manual.
Your issue might be that Python cannot find test -d a rather than test failing and subprocess.call identifying the error and rasing an seemingly appropriate OSError exception. See the exceptions region of the subprocess.call/Popen docs. The right way to call it would be subprocess.call(['test', '-d', 'a'])
This question already has answers here:
Can't execute shell script from python subprocess: permission denied
(2 answers)
Closed 6 years ago.
Currently I am testing a very simple piece of code. I simply want to write a python script that sets a variable and then pass that variable into a bash script for use.
Python Script:
from subprocess import check_call
a = str(3)
check_call(["/home/desktop/bash2pyTest/test.sh", a], shell=False)
Bash Script:
#!/bin/bash
echo "This number was sent from the py script: " $1
I have read other Q&As that are related to this topic; however, I am not finding a solution that I am conceptually understand; thus, the syntax above might be incorrect. I have tried a few other methods as well; however, I keep receiving the following error:
Traceback (most recent call last):
File "/home/cassandra/desktop/bash2pyTest/callVar.py", line 3, in <module>
check_call(["/home/cassandra/desktop/bash2pyTest/test.sh", a], shell=False)
File "/usr/lib64/python2.7/subprocess.py", line 537, in check_call
retcode = call(*popenargs, **kwargs)
File "/usr/lib64/python2.7/subprocess.py", line 524, in call
return Popen(*popenargs, **kwargs).wait()
File "/usr/lib64/python2.7/subprocess.py", line 711, in __init__
errread, errwrite)
File "/usr/lib64/python2.7/subprocess.py", line 1327, in _execute_child
raise child_exception
OSError: [Errno 13] Permission denied
Process finished with exit code 1
Any help would be greatly appreciated. I'm stumped.
Try
chmod +x /home/desktop/bash2pyTest/test.sh
in shell. The file you are trying to execute is not executable.
Or another option in python script:
check_call(["sh","/home/desktop/bash2pyTest/test.sh", a], shell=False)
The error says that permission is denied. I tested your code, and it works fine on my machine. However, you will need to be sure that the user running the command has sufficient privileges for the test.sh script. Most importantly, be sure that test.sh has execute permissions set. That's often the most easily missed permission.
I am trying to automate some tasks - the process requires I call some exe's and pass parameters. The particular directories for the exe's are in the PATH variable for windows. However, I consistently get a
WindowsError: [Error 2] The system cannot find the file specified
My current workaround is to set the os.cwd to the directory with the exe but that imposes some other limits on how we distribute the code. I want to note that in every case if I start a cmd window and type the same code I am passing to subprocess.check_output the code works no matter what directory I am in on the computer.
Just to be clear I am afraid for example of trying to automate a WinRAR task and WinRAR.exe is in a different folder on their computer.
Okay in response to the comment below here is the input and the output after I changed the cwd to root (c:)
The call to subprocess
rarProcess = check_output('''WinRAR a -r -v700m -sfx -agYYYYMMDD-NN -iiconD:\\RarResources\\de96.ico -iimgd:\\RarResources\\accounting2013.bmp d:\\testFTP\\compressed_test_ d:\\files_to_compress''')
and here is the Traceback message in all of it's glory
Traceback (most recent call last):
File "<pyshell#93>", line 1, in <module>
rarProcess = check_output('''WinRAR a -r -v700m -sfx -agYYYYMMDD-NN -iiconD:\\RarResources\\de96.ico -iimgd:\\RarResources\\accounting2013.bmp d:\\testFTP\\compressed_test_ d:\\files_to_compress''')
File "C:\Program Files (x86)\python\lib\subprocess.py", line 537, in check_output
process = Popen(stdout=PIPE, *popenargs, **kwargs)
File "C:\Program Files (x86)\python\lib\subprocess.py", line 679, in __init__
errread, errwrite)
File "C:\Program Files (x86)\python\lib\subprocess.py", line 893, in _execute_child
startupinfo)
WindowsError: [Error 2] The system cannot find the file specified
Now I can't prove that this is not a hypothetical question/problem. I get the intended results when I use the same command (adjusting for path separators) through the cmd window and if I change the directory to the directory with the exe before running the command as pasted above.
You don't need to set os.cwd and run the process. Instead you pass the location of your "Winrar.exe" file to the subprocess as a dict.
proc = subprocess.Popen(args, env={'PATH': '/path/to/winrar.exe'})