I want to call a program multiple times from a python code, and save the output of that program in a text file. My first problem right now is just calling the other code. I have to redirect to a different directory and call ./rank on output.txt. This is how Im trying to do it:
TheCommand = "~/src/rank-8-9-2011/rank output.txt"
os.system(TheCommand)
but im getting a parsing error.
[Parsing error on line ]Unknown error: 0
Im running python2.7 on Mac OS 10.5.8. Im not sure what the problem is. I also tried using subprocess:
subprocess.call(["~/src/rank-8-9-2011/rank", "output.txt"])
This does not find the directory (I have a feeling Im using the subprocess incorrectly), but I dont know what is wrong with the os.system.
the name of the program in the first argument to subprocess.Popen must not contain ~ as it doesn't pass the string to the shell for processing (which like always using parameterized queries in sql, protects one from string injection attacks, e.g. if instead of output.text one had ;rm -rf /, the system version would run rank and then run rm -rf . but the subprocess.Popen would only have rank open a file named ;rm -rf .), so one should expand it by calling os.path.expanduser:
subprocess.Popen([os.path.expanduser('~/src/rank-8-9-2011/rank'), "output.txt"])
although it is possible to turn shell processing on by passing shell=True, it is not recommended for the aforementioned reason.
you should try http://docs.python.org/library/os.path.html#os.path.expanduser
import os.path
subprocess.call([os.path.expanduser("~/src/rank-8-9-2011/rank"), "output.txt"])
I'm fairly certain your parsing error is coming from rank, not from your os.system command, as nothing there looks weird. What happens if you run rank manually?
subprocess seems to have a problem with '~', although I'm not immediately sure why. Put the full path and it should work (although you'll likely get that parsing error if it is indeed a problem with rank).
Related
I had this script working for me, before I decided I'm gonna rewrite everything and make it portable.
Without delving too much into the details, there's a central Bash script, which calls 5 other Bash scripts in their own respective folders. I have no intention of porting to Windows anytime soon, as of current this is just for Linux.
The execution path of the central Bash script is:
dos.1/1-init.sh dos.1/
dos.2/1-trace-to-file.sh dos.2/ dos.1/
dos.3/1-recognize-categories.sh dos.3/
dos.4/1-ping-in-groups.sh dos.4/ dos.3/
dos.5/init.sh dos.5/ dos.4/
I run with ./init.sh
Before the script was 'portable' I was using explicit file paths inside each respective script. All was well and good. The program itself is a combination of Bash and Python, and writes to files in one directory, so that they can be manipulated in various ways, before being read back into different parts of the program.
I understand that the fastest way to do this would be to write a monolithic Python script, using subprocess calls for the Bash side of things... However, I am doing it this way to ease maintenance, and (before I started making it 'portable') it was lightning fast.
My issue now is this: each time I have to read text into Python (either from SQL or from file) there's always this added garbage. Up until this point, I have been using sed, awk and Python's .rstrip() function to manage this... Which is all well and good, but this one damn function will not play nice... And I feel there must be a better way.
In bash I call it with:
$prog_dir=$1
$data_dir=$2
$prog_dir/2fast-ping.py $data_dir/group0.txt > $prog_dir/group0_averages.txt
$prog_dir/2fast-ping.py $data_dir/group1.txt > $prog_dir/group1_averages.txt
...
Now I know that I could write to file from within Python, but in this instance I have other reasons not to.
The issue, is that when the 2fast-ping.py script is ran, it reads the text file in with commas and a newline char. I have vigorously checked and I can confirm that the group#.txt files 100% do not contain commas. Here's the Python:
import sys
import subprocess
import select
from concurrent.futures import ThreadPoolExecutor
filename = sys.argv[1]
f = open(filename, "r")
ips = [elem.rstrip('\n') for elem in f]
print(ips)
f.close()
The script goes on to do some work on the IPs afterwards, but this is the painful part. If I call the script direct from CLI: ./2fast-ping.py ../dos.3/group0.txt, the text is processed PROPERLY and the superseding instructions actually function. But, when called from the first init script, the program basically sh*ts itself because each line is read in with commas. It works until the point where it starts to use the processed info, then:
<actual IP would be here>
ping: ('##.###.###.###',): Name or service not known
Of course, the issue is the ('',) But, Python is adding that in, and I don't know how to stop it :(
Any ideas?
Python code was okay, just passing an additional / with the argument :(
I have a a file structure like the following (Windows):
D:\
dir_1\
batch_1.bat
dir_1a\
batch_2.bat
dir_2\
main.py
For the sake of this question, batch_1.bat simply calls batch_2.bat, and looks like:
cd dir_1a
start batch_2.bat %*
Opening batch_1.bat from a command prompt indeed opens batch_2.bat as it's supposed to, and from there on, everything is golden.
Now I want my Python file, D:\dir_2\main.py, to spawn a new process which starts batch_1.bat, which in turn should start batch_2.bat. So I figured the following Python code should work:
import subprocess
subprocess.Popen(['cd "D:/dir_1"', "start batch_1.bat"], shell=True)
This results in "The system cannot find the path specified" being printed to my Python console. (No error is raised, of course.) This is due to the first command. I get the same result even if I cut it down to:
subprocess.Popen(['cd "D:/"'], shell=True)
I also tried starting the batch file directly, like so:
subprocess.Popen("start D:/dir_1/batch_1.bat", shell=True)
For reasons that I don't entirely get, this seems to just open a windows command prompt, in dir_2.
If I forego the start part of this command, then my Python process is going to end up waiting for batch_1 to finish, which I don't want. But it does get a little further:
subprocess.Popen("D:/dir_1/batch_1.bat", shell=True)
This results in batch_1.bat successfully executing... in dir_2, the directory of the Python script, rather than the directory of batch_1.bat, which results in it not being able to find dir_1a\ and hence, batch_2.bat is not executed at all.
I am left highly confused. What am I doing wrong, and what should I be doing instead?
Your question is answered here: Python specify popen working directory via argument
In a nutshell, just pass an optional cwd argument to Popen:
subprocess.Popen(["batch_1.bat"], shell=True, cwd=r'd:\<your path>\dir1')
I am writing a script to open notepad.exe using subprocess.Popen()
import subprocess
command = '%windir%\system32\\notepad.exe'
process = subprocess.Popen(command)
output = process.communicate()
print(output[0])
This throws a FileNotFoundError
Is it possible to change/add to the above code to make it work with relative paths?
I did try to run the script from C:\Windows> after moving it there, which again failed. Also set the shell=True, but failed as well.
Writing a similar script using os.popen() works ok with relative paths, regardless which directory the script is run from, but as far as I understand popen is not the way forward..
Early steps in the world of programming/Python. Any input much appreciated.
Use os.path.expandvars to expand %windir%:
command = os.path.expandvars('%windir%\\system32\\notepad.exe')
The result is a path that then can be passed to subprocess.Popen.
subprocess.Popen does not expand environment variables such as %windir%. The shell might but you really should not depend on shell=True to do that.
Pro tip: whenever you get an error asking the system to execute a command, print the command (and, if applicable, the current working directory). The results will often surprise you.
In your case, I suspect you're just missing a backslash. Use this instead:
command = '%windir%\\system32\\notepad.exe'
Before you make that change, try printing the value of command immediately after assignment. I think you'll find the leading "s" in "system" is missing, and that the mistake is obvious.
HTH.
You could use raw strings to avoid having to double-up your backslashes.
command = r'%windir%\system32\notepad.exe'
So, I'm making a cd burning app and I need to eject the drive to let the user put the disk in. It's a little more complicated, but simplest case I run into is this; I can use cdrecord via the command line to eject the cd tray using this command:
cdrecord --eject dev='/dev/sg1'
which should mean that I can do the same thing with subprocess.call, like this:
subprocess.call(["cdrecord", "--eject", "dev='/dev/sg1'"])
however, when I do that, I get this error:
wodim: No such file or directory.
Cannot open SCSI driver!
For possible targets try 'wodim --devices' or 'wodim -scanbus'.
For possible transport specifiers try 'wodim dev=help'.
For IDE/ATAPI devices configuration, see the file README.ATAPI.setup from
the wodim documentation.
and the tray doesn't open.
This is a very similar error to one I got before when trying to run it form the command line, but I fixed that error by loading the sg kernel module.
If I just run:
subprocess.call(["cdrecord", "--eject"])
it opens the tray just fine. However, this needs to work with possibly multiple cd trays, so that won't work.
How can I get this to eject the cd correctly?
Try this:
subprocess.call(["cdrecord", "--eject", "dev=/dev/sg1"])
The shell will take care of interpreting the quotes, but cdrecord will not.
The only reason you need the quotes in the first place is that the dev path might have spaces in it, causing the shell to split things into separate arguments. For example, if you type this:
cdrecord --eject dev=/dev/my silly cd name
The arguments to cdrecord will be --eject, dev=/dev/my, silly, cd, name. But if you do this:
cdrecord --eject dev='/dev/my silly cd name'
The arguments to cdrecord will be --eject, dev=/dev/my silly cd name.
When you're using subprocess.call, there's no shell to pull the arguments apart; you're passing them explicitly. So, if you do this:
subprocess.call(["cdrecord", "--eject", "dev=/dev/my silly cd name"])
The arguments to cdrecord will be --eject, dev=/dev/my silly cd name.
In some cases—e.g., because you get things in a hopelessly confused state in the first place (e.g., you're reading a config file that's meant to be used by your program or executed by the shell)—you really have no recourse but to run through the shell. If that happens, do this:
subprocess.call("cdrecord --eject dev='/dev/sg1'", shell=True)
But this generally isn't what you want, and it isn't what you want in this case.
You are not using cdrecord but a buggy fork called "wodim" that
might be the reason for your problems.
I recommend you to use recent original software from:
ftp://ftp.berlios.de/pub/cdrecord/alpha/
I'm trying to use python to run a program.
from subprocess import Popen
sa_proc = Popen(['C:\\sa\\sa.exe','--?'])
Running this small snippit gives the error:
WindowsError: [Error 2] The system cannot find the file specified
The program exists and I have copy and pasted directly from explorer the absolute path to the exe. I have tried other things and have found that if I put the EXE in the source folder with the python script and use './sa.exe' then it works. The only thing I can think of is that I'm running the python script (and python) from a separate partition (F:).
Any ideas?
Thanks
As the docs say, "On Windows: the Popen class uses CreateProcess() to execute the child program, which operates on strings. If args is a sequence, it will be converted to a string using the list2cmdline() method.". Maybe that method is messing things up, so why not try the simpler approach of:
sa_proc = Popen('C:\\sa\\sa.exe --?')
If this still fails, then: what's os.environ['COMSPEC'] just before you try this? What happens if you add , shell=True to Popen's arguments?
Edit: turns out apparently to be a case of simple mis-spellling, as 'sa' was actually the program spelled SpamAssassin -- double s twice -- and what the OP was writing was spamassasin -- one double s but a single one the second time.
You may not have permission to execute C:\sa\sa.exe. Have you tried running the program manually?