I'm trying to get the filename thats given in the command line. For example:
python3 ritwc.py < DarkAndStormyNight.txt
I'm trying to get DarkAndStormyNight.txt
When I try fileinput.filename() I get back same with sys.stdin. Is this possible? I'm not looking for sys.argv[0] which returns the current script name.
Thanks!
In general it is not possible to obtain the filename in a platform-agnostic way. The other answers cover sensible alternatives like passing the name on the command-line.
On Linux, and some related systems, you can obtain the name of the file through the following trick:
import os
print(os.readlink('/proc/self/fd/0'))
/proc/ is a special filesystem on Linux that gives information about processes on the machine. self means the current running process (the one that opens the file). fd is a directory containing symbolic links for each open file descriptor in the process. 0 is the file descriptor number for stdin.
You can use ArgumentParser, which automattically gives you interface with commandline arguments, and even provides help, etc
from argparse import ArgumentParser
parser = ArgumentParser()
parser.add_argument('fname', metavar='FILE', help='file to process')
args = parser.parse_args()
with open(args.fname) as f:
#do stuff with f
Now you call python2 ritwc.py DarkAndStormyNight.txt. If you call python3 ritwc.py with no argument, it'll give an error saying it expected argument for FILE. You can also now call python3 ritwc.py -h and it will explain that a file to process is required.
PS here's a great intro in how to use it: http://docs.python.org/3.3/howto/argparse.html
In fact, as it seams that python cannot see that filename when the stdin is redirected from the console, you have an alternative:
Call your program like this:
python3 ritwc.py -i your_file.txt
and then add the following code to redirect the stdin from inside python, so that you have access to the filename through the variable "filename_in":
import sys
flag=0
for arg in sys.argv:
if flag:
filename_in = arg
break
if arg=="-i":
flag=1
sys.stdin = open(filename_in, 'r')
#the rest of your code...
If now you use the command:
print(sys.stdin.name)
you get your filename; however, when you do the same print command after redirecting stdin from the console you would got the result: <stdin>, which shall be an evidence that python can't see the filename in that way.
I don't think it's possible. As far as your python script is concerned it's writing to stdout. The fact that you are capturing what is written to stdout and writing it to file in your shell has nothing to do with the python script.
Related
My perl script is at path:
a/perl/perlScript.pl
my python script is at path:
a/python/pythonScript.py
pythonScript.py gets an argument from stdin, and returns result to stdout. From perlScript.pl , I want to run pythonScript.py with the argument hi to stdin, and save the results in some variable. That's what I tried:
my $ret = `../python/pythonScript.py < hi`;
but I got the following error:
The system cannot find the path specified.
Can you explain the path can't be found?
The qx operator (backticks) starts a shell (sh), in which prog < input syntax expects a file named input from which it will read lines and feed them to the program prog. But you want the python script to receive on its STDIN the string hi instead, not lines of a file named hi.
One way is to directly do that, my $ret = qx(echo "hi" | python_script).
But I'd suggest to consider using modules for this. Here is a simple example with IPC::Run3
use warnings;
use strict;
use feature 'say';
use IPC::Run3;
my #cmd = ('program', 'arg1', 'arg2');
my $in = "hi";
run3 \#cmd, \$in, \my $out;
say "script's stdout: $out";
The program is the path to your script if it is executable, or perhaps python script.py. This will be run by system so the output is obtained once that completes, what is consistent with the attempt in the question. See documentation for module's operation.
This module is intended to be simple while "satisfy 99% of the need for using system, qx, and open3 [...]. For far more power and control see IPC::Run.
You're getting this error because you're using shell redirection instead of just passing an argument
../python/pythonScript.py < hi
tells your shell to read input from a file called hi in the current directory, rather than using it as an argument. What you mean to do is
my $ret = `../python/pythonScript.py hi`;
Which correctly executes your python script with the hi argument, and returns the result to the variable $ret.
The Some of the other answers assume that hi must be passed as a command line parameter to the Python script but the asker says it comes from stdin.
Thus:
my $ret = `echo "hi" | ../python/pythonScript.py`;
To launch your external script you can do
system "python ../python/pythonScript.py hi";
and then in your python script
import sys
def yourFct(a, b):
...
if __name__== "__main__":
yourFct(sys.argv[1])
you can have more informations on the python part here
Some problems with python subprocess.check_output.
output = subprocess.check_output(args)
where my args is:
args = "C:\\DO\\bin\\Config.exe --ChCfg7 --LFE -b1152000 C:\\DO\\PCM\\1.wav C:\\DO\\PCM\\2.wav C:\\DO\\PCM\\3.wav C:\\DO\\PCM\\4.wav C:\\DO\\PCM\\5.wav C:\\DO\\PCM\6.wav --ModeBCast -oC:\\DO\\OUT\\outfile > C:\\DO\\OUT\\log.txt
This works when executed from standard windows command line, but doesn't work when executed via Python subprocess.check_output. In win cmd case there is output file produced and log.txt too, and python script produces out file with size 0, and no log.txt at all.
output = subprocess.check_output(args,shell=True)
Run this with shell=True
Use a list of args and redirect the output to a file:
import subprocess
args = ['C:/DO/bin/Config.exe', '--ChCfg7', '--LFE', '-b1152000', 'C:/DO/PCM/1.wav', 'C:/DO/PCM/2.wav', 'C:/DO/PCM/3.wav', 'C:/DO/PCM/4.wav', 'C:/DO/PCM/5.wav', 'C:/DO/PCM/6.wav', '--ModeBCast', '-oC:/DO/OUT/outfile']
with open("C:/DO/OUT/log.txt", "w") as f:
subprocess.check_call(args, stdout=f)
You can use shell=Truebut for security reasons generally it is not a very good idea and the same can be quite easily achieved using the code above and simply redirecting the output to the file.
> is a shell redirection operator. Either run the command in a shell or (better) as #Padraic Cunningham suggested emulate it in Python:
#!/usr/bin/env python
import subprocess
args = r"C:\DO\bin\Config.exe --ChCfg7 --LFE -b1152000".split()
args += [r'C:\DO\PCM\%d.wav' % i for i in range(1, 7)]
args += ["--ModeBCast", r"-oC:\DO\OUT\outfile"]
with open(r"C:\DO\OUT\log.txt", "wb", 0) as output_file:
subprocess.check_call(args, stdout=output_file)
The code uses raw string literals for Windows paths to avoid escaping backslashes.
There is usually no point to use shell=True on Windows unless you want to run a built-in command such as dir. If args is not constructed using input from an external source then security considerations do not apply. shell=True starts additional process (%COMSPEC%) and it changes how the executable is searched and it changes what characters should be escaped (what characters are metacharacters) — do not use shell=True unless necessary.
I need to do this:
paste file1 file2 file3 > result
I have the following in my python script:
from subprocess import call
// other code here.
// Here is how I call the shell command
call ["paste", "file1", "file2", "file3", ">", "result"])
Unfortunately I get this error:
paste: >: No such file or directory.
Any help with this will be great!
You need to implement the redirection yourself, if you're wisely deciding not to use a shell.
The docs at https://docs.python.org/2/library/subprocess.html warn you not to use a pipe -- but, you don't need to:
import subprocess
with open('result', 'w') as out:
subprocess.call(["paste", "file1", "file2", "file3"], stdout=out)
should be just fine.
There are two approaches to this.
Use shell=True:
call("paste file1 file2 file3 >result", shell=True)
Redirection, >, is a shell feature. Consequently, you can only access it when using a shell: shell=True.
Keep shell=False and use python to perform the redirection:
with open('results', 'w') as f:
subprocess.call(["paste", "file1", "file2", "file3"], stdout=f)
The second is normally preferred as it avoids the vagaries of the shell.
Discussion
When the shell is not used, > is just another character on the command line. Thus, consider the error message:
paste: >: No such file or directory.
This indicates that paste had received > as an argument and was trying to open a file by that name. No such file exists. Therefore the message.
As the shell command line, one can create a file by that name:
touch '>'
If such a file had existed, paste, when called by subprocess with shell=False, would have used that file for input.
If you don't mind adding an additional dependency in your code base you might consider installing the sh Python module (from PyPI:sh using pip, of course).
This is a rather clever wrapper around Python's subprocess module's functionality. Using sh your code would look something like:
#!/usr/bin/python
from sh import paste
paste('file1', 'file2', 'file3', _out='result')
... although I think you'd want some exception handling around that so you could use something like:
#!/usr/bin/python
from __future__ import print_function
import sys
from sh import paste
from tempfile import TemporaryFile
with tempfile.TemporaryFile() as err:
try:
paste('file1', 'file2', 'file3', _out='result', _err=err)
except (EnvironmentError, sh.ErrorReturnCode) as e:
err.seek(0)
print("Caught Error: %s" % err.read(), file=sys.stderr)
sh makes such things almost trivially easy although there are some tricks as you get more advanced. You also have to note the difference between _out= and other keyword arguments of that form, vs. sh's magic for most other keyword arguments.
All that sh magic make confuse anyone else who ever reads your code. You might also find that using Python modules with sh code interlaced into it makes you complacent about portability issues. Python code is generally fairly portable while Unix command line utilities can vary considerably from one OS to another and even from one Linux distribution or version to another. Having lots of shell utilities interlaced with your Python code in such a transparent way may make that problem less visible.
I am wanting to run an executable that would normally be run directly on the command line but ultimately via a Python script.
I used subprocess.Popen after reading through here and multiple Google results to achieve some limited success.
>>>import subprocess
>>>exe_path = sys.argv[1]
>>>dir_path_in = sys.argv[2]
>>>dir_path_out = sys.argv[3]
>>>subprocess.Popen([exe_path])
It then displays
<subprocess.Popen object at 0x021B7B30>
Followed by
>>>usage: <path to exe> [options] <dir_path> <dir_path_out>
But if I enter what you would normally expect to on the command line if used exclusively it returns:
>>>SyntaxError: invalid token
I have tested what is entered exclusively on the command line with the exe and it works fine just not via Python
I have had a look through StackOverFlow and the best kind of comparison I found was here How to handle an executable requiring interactive responses?
Ultimately the "usage" part will not even be required in the end as the declared sys.argvs will provide all the information the executable requires to run automatically.
The subprocess.call() achieved the desired result by declaring the argv variables and then concatenating the variables and using that final variable in a subprocess.call() as opposed to using shlex.split() which I first tried but it struggled with paths even with the '\' escaped for Windows
import subprocess
exe_path = sys.argv[1]
dir_path_in = sys.argv[2]
dir_path_out = sys.argv[3]
command = exe_path, dir_path_in, dir_path_out
p = subprocess.call(command)
I am running a python program I have coded with a subprocess WolfPsort Program.
It is a bioinformatics tool for protein localization detection program.
However, the python subprocess does not excute my input file.
This is the code
#!/usr/bin/python
# secref.py is for secretome refining
import os
import sys
import subprocess
if len(sys.argv) != 2:
print >> sys.stderr, 'Usage: python secref.py [*.fasta]'
exit(1)
if sys.argv[1].endswith('.fasta'):
filename = sys.argv[1]
else:
print >> sys.stderr, 'Input correct file... [*.fasta]'
exit(1)
filehandle = open(filename,'r')
progWolf = subprocess.Popen(['runWolfPsortSummary','fungi','<',filename,'>','tmpWolfResult'])
progWolf.wait()
If I run the code it gives error message like this:
[karyo#hostname secref.1.0]$ python secref.py A.carbonarius.fasta
Command Line Parsing Error; Do not know what to do with argument "<"
Usage:
runWolfPsortSummary [*OPTIONS*] *organismType*
runWolfPsortSummary (--usage|--help|--man)
Pipe sequences in from standard in.
The subprocess dose not recognise "<" symbol, but WolfPsort Program needs "<" to recognise the input fasta file and ">" is required to write a temporary result file.
How can I make the subprocess to understand the argument "<"?
Please, help me out!
I'm guessing you're trying to use shell magic to read from filename and write to tmpWolfResult. In order to make that happen, you need:
progWolf = subprocess.Popen('runWolfPsortSummary fungi < %s > tmpWolfResult'%filename, shell=True)
I feel obligated to mention that since this input is coming from a commandline argument, it's technically not safe/trusted and a malicious user with access to run this script on your system could do some nasty things.
However, it's probably more likely that you're distributing this script (or just using it yourself) and you and your users probably aren't interested in messing up your own system...
<, > are usually interpreted by the shell that Popen() doesn't spawn unnecessarily by default. You could use stdin, stdout parameters instead to redirect input/output from/to files:
from subprocess import check_call
with open(filename) as file, open('tmpWolfResult', 'w') as output_file:
check_call(['runWolfPsortSummary', 'fungi'], stdin=file, stdout=output_file)
Note: check_call() raises an exception if runWolfPsortSummary exits with non-zero status.
The Popen function takes an argument list separated by commas. The way you wrote it,
'<'
filename
'>'
is sent as three separate arguments. I'm assuming you want to concatenate that into one argument.
progWolf = subprocess.Popen(['runWolfPsortSummary','fungi','<' + filename + '>','tmpWolfResult'])