I want to pass data to a Python file using a pipe and also specifying an input file like:
cat file.txt|python script.py -u configuration.txt
I currently have this:
for line in fileinput.input(mode='rU'):
print(line)
I know there can be something with sys.argv but maybe using fileinput there is a clean way to do it?
Thanks.
From the documentation:
If a filename is '-', it is also replaced by sys.stdin. To specify an alternative list of filenames, pass it as the first argument to input().
So you can create a list containing '-' as well as the contents of sys.argv[1:] (the default), and pass that to input(). Or alternatively just put - in the list of arguments of your Python program:
cat file.txt|python script.py -u - configuration.txt
or
cat file.txt|python script.py -u configuration.txt -
depending on whether you want data provided on standard input to be processed before or after the contents of configuration.txt.
If you want to do anything more complicated than just processing the contents of standard input as if it were an input file, you probably should not be using the fileinput module.
Related
I am generating an ImageMagick bash command using Python. Something like
import subprocess
input_file = "hello.png"
output_file = "world.jpg"
subprocess.run(["convert", input_file, output_file])
where there might be more arguments before input_file or output_file. My question is, if either of the filenames is user provided and the user provides a filename that can be parsed as a command line option for ImageMagick, isn't that unsafe?
If the filename starts with a dash, ImageMagick indeed could think that this is an option instead of a filename. Most programs - including AFIK the ImageMagick command line tools - follow the convention that a double-dash (--) denotes the end of the options. If you do a
subprocess.run(["convert", "--", input_file, output_file])
you should be safe in this respect.
From the man page (and a few tests), convert requires an input file and an output file. If you only allow two tokens and if a file name is interpreted as an option then convert is going to miss at least one of the files, so you'll get an ugly message but you should be fine.
Otherwise you can prefix any file name that starts with - with ./ (except - itself, which is stdin or stdout depending on position), so that it becomes an unambiguous file path to the same file.
I have a simple shell script script.sh:
echo "ubuntu:$1" | sudo chpasswd
I need to open the script, read it, insert the argument, and save it as a string like so: 'echo "ubuntu:arg_passed_when_opening" | sudo chpasswd' using Python.
All the options suggested here actually execute the script, which is not what I want.
Any suggestions?
You would do this the same way that you read any text file, and we can use sys.argv to get the argument passed when running the python script.
Ex:
import sys
with open('script.sh', 'r') as sfile:
modified_file_contents = sfile.read().replace('$1', sys.argv[1])
With this method, modified_file_contents is a string containing the text of the file, but with the specified variable replaced with the argument passed to the python script when it was run.
I have a program that takes an input file:
python subprogram.py < input.txt > out.txt
If I have a number of input files, how can I write a single python program runs on those inputs and produces a single output? I believe the program should run like:
python program.py < input_1.txt input_2.txt > out.txt
And the program itself should look something like:
from subprogram import MyClass
import sys
if __name__ == '__main__':
myclass = MyClass()
myclass.run()
Have a look at the fileinput module
import fileinput
for line in fileinput.input():
process(line)
This iterates over the lines of all files listed in sys.argv[1:], defaulting to sys.stdin if the list is empty. If a filename is '-', it is also replaced by sys.stdin. To specify an alternative list of filenames, pass it as the first argument to input(). A single file name is also allowed.
Make your program accept command line parameters:
python program.py input_1.txt input_2.txt > out.txt
And you can access them like:
from subprogram import MyClass
import sys
if __name__ == '__main__':
class = MyClass()
class.run(sys.argv)
The way you're using is not about Python, it's about your shell. You are just redirect standart input/output to files. If you want to achieve that:
cat input1.txt input2.txt | python subprogram.py > out.txt
Let your shell do the work for you:
cat input_1.txt input_2.txt | python program.py > out.text
The cat command will concatenate the two input files together and your python program can just read from stdin and treat them like one big file.
I'm writing a small script, that should be able to handle multiple files. So I've added that files can be passed comma seperated, and do a arg.split(',') and then handle each one.
Now I've wanted to add asterisk as input possibility like
python myPythonScript.py -i folder/*
If I print the the argument to option -i right when I access it the first time I get
folder/firstFileInFolder.txt
But if I call my script with
python myPythonScript.py -i someFolder/someFile,folder/*
it works just fine. Does anyone have an idea, why python might behave that way?
Try to run this script
import sys
for arg in sys.argv:
print arg
python script.py *
your shell expands the asterisk before python sees it.
As mentioned in the comments, your shell is expanding the asterisk for the non-comma separated case. If you know that the user may specify an asterisk as part of a file name as in your second example, you can have Python do the path expansion by using the glob module.
from glob import glob
glob('*')
code which would allow either the shell or Python to do asterisk expansion may look something like this:
import glob
file_list = []
for pattern in sys.argv[1:]:
file_list.extend(glob.glob(pattern))
In your case, using a comma as a separator would then prevent you from using a comma as part of a filename.
Hi is there a way to load a file as such, but from command line
so i can type python test.py data.txt
instead of
data = [line.strip() for line in open("/home/user/data.txt", "rb").readlines()]
import sys
data = [line.strip() for line in open(sys.argv[1], "rb")]
No need for .readlines(), by the way.
To pass a parameter to your script from the command line you would want to utilize sys.argv or fileinput.
sys.argv
The list of command line arguments passed to a Python script. argv[0]
is the script name (it is operating system dependent whether this is a
full pathname or not). If the command was executed using the -c
command line option to the interpreter, argv[0] is set to the string
'-c'. If no script name was passed to the Python interpreter, argv[0]
is the empty string.
fileinput:
import fileinput
for line in fileinput.input():
process(line)
This iterates over the lines of all files listed in sys.argv[1:],
defaulting to sys.stdin if the list is empty. If a filename is '-', it
is also replaced by sys.stdin. To specify an alternative list of
filenames, pass it as the first argument to input(). A single file
name is also allowed.
Have a look at the argparse module. It is fairly easy to use and even allows you to directly supply and open files from the command line arguments.