Develop Reference

Applying python script to file

This is a simple ask but I can't find any information on how to do it: I have a python script that is designed to take in a text file of a specific format and perform functions on it--how do I pipe a test file into the python script such that it is recognized as input()? More specifically, the Python is derived from skeleton code I was given that looks like this:
def main():
N = int(input())
lst = [[int(i) for i in input().split()] for _ in range(N)]
intervals = solve(N, lst)
print_solution(intervals)
if __name__ == '__main__':
main()
I just need to understand how to, from the terminal, input one of my test files to this script (and see the print_solution output)

Use the fileinput module
input.txt
...input.txt contents
script.py
#!/usr/bin/python3
import fileinput
def main():
for line in fileinput.input():
print(line)
if __name__ == '__main__':
main()
pipe / input examples:
$ cat input.txt | ./script.py
...input.txt contents
$ ./script.py < input.txt
...input.txt contents

You can take absolute or relative path in your input() function and then open this path via open()
filename = input('Please input absolute filename: ')
with open(filename, 'r') as file:
# Do your stuff
Please let me know if I misunderstood your question.

You can either:
A) Use sys.stdin (import sys at the top of course)
or
B) Use the ArgumentParser (from argparse import ArgumentParser) and pass the file as an argument.
Assuming A it would look something like this:
python script.py < file.extension
Then in the script it would look like:
fData = []
for line in sys.stdin.readLines():
fData.append(line)
# manipulate fData
There are a number of ways to achieve what you want. This is what I came up with off the top of my head. It may not be the best / efficient way, but it should work. I do a lot of file I/O with python at work and this is one of the ways I've achieved it in the past.
Note: If you want to write the manipulated lines back to the file use the argparse library.
Edit:
from argparse import ArgumentParser
def parseInput():
parser = ArgumentParser(description = "Takes input file to read")
parser.add_argument('-f', type = str, default = None, required =
True, help = "File to perform I/O on")
args = parser.parse_args()
return args
def main():
args = parseInput()
fData = []
# perform rb
with open(args.f, 'r') as f:
for line in f.readlines():
fData.append(line)
# Perform data manipulations
# perform wb
with open(args.f, 'w') as f:
for line in fData:
f.write(line)
if __name__ == "__main__":
main()
Then on command line it would look like:
python yourScript.py -f fileToInput.extension

Python script open a json file with arguments

I am new on python. I am using a python script where I load a json file to set certain values to the script, but my idea is to import that file more dynamically using arguments (I think is the correct use), so I don´t need to always include the name of the json file in the python script, here is may code example:
import json
from pprint import pprint
with open("VariableSettings.json") as json_data:
data = json.load(json_data)
so my idea is to change the code: "with open("VariableSettings.json") as json_data" with args to open the json file dynamically.
I think that on command prompt I can use the command py test.py arg1 (this represent the file path).
So I know that probably my explanation is a bit confusing but if some can help I appreciate it.

You can use sys to do that. In the example below I created a file test.json with the content
{"foo": "bar"}
And modified your code as
import json
import sys
with open(sys.argv[1]) as json_data:
data = json.load(json_data)
print(data)
You need to call execute as
python test.py test.json
and the output will be
{'foo': 'bar'}
More details can be found in this other post

You can also use argparse:
import json
import argparse
parser = argparse.ArgumentParser()
parser.add_argument("-f", "--filename", required=True, type=str)
args = parser.parse_args()
with open(args.filename) as json_data:
data = json.load(json_data)
print(data)
Which can be called with the alias:
python test.py -f test.json
Or full argument name:
python test.py --filename test.json
And if you don't supply a file, you get:
usage: test.py [-h] -f FILENAME
test.py: error: the following arguments are required: -f/--filename
since I passed required=True. You can remove this if you want the argument to be optional.
Addtionally, you could also extend your program to check that if the JSON file has correct format by catching json.JSONDecodeError with try/except:
with open(args.filename) as json_data:
try:
data = json.load(json_data)
print(data)
except json.JSONDecodeError:
print('Invalid JSON format')

Use the sys module
Ex:
import sys
import json
from pprint import pprint
if len(sys.argv) < 2:
print("Input File Missing")
sys.exit()
with open(sys.argv[1]) as json_data:
data = json.load(json_data)
print(data)
To Run Use
python yourScriptName.py full_path_to.json

Save file in Python from Output

Question here for you. I have this script for python that checks large datasets for emails and extracts them. On my mac it just displays all the email addresses in the terminal. Sometimes the files are 1-2 gigs so it can take a bit and the output is insane. I was wondering how easy in Python is it to have it just save to a file instead of printing it all out in terminal.
I dont even need to see it all being dumped into the terminal.
Here is the script I am working with
#!/usr/bin/env python
#
# Extracts email addresses from one or more plain text files.
#
# Notes:
# - Does not save to file (pipe the output to a file if you want it saved).
# - Does not check for duplicates (which can easily be done in the terminal).
#
from optparse import OptionParser
import os.path
import re
regex = re.compile(("([a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+\/=?^_`"
"{|}~-]+)*(#|\sat\s)(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?(\.|"
"\sdot\s))+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)"))
def file_to_str(filename):
"""Returns the contents of filename as a string."""
with open(filename) as f:
return f.read().lower() # Case is lowered to prevent regex mismatches.
def get_emails(s):
"""Returns an iterator of matched emails found in string s."""
# Removing lines that start with '//' because the regular expression
# mistakenly matches patterns like 'http://foo#bar.com' as '//foo#bar.com'.
return (email[0] for email in re.findall(regex, s) if not email[0].startswith('//'))
if __name__ == '__main__':
parser = OptionParser(usage="Usage: python %prog [FILE]...")
# No options added yet. Add them here if you ever need them.
options, args = parser.parse_args()
if not args:
parser.print_usage()
exit(1)
for arg in args:
if os.path.isfile(arg):
for email in get_emails(file_to_str(arg)):
print email
else:
print '"{}" is not a file.'.format(arg)
parser.print_usage()

Instead of printing, just write to a file instead.
with open('filename.txt', 'w') as f:
f.write('{}\n'.format(email))

First, you need to open a file:
file = open('output', 'w')
Then, instead of printing the email, write it in the file: file.write(email + '\n')
You can also just redirect the output of the program to a file at execution time as jasonharper said.

While printing , replace with write statement
for arg in args:
if os.path.isfile(arg):
for email in get_emails(file_to_str(arg)):
print email
In that ,just replace with
for arg in args:
if os.path.isfile(arg):
for email in get_emails(file_to_str(arg)):
with open (tempfile , 'a+') as writefile:
writefile.write(name+'\n')
tempfile is location of your output file

How to save python output to text file

I tried many way to save output to text file but it don't work for me
this is code
from optparse import OptionParser
import os.path
import re
regex = re.compile(("([a-z0-9!#$%&'*+\/=?^_`{|}~-]+(?:\.[a-z0-9!#$%&'*+\/=?^_`"
"{|}~-]+)*(#|\sat\s)(?:[a-z0-9](?:[a-z0-9-]*[a-z0-9])?(\.|"
"\sdot\s))+[a-z0-9](?:[a-z0-9-]*[a-z0-9])?)"))
def file_to_str(filename):
"""Returns the contents of filename as a string."""
with open(filename) as f:
return f.read().lower() # Case is lowered to prevent regex mismatches.
def get_emails(s):
"""Returns an iterator of matched emails found in string s."""
# Removing lines that start with '//' because the regular expression
# mistakenly matches patterns like 'http://foo#bar.com' as '//foo#bar.com'.
return (email[0] for email in re.findall(regex, s) if not email[0].startswith('//'))
if __name__ == '__main__':
parser = OptionParser(usage="Usage: python %prog [FILE]...")
# No options added yet. Add them here if you ever need them.
options, args = parser.parse_args()
if not args:
parser.print_usage()
exit(1)
for arg in args:
if os.path.isfile(arg):
for email in get_emails(file_to_str(arg)):
print email
else:
print '"{}" is not a file.'.format(arg)
parser.print_usage()
when you run script
it will print emails in dos screen
I want save it to text file

You can replace the below code with your own.
file = open(output_path, 'w')
for arg in args:
if os.path.isfile(arg):
for email in get_emails(file_to_str(arg)):
file.write(email + '\n')
else:
print '"{}" is not a file.'.format(arg)
file.close()

Your code already has a few print statements (You should use a logger instead) but instead of adding to code to write to a file, why not just
$ python myscript.py >> output.txt
That will give you the exact same output without adding code.

$ python your_script.py > path/to/output_file/file_name.txt
OR
$ python your_script.py >> path/to/output_file/file_name.txt
This will give the output given by your print statements into file_name.txt file.

Is it possible to pass arguments to a python made exe at runtime?

I'm experimenting with file I/O. I have a small practice program that creates a text file when run. I packaged it with pyinstaller so that double clicking on the exe creates a new folder and places a text file with "hello world" inside of it. Easy peasy.
Then I started wondering about main(). This is just a function like any other, right? So does that mean I can pass arguments to it at runtime?
I was thinking about the Steam client and how you can put stuff like '-dev' and '-console' in the shortcut. Is there a way to do this to a python exe that I have made?
I may be explaining terribly, so here's an example:
def makeFile(string):
if string:
f = open('mytext.txt', 'w') #create text file in local dir
print >> f, 'hello, ' + string + '! \nHow are ya?'
f.close()
else:
f = open('mytext.txt', 'w') #create text file in local dir
print >> f, 'hello, person! \nHow are ya?'
f.close()
def main(string = None):
makeFile(string)
So if I take this code and make it an exe, would I be able to add my optional arguments somehow.
I tried the above code, and the running test.exe --"myname" but that didn't work.
Is there a way to do this?

What you're looking for is either the sys module, or the optparse module.
sys will give you very basic control over command line args.
For example:
import sys
if __name__ == "__main__":
if len(sys.argv)>1:
print sys.argv[1]
In the above example, if you were to open up a shell and type -
test.exe "myname"
The resultant output would be:
myname
Note that sys.argv[0] is the name of the script you are currently running. Each subsequent argument is defined by a space, so in your example above
test.exe -- myname
argv[0] = "test.exe"
argv[1] = "--"
argv[2] = "myname"
Optparse gives a much more robust solution that allows you to define command line switches with multiple options and defines variables that will store the appropriate options that can be accessed at runtime.
Re-writing your example:
from optparse import OptionParser
def makeFile(options = None):
if options:
f = open('mytext.txt', 'w') #create text file in local dir
print >> f, 'hello, ' + options.name + '! \nHow are ya?'
f.close()
else:
f = open('mytext.txt', 'w') #create text file in local dir
print >> f, 'hello, person! \nHow are ya?'
f.close()
if __name__ == "__main__":
parser = OptionParser()
parser.add_option('-n','--name',dest = 'name',
help='username to be printed out')
(options,args) = parser.parse_args()
makeFile(options)
You would run your program with :
test.exe -n myname
and the output (in myfile.txt) would be the expected:
Hello, myname!
How are ya?
Hope that helps!

Yes, you can do it with sys.argv. Check out this link: http://docs.python.org/library/sys.html#sys.argv. But remember not to forget import sys, and then you can use it.
import sys
# If there is an argument passed to your file
if len(sys.argv) > 1:
# argv[1] has your filename
filename = sys.argv[1]
print (filename)
# Output...
# new-host:~ yanwchan$ python3.2 test.py text.txt
# text.txt
argv[0] has test.py
argv[1] has text.txt
Edit: However, I do some more research on this topic and found out this: https://stackoverflow.com/a/4188500/1276534
As katrielalex points out, maybe you can look into argparse as well.? It provides a lot more functionality as well as safety check. Interesting information.
And here is a great tutorial: http://www.doughellmann.com/PyMOTW/argparse/

What you are looking for is something like the python argparse module
Or you can read the values directly using sys.argv
import sys
sys.argv[0] # the name of the command that was called
sys.argv[1] # the first argument, eg '--dev'
sys.argv[2] # the second...

Just a note for completeness: there is docopt now, which makes it really easy to write even complex command line interfaces by describing it in a simple language. Documenting and parsing the interface actually becomes the same task with docopt.