Develop Reference

Applying python script to file

This is a simple ask but I can't find any information on how to do it: I have a python script that is designed to take in a text file of a specific format and perform functions on it--how do I pipe a test file into the python script such that it is recognized as input()? More specifically, the Python is derived from skeleton code I was given that looks like this:
def main():
N = int(input())
lst = [[int(i) for i in input().split()] for _ in range(N)]
intervals = solve(N, lst)
print_solution(intervals)
if __name__ == '__main__':
main()
I just need to understand how to, from the terminal, input one of my test files to this script (and see the print_solution output)

Use the fileinput module
input.txt
...input.txt contents
script.py
#!/usr/bin/python3
import fileinput
def main():
for line in fileinput.input():
print(line)
if __name__ == '__main__':
main()
pipe / input examples:
$ cat input.txt | ./script.py
...input.txt contents
$ ./script.py < input.txt
...input.txt contents

You can take absolute or relative path in your input() function and then open this path via open()
filename = input('Please input absolute filename: ')
with open(filename, 'r') as file:
# Do your stuff
Please let me know if I misunderstood your question.

You can either:
A) Use sys.stdin (import sys at the top of course)
or
B) Use the ArgumentParser (from argparse import ArgumentParser) and pass the file as an argument.
Assuming A it would look something like this:
python script.py < file.extension
Then in the script it would look like:
fData = []
for line in sys.stdin.readLines():
fData.append(line)
# manipulate fData
There are a number of ways to achieve what you want. This is what I came up with off the top of my head. It may not be the best / efficient way, but it should work. I do a lot of file I/O with python at work and this is one of the ways I've achieved it in the past.
Note: If you want to write the manipulated lines back to the file use the argparse library.
Edit:
from argparse import ArgumentParser
def parseInput():
parser = ArgumentParser(description = "Takes input file to read")
parser.add_argument('-f', type = str, default = None, required =
True, help = "File to perform I/O on")
args = parser.parse_args()
return args
def main():
args = parseInput()
fData = []
# perform rb
with open(args.f, 'r') as f:
for line in f.readlines():
fData.append(line)
# Perform data manipulations
# perform wb
with open(args.f, 'w') as f:
for line in fData:
f.write(line)
if __name__ == "__main__":
main()
Then on command line it would look like:
python yourScript.py -f fileToInput.extension

Pdf-latex with python script, custom python variables into latex output

I currently have the following code which works to produce a PDF output. Is there a better way of writing up the content for the PDF, other than done here? This is a basic pdf, but am hoping to include multiple variables in later versions. I have inserted variable x, defined before the PDF content, into the latex pdf. Many thanks for any advice you can give.
PDF Output - image
import os
import subprocess
x = 7
content = \
r'''\documentclass{article}
\usepackage[utf8]{inputenc}
\usepackage[margin=1cm,landscape]{geometry}
\title{Spreadsheet}
\author{}
\date{}
\begin{document}''' + \
r'This is document version: ' + str(x) +\
r'\end{document}'
parser = argparse.ArgumentParser()
parser.add_argument('-c', '--course')
parser.add_argument('-t', '--title')
parser.add_argument('-n', '--name',)
parser.add_argument('-s', '--school', default='My U')
args = parser.parse_args()
with open('doc.tex','w') as f:
f.write(content%args.__dict__)
cmd = ['pdflatex', '-interaction', 'nonstopmode', 'doc.tex']
proc = subprocess.Popen(cmd)
proc.communicate()
retcode = proc.returncode
if not retcode == 0:
os.unlink('doc.pdf')
raise ValueError('Error {} executing command: {}'.format(retcode, ' '.join(cmd)))
os.unlink('doc.tex')
os.unlink('doc.log')```

As explained in this video, I think a better approach would be to export the variables from Python and save them into a .dat file using the following function.
def save_var_latex(key, value):
import csv
import os
dict_var = {}
file_path = os.path.join(os.getcwd(), "mydata.dat")
try:
with open(file_path, newline="") as file:
reader = csv.reader(file)
for row in reader:
dict_var[row[0]] = row[1]
except FileNotFoundError:
pass
dict_var[key] = value
with open(file_path, "w") as f:
for key in dict_var.keys():
f.write(f"{key},{dict_var[key]}\n")
Then you can call the above function and save all the variables into mydata.dat. For example, in Python, you could save a variable and call it document_version using the following line of code:
save_var_latex("document_version", 21)
In LaTeX (in the preamble of your main file), you just have to import the following packages:
% package to open file containing variables
\usepackage{datatool, filecontents}
\DTLsetseparator{,}% Set the separator between the columns.
% import data
\DTLloaddb[noheader, keys={thekey,thevalue}]{mydata}{../mydata.dat}
% Loads mydata.dat with column headers 'thekey' and 'thevalue'
\newcommand{\var}[1]{\DTLfetch{mydata}{thekey}{#1}{thevalue}}
Then in the body of your document just use the \var{} command to import the variable, as follows:
This is document version: \var{document_version}

Script called from another script with arguments from another variable

I have created the Python script below which i would like to run and call another script from and then give the called script a variable in this case it would be an email address as "line" from a text file. What would be the easiest way to accomplish this please?
The problem now is that the script that is being called will not take the 'line' variable as an argument.
import bob
import os
# file handle fh fh = open('mailout.txt') while True:
# read line
line = fh.readline()
line = line.replace("\r", "").replace("\n", "")
command = 'python3 bob.py ' + line
os.system(command)
# check if line is not a empty value
if not line:
break fh.close()

As #Zach's comment, you can call it by giving line as argument. Otherwise, you can do it by using argparse. Assume that you have two functions inner.py and outer.py.
inner.py
import argparse
parser = argparse.ArgumentParser()
parser.add_argument('-s', '--sentence')
args = parser.parse_args()
print(args.sentence)
outer.py
import os
f = open('email.txt')
line = f.readline()
line = line.replace("\r", "").replace("\n", "")
line = "\""+line+"\""
command = 'python inner.py -s' + line
os.system(command)
Then calling python outer.py returns
Just a line to try

How can I call Vim from Python?

I would like a Python script to prompt me for a string, but I would like to use Vim to enter that string (because the string might be long and I want to use Vim's editing capability while entering it).

You can call vim with a file path of your choice:
from subprocess import call
call(["vim","hello.txt"])
Now you can use this file as your string:
file = open("hello.txt", "r")
aString = file.read()

Solution:
#!/usr/bin/env python
from __future__ import print_function
from os import unlink
from tempfile import mkstemp
from subprocess import Popen
def callvim():
fd, filename = mkstemp()
p = Popen(["/usr/bin/vim", filename])
p.wait()
try:
return open(filename, "r").read()
finally:
unlink(filename)
data = callvim()
print(data)
Example:
$ python foo.py
This is a big string.
This is another line in the string.
Bye!

filter a Linux log file using Python

I want to filter a log file to keep all lines matching a certain pattern. I want to do this with Python.
Here's my first attempt:
#!/usr/bin/env python
from sys import argv
script, filename = argv
with open(filename) as f:
for line in f:
try:
e = line.index("some_term_I_want_to_match")
except:
pass
else:
print(line)
How can I improve this to:
save the result to a new file of similar name (i.e., a different extension)
use regex to make it more flexible/powerful.
(I'm just learning Python. This question is as much about learning Python as it is about accomplishing this particular result.)
OK, here's what I came up with so far... But how do you do the equivalent of prepending an r as in the following line
re.compile(r"\s*")
where the string is not a string literal, as in the next line?
re.compile(a_string_variable)
Other than that, I think this updated version does the job:
#!/usr/bin/env python
from sys import argv
import re
import os
import argparse #requires Python 2.7 or above
parser = argparse.ArgumentParser(description='filters a text file on the search phrase')
parser.add_argument('-s','--search', help='search phrase or keyword to match',required=True)
parser.add_argument('-f','--filename', help='input file name',required=True)
parser.add_argument('-v','--verbose', help='display output to the screen too', required=False, action="store_true")
args = parser.parse_args()
keyword = args.search
original_file = args.filename
verbose = args.verbose
base_file, ext = os.path.splitext(original_file)
new_file = base_file + ".filtered" + ext
regex_c = re.compile(keyword)
with open(original_file) as fi:
with open(new_file, 'w') as fo:
for line in fi:
result = regex_c.search(line)
if(result):
fo.write(line)
if(verbose):
print(line)
Can this be easily improved?

Well, you know, you have answered most of your questions yourself already :)
For regular expression matching use re module (the doc has pretty explanatory examples).
You already have made use open() function for opening a file. Use the same function for open files for writing, just provide a corresponding mode parameter ("w" or "a" combined with "+" if you need, see help(open) in the Python interactive shell). That's it.