Develop Reference

Preserve function context in Python

I'm having a function in Python that when being called first, reads the content of a file to a list and checks whether or not an element is within that list.
def is_in_file(element, path):
with open(path, 'r') as f:
lines = [line.strip() for line in f.readlines()]
return element in lines
When the function is being called again, however, the content of the file should not be read again; the function should instead remember the value of lines from the first call.
Is there a way to preserve the context of a function when calling the function again? I don't want to make lines global to not litter the above namespace. I guess it's quite similar to the use of a generator and the yield statement...

My opinion is that the correct way is to encapsulate this in a class. The path is set at instance creation, and method calls use the list of lines. That way you can even have different files at the same time:
class finder:
def __init__(self, path):
with open(path, 'r') as f:
self.lines = [line.strip() for line in f]
def is_in_file(self, element):
return element in lines
That is not exactly what you have asked for, but is much more OO.

Dirty hack: add variable to function object and store value there.
def is_in_file(element, path):
if not hasattr(is_in_file, "__lines__"):
with open(path, 'r') as f:
setattr(is_in_file, "__lines__", [line.strip() for line in f.readlines()])
return element in is_in_file.__lines__

You could save the lines in a keyword argument declared with a mutable default value:
def is_in_file(element, path, lines=[]):
if lines:
return element in lines
with open(path, 'r') as f:
lines += [line.strip() for line in f.readlines()]
return element in lines
Caveat:
you must be sure that this function is only called with one file; if you call it with a second file, it will not open it and continue to return values based on the first file opened.
A more flexible solution:
A more flexible solution is maybe to use a dictionary of lines, where each new file can be opened once and stored, using the path as key; you can then call the function with different files, and get the correct results while memoizing the contents.
def is_in_file(element, path, all_lines={}):
try:
return element in all_lines[path]
except KeyError:
with open(path, 'r') as f:
all_lines[path] = [line.strip() for line in f.readlines()]
return element in lines
OO solution:
Create a class to encapsulate the content of a file, like what #SergeBallesta proposed; although it does not address exactly what you requested, it is likely the better solution in the long run.

Use the functools.lru_cache decorator to set up a helper function that reads in any given file only once and then stores the result.
from functools import lru_cache
#lru_cache(maxsize=1)
def read_once(path):
with open(path) as f:
print('reading {} ...'.format(path))
return [line.strip() for line in f]
def in_file(element, path):
return element in read_once(path)
Demo:
>>> in_file('3', 'file.txt')
reading file.txt ...
True
>>> in_file('3', 'file.txt')
True
>>> in_file('3', 'anotherfile.txt')
reading anotherfile.txt ...
False
>>> in_file('3', 'anotherfile.txt')
False
This has the serious advantage that in_file does not have to be called with the same file name every time.
You can adjust the maxsize argument to a higher number if you want more than one file to be cached at any given time.
Lastly: consider at set for the return value of read_once if all you are interested in are membership tests.

This answer proposes a class similar similar to Serge Ballesta's idea.
The difference is that it totally feels like a function because we use it's __call__ method instead of dot-notation in order to conduct the search.
In addition, you can add as many searchable files as you want.
Setup:
class in_file:
def __init__(self):
self.files = {}
def add_path(self, path):
with open(path) as f:
self.files[path] = {line.strip() for line in f}
def __call__(self, element, path):
if path not in self.files:
self.add_path(path)
return element in self.files[path]
in_file = in_file()
Usage
$ cat file1.txt
1
2
3
$ cat file2.txt
hello
$ python3 -i demo.py
>>> in_file('1', 'file1.txt')
True
>>> in_file('hello', 'file1.txt')
False
>>> in_file('hello', 'file2.txt')
True

Python, open for writing if exists, otherwise raise error

Is there an option I can pass open() that will cause an IOerror when trying to write a nonexistent file? I am using python to read and write block devices via symlinks, and if the link is missing I want to raise an error rather than create a regular file. I know I could add a check to see if the file exists and manually raise the error, but would prefer to use something built-in if it exists.
Current code looks like this:
device = open(device_path, 'wb', 0)
device.write(data)
device.close()

Yes.
open(path, 'r+b')
Specifying the "r" option means the file must exist and you can read.
Specifying "+" means you can write and that you will be positioned at the end.
https://docs.python.org/3/library/functions.html?#open

Use os.path.islink() or os.path.isfile() to check if the file exists.

Doing the check each time is a nuisance, but you can always wrap open():
import os
def open_if_exists(*args, **kwargs):
if not os.path.exists(args[0]):
raise IOError('{:s} does not exist.'.format(args[0]))
f = open(*args, **kwargs)
return f
f = open_if_exists(r'file_does_not_exist.txt', 'w+')
This is just quick and dirty, so it doesn't allow for usage as: with open_if_exists(...).
Update
The lack of a context manager was bothering me, so here goes:
import os
from contextlib import contextmanager
#contextmanager
def open_if_exists(*args, **kwargs):
if not os.path.exists(args[0]):
raise IOError('{:s} does not exist.'.format(args[0]))
f = open(*args, **kwargs)
try:
yield f
finally:
f.close()
with open_if_exists(r'file_does_not_exist.txt', 'w+') as f:
print('foo', file=f)

I am afraid you can't perform the check of file existence and raise error using the open() function.
Below is the signature of open() in python where name is the file_name, mode is the access mode and buffering to indicate if buffering is to be performed while accessing a file.
open(name[, mode[, buffering]])
Instead, you can check if the file exists or not.
>>> import os
>>> os.path.isfile(file_name)
This will return True or False depending on if the file exists. To test a file specifically, you can use this.
To test the existence of both files and directories, you can use:
>>> os.path.exists(file_path)

Smart way to end a subroutine if file doesnt exist in cwd

I am trying to open file "data.txt" in the cwd and readthe lines,is there a oneliner(or close ) to exit the subroutine if the file doesnt exist..i know there are ways like using os.path.exists and try/except IO error but am interested in one-liner or the smartest way
def readfile ():
f = open('data.txt')
lines = f.readlines()
f.close()

you can use with open
with open('data.txt') as f:
lines = f.readlines()
The with statement will automatically close the file after the nested block of code. The advantage of using a with statement is that it is guaranteed to close the file no matter how the nested block exits. If an exception occurs before the end of the block, it will close the file before the exception is caught by an outer exception handler.
If the nested block were to contain a return statement, or a continue or break statement, the with statement would automatically close the file in those cases, too.

You can use os.path.exists and a conditional expression:
import os.path as opath
def readlines(path):
return open(path).readlines() if opath.exists(path) else []
And if you're into lazy evaluation (it will only read as many lines as the caller consumes), combine it with a generator expression:
import os.path as opath
def readlines(path):
try:
return (l for l in open(path)) if opath.exists(path) else ()
except FileNotFoundError:
return ()

Fallback to stdout if no file name provided

I have a script that accepts as an argument a filename than opens it and writes some stuff.
I use the with statement:
with open(file_name, 'w') as out_file:
...
out_file.write(...)
Now what if I want to write to sys.stdout if no file_name is provided?
Do I necessarily need to wrap all actions in a function and put a condition before?
if file_name is None:
do_everything(sys.stdout)
else:
with open(file_name, 'w') as out_file:
do_everything(out_file)

from contextlib import contextmanager
#contextmanager
def file_or_stdout(file_name):
if file_name is None:
yield sys.stdout
else:
with open(file_name, 'w') as out_file:
yield out_file
Then you can do
with file_or_stdout(file_name) as wfile:
do_stuff_writing_to(wfile)

How do you handle command line arguments? If you use argparse you could use the type and default parameters of add_argument to handle this. For example, try something like the following:
import sys
import argparse
def main(argv=None):
if argv is None:
argv=sys.argv[1:]
parser = argparse.ArgumentParser()
parser.add_argument('infile', nargs='?',
type=argparse.FileType('w'),
default=sys.stdin)
args = parser.parse_args(argv)
print args.infile
return 0
if __name__=="__main__":
sys.exit(main(sys.argv[1:]))
If a file name is present as an argument the the script argparse will automatically open and close this file and args.infile will be a handle to this file. Otherwise args.infile will simply be sys.stdin.

You could write your own context manager. I'll post sample code later if noone else does

if file_name is None:
fd = sys.stdout
else:
fd = open(file_name, 'w')
# write to fd
if fd != sys.stdout:
fd.close();

Using the with ... as construct is useful to close the file automatically. This means that using it with sys.stdout, as I guess you know, would crash your program, because it would attempt at closing the system stdout!
This means something like with open(name, 'w') if name else sys.stdout as: would not work.
This make me say there isn't any simple-nice way to write your snippet better... but there are probably better ways to think on how to construct such a code!
The main point to clarify is when you need to open (and, more importantly, close) the filehandler for file_name, when file_name exists.
Personally I would simply drop the with .. as and take care of opening the file - and, more importantly, close it! - somewhere else. Mileage for that might vary depending on how your software is working.
This means you can simply do:
out_file = open(file_name, 'w') if file_name else sys.stdout
and work with out_file throughout your program.
When you close, remember to check if it's a file or not :)
And have you thought about simply using the logging module? That easily allows you to add different handlers, print to file, print to stdout...

Getting doc string of a python file

Is there a way of getting the doc string of a python file if I have only the name of the file ? For instance I have a python file named a.py. I know that it has a doc string ( being mandated before) but don't know of its internal structure i.e if it has any classes or a main etc ? I hope I not forgetting something pretty obvious
If I know it has a main function I can do it this way that is using import
filename = 'a.py'
foo = __import__(filename)
filedescription = inspect.getdoc(foo.main())
I can't just do it this way:
filename.__doc__ #it does not work

You should be doing...
foo = __import__('a')
mydocstring = foo.__doc__
or yet simpler...
import a
mydocstring = a.__doc__

import ast
filepath = "/tmp/test.py"
file_contents = ""
with open(filepath) as fd:
file_contents = fd.read()
module = ast.parse(file_contents)
docstring = ast.get_docstring(module)
if docstring is None:
docstring = ""
print(docstring)

And if you need the docstrings of the module your are already in :
import sys
sys.modules[__name__].__doc__