Develop Reference

How to mock imwrite in a loop

I am trying to use Python unittest library to test the following code.
from cv2 import imwrite
import h5py
import lumpy as np
class Myclass:
def __init__(self):
self.data = []
def get_data(self):
return self.data
def load_data(self, path):
count = 10
for i in range(count):
mat_file = path + f'{i}.mat'
with h5py.File(mat_file, 'r') as fin:
data = np.array(fin['data'])
for j in range(len(data)):
filename = path + f'{i}_{j}.jpg'
imwrite(filename, data)
self.data = data
I tried to do the following. But I am getting AttributeError saying Myclass has no attribute 'imwrite'. And I do not know how to mock the nested loops with image writing imwrite.
from mock import MagicMock, patch
def test():
m = MagicMock()
m.__enter__.return_value = data
with patch("h5py.File", return_value=m):
with patch(Myclass.imwrite) as mock_imwrite:
testclass = Myclass()
testclass.load_data(path='test')
assert testclass.get_data() == data
I hope someone can help me out. Any help is very much appreciated

The issue here is that your class Myclass does not have a function imwrite. Only the module (aka the file the class is defined in) is aware of this function.
If you want to patch the imwrite function of the package cv2, you have to write:
(untested code)
with patch('cv2.imwrite') as mock_imwrite:
testclass = Myclass()
...
But the as mock_imwrite part is only needed, if you run an assert on the mock. Otherwise, you may just skip it.

Python unittest a module instead of a function

I'm trying to write a test.py file to test a module I wrote. The specifications of the program are that I take serial user input and then print, not return a single answer. The first line of user input indicates how many inputs will follow. With an example program, "4\n1\n2\n3\n4\n" would mean there are 4 inputs and the inputs are [1,2,3,4]. Here is an example of the program that would take the input (sumEx.py):
import sys
def sum():
n = int(sys.stdin.readline().strip())
nums = []
for _ in range(n):
nums.append(int(sys.stdin.readline().strip()))
result = 0
for num in nums:
result += num
print(result)
if __name__ == "__main__":
sum()
I realize that in this example the for loop is redundant, but this is just an example for the actual program I am working on to abstract the problem. Currently, this is the test file I have:
from io import StringIO
import sys
from _pytest.monkeypatch import MonkeyPatch
import unittest
from sumEx import sum as program
class Testing(unittest.TestCase):
def test_string(self):
monkeypatch = MonkeyPatch()
monkeypatch.setattr('sys.stdin', StringIO("8\n1\n2\n3\n4\n5\n6\n7\n8\n"))
self.assertEqual(program(), 36)
def test_boolean(self):
monkeypatch = MonkeyPatch()
monkeypatch.setattr('sys.stdin', StringIO("4\0\n1\n2\n3\n"))
self.assertEqual(program(), 6)
if __name__ == '__main__':
unittest.main()
The problem is that my tests will only work if I returned them instead of printing them. Ideally, my test file would call the file sumEx.py and then
if __name__ == "__main__":
sum()
would call the sum function, the test would supply the input (like an actual person typing each line), and then whatever sum prints would be considered the output for the test. Any help is greatly appreciated. Please ask any questions if something is too vague. Thank you!

If anyone is curious, this is what I'm gonna go with for now. This takes input from a file and mimics user input through sys.stdin. It then reads the correct output from a file and compares it to the output of the program. It also runs the same test case with different inputs with the help of parameterization. Thank you #MrBeanBremen for the suggestion!
class Testing(unittest.TestCase):
def load_file(self, fileName, inOut):
try:
inputFile = open(fileName, 'r')
fileInput = r'{}'.format(inputFile.readline())
for line in inputFile.readlines():
fileInput = r'{}'.format(fileInput + line)
if inOut == 'in':
fileInput = r'{}'.format(fileInput+'\n')
inputFile.close()
return fileInput
except:
print("ERROR LOADING FILE")
#parameterized.expand([
["./tests/test0in.txt", "./tests/test0out.txt"],
["./tests/test1in.txt", "./tests/test1out.txt"],
["./tests/test2in.txt", "./tests/test2out.txt"]])
#patch('sys.stdout', new_callable=StringIO)
def test(self, inputFile, outputFile, mock_stdout):
monkeypatch = MonkeyPatch()
monkeypatch.setattr('sys.stdin', StringIO(self.load_file(inputFile, "in")))
program.main()
self.assertEqual(mock_stdout.getvalue(), self.load_file(outputFile, "out"))
if __name__ == '__main__':
unittest.main(verbosity=2)

context manager and helper function in python

I have a few functions that use context manager:
def f1():
with open("test.txt","r+") as f:
f.write("common Line")
f.write("f1 Line")
def f2():
with open("test.txt","r+") as f:
f.write("common Line")
f.write("f2 Line")
def f3():
with open("test.txt","r+") as f:
f.write("common Line")
f.write("f3 Line")
These functions have a few common lines. So I want to add a helper function. Something like this
def helperF():
with open("test.txt","r+") as f:
f.write("common Line")
And then somehow call it from my f1,f2,f3 functions to make the code DRY.
But I'm not quite sure how to deal with context manager in this situation. The following will not work because f is already closed by the time the function is called:
def f1():
commonHelper()
f.write("f1 Line")
def f2():
commonHelper()
f.write("f2 Line")
def f3():
commonHelper()
f.write("f3 Line")

If the three functions are each writing quite a bit to the file, I would recommend refactoring so that they return a list of strings to be written, instead of having multiple functions which write directly to the file.
def write_with_common_header(lines):
with open("test.txt", "r+") as f:
f.write("common Line")
for line in lines:
f.write(line)
def f1():
return ["f1 Line"]
def f2():
return ["f2 Line"]
def f3():
return ["f3 Line"]
# usage example:
write_with_common_header(f2())
If the lists returned by each function will always be the same, then there is no need for them to even be functions; you can just declare them as lists.
In the more general case where the context manager isn't necessarily a file, and the separate functions are doing more than just calling a single method, then we can't just pass them in as data, but the same technique can be applied: make the write_with_common_header function accept an argument so its behaviour can be parameterised. For full generality, the argument should be a function which accepts a reference to the managed resource f.
def common_helper(callback):
with open("test.txt", "r+") as f:
f.write("common Line")
callback(f)
def f1(f):
f.write("f1 Line")
def f2(f):
f.write("f2 Line")
def f3(f):
f.write("f3 Line")
# usage example:
common_helper(f2)

Reduce indentation of nested functions in Python code generation

My goal is to generate functions dynamically and then save them in a file. For e.g, in my current attempt, On calling create_file
import io
def create_file():
nested_func = make_nested_func()
write_to_file([nested_func, a_root_func], '/tmp/code.py')
def a_root_func(x):
pass
def make_nested_func():
def a_nested_func(b, k):
return b, k
return a_nested_func
def write_to_file(code_list, path):
import inspect
code_str_list = [inspect.getsource(c) for c in code_list]
with open(path, 'w') as ofh:
for c in code_str_list:
fh = io.StringIO(c)
ofh.writelines(fh.readlines())
ofh.write('\n')
create_file()
The output I want is('/tmp/code.py'):
def a_nested_func(b, k):
return b, k
def a_root_func(x):
pass
The output I get is('/tmp/code.py'):
def a_nested_func(b, k):
return b, k
def a_root_func(x):
pass
a_nested_func is indented. How can I reduce the indentation? I can do lstrip etc. but I wonder if there is a better way.

Use the textwrap.dedent() function to remove the common leading whitespace:
import inspect
from textwrap import dedent
def write_to_file(code_list, path):
code_str_list = [inspect.getsource(c) for c in code_list]
with open(path, 'w') as ofh:
for c in code_str_list:
dedented = dedent(c)
ofh.write(dedented + '\n')
Note that there is no need for a StringIO(string).readlines() dance here.

There's a function in built-in module, textwrap.dedent.
import textwrap
s = """
abc
def
"""
s2 = """
abc
def
"""
assert textwrap.dedent(s) == s2

How to use the same line of code in all functions?

I am newbie in Python.
I wonder if it is possible that all functions inherit the same line of code?
with open(filename, 'r') as f: as this line of code is the same in all three functions. Is it possible to inherit the code without using classes?
I tried to find the answer on stackoverflow and python documentation, but with no luck.
def word_count(filename):
with open(filename, 'r') as f:
return len(f.read().split())
def line_count(filename):
with open(filename, 'r') as f:
return len(f.read().splitlines())
def character_count(filename):
with open(filename, 'r') as f:
return len(f.read())

The common code in your case is
with open(filename, 'r') as f:
contents = f.read()
So just move it to its own function:
def get_file_contents(filename):
with open(filename, 'r') as f:
return f.read()
def word_count(filename):
return len(get_file_contents(filename).split())
def line_count(filename):
return len(get_file_contents(filename).splitlines())
def character_count(filename):
return len(get_file_contents(filename))

What I've done in the past is split the code out into another function, in your example
with open(filename, 'r') as f:
f.read()
Is common within all of your methods, so I'd look at rewriting it like so.
def read_file(filename):
with open(filename, 'r') as f:
return f.read()
def word_count(filename):
return len(read_file(filename).split())
def line_count(filename):
return len(read_file(filename).splitlines())
def character_count(filename):
return len(read_file(filename))

I would use a class:
class Count:
""" Object holds everything count-related """
def __init__(self, filename):
""" specify filename in class instance """
with open(filename, 'r') as f:
self.content = f.read()
def word_count(self):
return len(self.content.split())
def line_count(self):
return len(self.content.splitlines())
def character_count(self):
return len(self.content)
file = Count("whatever.txt")
print(file.word_count())
print(file.line_count())
print(file.character_count())

What you do differently is after you open the file, so if I were in your shoes, I would write a function which takes another function that is executed after the file is opened.
Let's illustrate this in an example:
>>> def operate_file(filename, func):
... with open(filename, 'r') as f:
... return func(f)
>>> def line_count(f):
... return len(f.read().splitlines())
>>> def word_count(f):
... return len(f.read().split())
>>> def character_count(f):
... return len(f.read())
>>> print operate_file('/tmp/file.txt', line_count)
1200
>>> print operate_file('/tmp/file.txt', word_count)
2800
>>> print operate_file('/tmp/file.txt', character_count)
29750

I would recommend decorators. It's sort of like the making the repeated line of code into a function, but since you are going to call that function on each input anyway, decorators can let you just write the functions as id f was the input.
The #open_file is a shorthand for word_count=open_file(word_count).
here is a good place to read more about python decorators.
def open_file(func):
def wrapped_func(filename):
with open(filename, 'r') as f:
return func(f)
return wrapped_func
#open_file
def word_count(f):
return len(f.read().split())
#open_file
def line_count(f):
return len(f.read().splitlines())
#open_file
def character_count(f):
return len(f.read())

It depends on, what you want to do with the results of your 3 functions. Every function is opening the same file. That happens 3 times just to get 3 different properties.
One good solution would be a class. But another would be to rearange your functions to just one. That could return a dictionary or named tuple with the results.
It would look something like this:
def file_count(filename):
with open(filename, 'r') as f:
content = f.read()
properties = {}
properties['words'] = len(content.split())
properties['lines'] = len(content.splitlines())
properties['chars'] = len(content)
return properties