I have a Python script that accepts one input (a text file): ./myprog.py file.txt. The script outputs a string based on the given input.
I have a set of test files that I would like to test my program with. I know the expected output for each file and want to make sure my script produces the correct output for each file.
What is the generally accepted way to do this type of testing?
I was thinking of using Python's unittest module as the testing framework, and then running my script through subprocess.check_output(stderr=subprocess.STDOUT), capturing stdout and stderr, and then doing a unittest assertEqual to compare the actual and expected strings. I want to make sure I'm not missing some nicer solution.
There's two problems here. Testing a program, as opposed to a library of functions, and testing something that prints, as opposed to values returned from a function. Both make testing more difficult, so it's best to side step these problems as much as possible.
The usual technique is to create a library of functions and then have your program be a thin wrapper around that. These functions return their results, and only the program does the printing. This means you can use normal unit testing techniques for most of the code.
You can have a single file which is both a library and a program. Here's a simple example as hello.py.
def hello(greeting, place):
return greeting + ", " + place + "!"
def main():
print(hello("Hello", "World"))
if __name__ == '__main__':
main()
That last bit is how a file can tell if it was run as a program or if it was imported as a library. It allows access to the individual functions with import hello, and it also allows the file to run as a program. See this answer for more information.
Then you can write a mostly normal unit test.
import hello
import unittest
import sys
from StringIO import StringIO
import subprocess
class TestHello(unittest.TestCase):
def test_hello(self):
self.assertEqual(
hello.hello("foo", "bar"),
"foo, bar!"
)
def test_main(self):
saved_stdout = sys.stdout
try:
out = StringIO()
sys.stdout = out
hello.main()
output = out.getvalue()
self.assertEqual(output, "Hello, World!\n")
finally:
sys.stdout = saved_stdout
def test_as_program(self):
self.assertEqual(
subprocess.check_output(["python", "hello.py"]),
"Hello, World!\n"
)
if __name__ == '__main__':
unittest.main()
Here test_hello is unit testing hello directly as a function; and in a more complicated program there would be more functions to test. We also have test_main to unit test main using StringIO to capture its output. Finally, we ensure the program will run as a program with test_as_program.
The important thing is to test as much of the functionality as functions returning data, and to test as little as possible as printed and formatted strings, and almost nothing via running the program itself. By the time we're actually testing the program, all we need to do is check that it calls main.
Related
When I run the following code, no error messages are printed, and it seems to fail silently. I expected the string "BLAH" to be printed to the console.
from contextlib import redirect_stdout
import io # the initialism `io` represents `INPUT OUTPUT LIBRARY`
def run_solver():
print("solver is running")
with io.StringIO() as fake_std_out:
with redirect_stdout(fake_std_out):
print("BLAH") # THIS IS NEVER PRINTED
run_solver()
data = fake_std_out.getvalue()
print(data)
print("THE END")
The output I expect is:
BLAH
solver is running
THE END
Instead, we have:
THE END
Edits
I realize now that I wanted to copy standard output, not re-direct it.
Using print to print the contents of the string stream won't work because the destination of the print function is now the string stream instead of the system console. After calling getvalue() it makes no sense to attempt to use print statements anymore.
The code is working fine. print writes to standard output, not the console directly. You can check for the values written to fake_std_out yourself following the inner with statement.
from contextlib import redirect_stdout
import io
def run_solver():
print("solver is running")
with io.StringIO() as fake_std_out:
with redirect_stdout(fake_std_out):
print("BLAH")
run_solver()
assert fake_std_out.getvalue() == 'BLAH\nsolver is running\n'
print("THE END")
Long story short, I am writing python code that occasionally causes an underlying module to spit out complaints in the terminal that I want my code to respond to. My question is if there is some way that I can take in all terminal outputs as a string while the program is running so that I might parse it and execute some handler code. Its not errors that crash the program entirely and not a situation where I can simply do a try catch. Thanks for any help!
Edit: Running on Linux
there are several solutions to your need. the easiest would be to use a shared buffer of sort and get all your package output there instead of stdout (with regular print) thus keeping your personal streams under your package control.
since you probably already have some code with print or you want for it to work with minimal change, i suggest use the contextlib.redirect_stdout with a context manager.
give it a shared io.StringIO instance and wrap all your method with it.
you can even create a decorator to do it automatically.
something like:
// decorator
from contextlib import redirect_stdout
import io
import functools
SHARED_BUFFER = io.StringIO()
def std_redirecter(func):
#functools.wrap(func)
def inner(*args, **kwargs):
with redirect_stdout(SHARED_BUFFER) as buffer:
print('foo')
print('bar')
func(*args, **kwargs)
return inner
// your files
#std_redirecter
def writing_to_stdout_func():
print('baz')
// invocation
writing_to_stdout_func()
string = SHARED_BUFFER.getvalue() // 'foo\nbar\nbaz\n'
I have a simple file program (removed code from functions since its irrelevant for the problem):
import random
import sys
def poziom(level):
*does something and returns something*
def gra(chance):
*does something and returns something*
def save(name):
*does something and returns something*
while True:
*does something in loop, executing other functions above*
I'm trying to run a simple unit test against one of its functions:
import unittest
from my_game import save
from my_game import gra
from my_game import poziom
class TestSum(unittest.TestCase):
def test_list_int(self):
"""
Test that it can sum a list of integers
"""
check = "Test string"
result = poziom(check)
self.assertEqual(result, 0)
if __name__ == '__main__':
unittest.main()
However, this test instead of running a poziom() function, runs through entire file, also executing the code behind the While True statement.
Do I do something wrong? Is it possible in such case to write a test that checks ONLY one function and value returned by it?
However, this test instead of running a poziom() function, runs through entire file, also executing the code behind the While True statement. Do I do something wrong?
This is how import works. It executes the code in the file that you import. To solve the problem put the while loop in a if __name__ == '__main__': just like you do in your test file.
I have this code:
import sys
def random(size=16):
return open(r"C:\Users\ravishankarv\Documents\Python\key.txt").read(size)
def main():
key = random(13)
print(key)
When I try running the script, there are no errors, but nothing appears to happen. I expected it to print some content from the key file, but nothing is printed.
What is wrong? How do I make the code run?
You've not called your main function at all, so the Python interpreter won't call it for you.
Add this as the last line to just have it called at all times:
main()
Or, if you use the commonly seen:
if __name__ == "__main__":
main()
It will make sure your main method is called only if that module is executed as the starting code by the Python interpreter. More about that here: What does if __name__ == "__main__": do?
If you want to know how to write the best possible 'main' function, Guido van Rossum (the creator of Python) wrote about it here.
Python isn't like other languages where it automatically calls the main() function. All you have done is defined your function.
You have to manually call your main function:
main()
Also, you may commonly see this in some code:
if __name__ == '__main__':
main()
There's no such main method in python, what you have to do is:
if __name__ == '__main__':
main()
Something does happen, it just isn't noticeable
Python runs scripts from top to bottom. def is a statement, and it executes when it is encountered, just like any other statement. However, the effect of this is to create the function (and assign it a name), not to call it. Similarly, import is a statement that loads the other module (and makes its code run top to bottom, with its own global-variable context), and assigns it a name.
When the example code runs, therefore, three things happen:
The code for the sys standard library module runs, and then the name sys in our own module's global variables is bound to that module
A function is created from the code for random, and then the name random is bound to that function
A function is created from the code for main, and then the name main is bound to that function
There is nothing to call the functions, so they aren't called. Since they aren't called, the code inside them isn't run - it's only used to create the functions. Since that code doesn't run, the file isn't read and nothing is printed.
There are no "special" function names
Unlike in some other languages, Python does not care that a function is named main, or anything else. It will not be run automatically.
As the Zen of Python says, "Explicit is better than implicit". If we want a function to be called, we have to call it. The only things that run automatically are the things at top level, because those are the instructions we explicitly gave.
The script starts at the top
In many real-world scripts, you may see a line that says if __name__ == '__main__':. This is not "where the script starts". The script runs top to bottom.
Please read What does if __name__ == "__main__": do? to understand the purpose of such an if statement (short version: it makes sure that part of your top-level code is skipped if someone else imports this file as a module). It is not mandatory, and it does not have any kind of special "signalling" purpose to say where the code starts running. It is just a perfectly normal if statement, that is checking a slightly unusual condition. Nothing requires you to use it in a script (aside from wanting to check what it checks), and nothing prevents you from using it more than once. Nothing prevents you from checking whether __name__ is equal to other values, either (it's just... almost certainly useless).
You're not calling the function. Put main() at the bottom of your code.
I've acquired some code that I need to test before refactoring. It uses deep recursion so sets new limits and then runs itself in a fresh thread:
sys.setrecursionlimit(10**6)
threading.stack_size(2**27)
...
threading.Thread(target=main).start()
The code relies heavily on sys.stdin and sys.stdout e.g.
class SpamClass:
def read(self):
self.n = int(sys.stdin.readline())
...
for i in range(self.n):
[a, b, c] = map(int, sys.stdin.readline().split())
...
def write(self)
print(" ".join(str(x) for x in spam()))
To test the code, I need to pass in the contents of a series of input files and compare the results to the contents of some corresponding sample output files.
So far, I've tried three or four different types of mocking and patching without success. My other tests are all written for pytest, so it would be a real nuisance to have to use something else.
I've tried patching module.sys.stdin with StringIO, which doesn't seem to work because pytest's capsys sets sys.stdin to null and hence throws an error despite the patch.
I've also tried using pytest's monkeypatch fixture to replace the module.SpamClss.read method with a function defined in the test, but that produces a segmentation error due, I think, to the thread exiting before the test (or …?).
'pytest test_spam.py' terminated by signal SIGBUS (Misaligned address error)
Any suggestions for how to do this right? Many thanks.
Well, I still don't know what the problem was or if I'm doing this right, but it works for now. I'm not confident the threading aspect is working correctly, but the rest seems fine.
#pytest.mark.parametrize("inputs, outputs", helpers.get_sample_tests('spampath'))
def test_tree_orders(capsys, inputs, outputs):
"""
"""
with patch('module.sys.stdin', StringIO("".join(inputs))):
module.threading.Thread(target=module.main()).start()
captured = capsys.readouterr()
assert "".join(outputs) == captured.out
For anyone else who's interested, it helps to do your debugging prints as print(spam, file=sys.stderr), which you can then access in the test as captured.err, cf. the captured.out used for testing.