Earlier I was using python unittest in my project, and with it came unittest.TextTestRunner and unittest.defaultTestLoader.loadTestsFromTestCase. I used them for the following reasons,
Control the execution of unittest using a wrapper function which calls the unittests's run method. I did not want the command line approach.
Read the unittest's output from the result object and upload the results to a bug tracking system which allow us to generate some complex reports on code stability.
Recently there was a decision made to switch to py.test, how can I do the above using py.test ? I don't want to parse any CLI/HTML to get the output from py.test. I also don't want to write too much code on my unit test file to do this.
Can someone help me with this ?
You can use the pytest's hook to intercept the test result reporting:
conftest.py:
import pytest
#pytest.hookimpl(hookwrapper=True)
def pytest_runtest_logreport(report):
yield
# Define when you want to report:
# when=setup/call/teardown,
# fields: .failed/.passed/.skipped
if report.when == 'call' and report.failed:
# Add to the database or an issue tracker or wherever you want.
print(report.longreprtext)
print(report.sections)
print(report.capstdout)
print(report.capstderr)
Similarly, you can intercept one of these hooks to inject your code at the needed stage (in some cases, with the try-except around yield):
pytest_runtest_protocol(item, nextitem)
pytest_runtest_setup(item)
pytest_runtest_call(item)
pytest_runtest_teardown(item, nextitem)
pytest_runtest_makereport(item, call)
pytest_runtest_logreport(report)
Read more: Writing pytest plugins
All of this can be easily done either with a tiny plugin made as a simple installable library, or as a pseudo-plugin conftest.py which just lies around in one of the directories with the tests.
It looks like pytest lets you launch from Python code instead of using the command line. It looks like you just pass the same arguments to the function call that would be on the command line.
Pytest will create resultlog format files, but the feature is deprecated. The documentation suggests using the pytest-tap plugin that produces files in the Test Anything Protocol.
I'm pretty new to Python. However, I am writing a script that loads some data from a file and generates another file. My script has several functions and it also needs two user inputs (paths) to work.
Now, I am wondering, if there is a way to test each function individually. Since there are no classes, I don't think I can do it with Unit tests, do I?
What is the common way to test a script, if I don't want to run the whole script all the time? Someone else has to maintain the script later. Therefore, something similar to unit tests would be awesome.
Thanks for your inputs!
If you write your code in the form of functions that operate on file objects (streams) or, if the data is small enough, that accept and return strings, you can easily write tests that feed the appropriate data and check the results. If the real data is large enough to need streams, but the test data is not, use the StringIO function in the test code to adapt.
Then use the __name__=="__main__" trick to allow your unit test driver to import the file without running the user-facing script.
I am writing a Python script for collecting data from running tests under different conditions. At the moment, I am interested in adding support for Py.Test.
The Py.Test documentation clearly states that running pytest inside Python code is supported:
You can invoke pytest from Python code directly... acts as if you would call “pytest” from the command line...
However, the documentation does not describe in detail return value of calling pytest.main() as prescribed. The documentation only seems to indicate how to read the exit code of calling the tests.
What are the limits of data resolution available through this interface? Does this method simply return a string indicating the results of the test? Is support more friendly data structures supported (e.g., outcome of each test case assigned to key, value pair)?
Update: Examining the return data structure in the REPL reveals that calling pytest.main yeilds an integer return type indicating system exit code and directs a side-effect (stream of text detailing test result) to standard out. Considering this is the case, does Py.Test provide an alternate interface for accessing the result of tests run from within python code through some native data structure (e.g., dictionary)? I would like to avoid catching and parsing the std.out result because that approach seems error prone.
I don`t think so, the official documentation tells us that pytest.main
returns an os error code like is described in the example.
here
You can use the pytest flags if you want to, even the traceback (--tb) option to see if some of those marks helps you.
In your other point about parsing the std.out result because that approach seems error prone.
It really depends on what you are doing. Python has a lot of packages to do it like subprocess for example.
I have a completely non-interactive python program that takes some command-line options and input files and produces output files. It can be fairly easily tested by choosing simple cases and writing the input and expected output files by hand, then running the program on the input files and comparing output files to the expected ones.
1) What's the name for this type of testing?
2) Is there a python package to do this type of testing?
It's not difficult to set up by hand in the most basic form, and I did that already. But then I ran into cases like output files containing the date and other information that can legitimately change between the runs - I considered writing something that would let me specify which sections of the reference files should be allowed to be different and still have the test pass, and realized I might be getting into "reinventing the wheel" territory.
(I rewrote a good part of unittest functionality before I caught myself last time this happened...)
I guess you're referring to a form of system testing.
No package would know which parts can legitimately change. My suggestion is to mock out the sections of code that result in the changes so that you can be sure that the output is always the same - you can use tools like Mock for that. Comparing two files is pretty straightforward, just dump each to a string and compare strings.
Functional testing. Or regression testing, if that is its purpose. Or code coverage, if you structure your data to cover all code paths.
Google has a Python tutorial, and they describe boilerplate code as "unfortunate" and provide this example:
#!/usr/bin/python
# import modules used here -- sys is a very standard one
import sys
# Gather our code in a main() function
def main():
print 'Hello there', sys.argv[1]
# Command line args are in sys.argv[1], sys.argv[2] ..
# sys.argv[0] is the script name itself and can be ignored
# Standard boilerplate to call the main() function to begin
# the program.
if __name__ == '__main__':
main()
Now, I've heard boilerplate code being described as "seemingly repetitive code that shows up again and again in order to get some result that seems like it ought to be much simpler" (example).
Anyways, in Python, the part considered "boilerplate" code of the example above was:
if __name__ == '__main__':
main()
Now, my questions are as follows:
1) Does boilerplate code in Python (like the example provided) take on the same definition as the definition I provided? If so, why?
2) Is this code even necessary? It seems to me like the code runs whether or not there's a main method. What makes using this code better? Is it even better?
3) Why do we use that code and what service does it provide?
4) Does this occur throughout Python? Are there other examples of "boilerplate code"?
Oh, and just an off topic question: do you need to import sys to use command line arguments in Python? How does it handle such arguments if its not there?
It is repetitive in the sense that it's repeated for each script that you might execute from the command line.
If you put your main code in a function like this, you can import the module without executing it. This is sometimes useful. It also keeps things organized a bit more.
Same as #2 as far as I can tell
Python is generally pretty good at avoiding boilerplate. It's flexible enough that in most situations you can write code to produce the boilerplate rather then writing boilerplate code.
Off topic question:
If you don't write code to check the arguments, they are ignored.
The reason that the if __name__ == "__main__": block is called boilerplate in this case is that it replicates a functionality that is automatic in many other languages. In Java or C++, among many others, when you run your code it will look for a main() method and run it, and even complain if it's not there. Python runs whatever code is in your file, so you need to tell it to run the main() method; a simple alternative would be to make running the main() method the default functionality.
So, if __name__ == "__main__": is a common pattern that could be shorter. There's no reason you couldn't do something different, like:
if __name__ == "__main__":
print "Hello, Stack Overflow!"
for i in range(3):
print i
exit(0)
This will work just fine; although my example is a little silly, you can see that you can put whatever you like there. The Python designers chose this behavior over automatically running the main() method (which may well not exist), presumably because Python is a "scripting" language; so you can write some commands directly into a file, run it, and your commands execute. I personally prefer it the Python way because it makes starting up in Python easier for beginners, and it's always nice to have a language where Hello World is one line.
The reason you use an "if main" check is so you can have a module that runs some part of its code at toplevel (to create the things – constants, functions, or classes – it exports), and some part only when executed as a script (e.g. unit tests for its functionality).
The reason the latter code should be wrapped in a function is because local variables of the main() block would leak into the module's scope.
Now, an alternate design could be that a file executed as a script would have to declare a function named, say, __main__(), but that would mean adding a new magic function name to the language, while the __name__ mechanism is already there. (And couldn't be removed, because every module has to have a __name__, and a module executed as a script has to have a "special" name because of how module names are assigned.) Introducing two mechanisms to do the same thing just to get rid of two lines of boilerplate – and usually two lines of boilerplate per application – just doesn't seem worth it.
You don't need to add a if __name__ == '__main__' for one off scripts that aren't intended to be a part of a larger project. See here for a great explanation. You only need it if you want to run the file by itself AND include it as a module along with other python files.
If you just want to run one file, you can have zero boilerplate:
print 1
and run it with $ python your_file.py
Adding the shebang line #!/usr/bin/python and running chmod +x print_one.py gets you the ability to run with
./print_one.py
Finally, # coding: utf-8 allows you to add unicode to your file if you want to put ❤'s all over the place.
1) main boilerplate is common, but cannot be any simpler
2) main() is not called without the boilerplate
3) the boilerplate allows module usage both as a standalone script, and as a library in other programs
4) it’s very common. doctest is another one.
Train to become a Python guru…and good luck with the thesis! ;-)
Let’s take a moment to see what happened when you called import sys:
Python looks at a list and brings in the sys module
It finds the argv function and runs it
So, what’s happening here?
A function written elsewhere is being used to perform certain operations within the scope of the current program. Programming in this fashion has a lots of benefits. It separates the logic from actual labour.
Now, as far as the boilerplate is concerned, there are two parts:
the program itself (the logic), defined under main, and
the call part that checks if main exists
You essentially write your program under main, using all the functions you defined just before defining main (or elsewhere), and let Python look for main.
I am equally confused by what the tutorial means by "boilerplate code": does it mean that this code can be avoided in a simple script? Or it is a criticism towards Python features that force the use of this syntax? Or even an invitation to use this "boilerplate" code?
I don't know, however, after many years of Python programming, I have at least clear what the different syntaxes do, even if I am probably still not sure on what is the best way of doing it.
Often you want to put at the end of the script code for tests or code that want to execute, but this has some implications/side-effects:
the code gets executed even when the script is imported, that it is rarely what is wanted.
variables and values in the code are defined and exported in the calling namespace
the code at the end of the script can be executed by calling the script (python script.py) or by running from ipython shell (%run script.py), but there is no way to run it from other scripts.
The most basic mechanism to avoid to execute following code in all conditions, is the syntax:
if __name__ == '__main__':
which makes the code run only if the script is called or run, avoiding problem 1. The other two points still hold.
The "boilerplate" code with a separate main() function, adds a further step, excluding also above points 2 and 3, so for example you can call a number of tests from different scripts, that sometimes can take another level (e.g.: a number of functions, one for each test, so they can be individually be called from outside, and a main that calls all test functions, without needs to know from outside which one they are).
I add that the main reason I find this structures often unsatisfying, apart from its complexity, is that sometimes I would like to maintain point 2 and I lose this possibility if the code is moved to a separate function.