I have started using nose to run tests. I discovered the multiprocessing plugin has a timeout, that I can change on the command line.
Is there a way to extend the timeout for individual tests (in the test code) so I don't have a massive global timeout?
I haven't any experience with the multiprocessing plugin but if you subclass the plugin with something like this:
from nose.plugins.multiprocess import MultiProcess
PLUGIN = None
class TimeoutMultiProcess(MultiProcess):
def configure(self, options, conf):
global PLUGIN
PLUGIN = self
super(TimeoutMultiProcess, self).configure(options, conf)
if not self.enabled:
return
then you can create your own test running script like:
import unittest
class TestA(unittest.TestCase):
def setUp(self):
from runtest import PLUGIN
print PLUGIN.config.multiprocess_timeout
def test_a(self):
pass
def test_b(self):
pass
if __name__ == '__main__':
from runtest import TimeoutMultiProcess
import nose
nose.main(addplugins=[TimeoutMultiProcess()], defaultTest="./test.py")
You'll be able to change the config.multiprocess_timeout to different values within your tests. I'm not sure if it will work for you, but it's worth a shot.
Related
I am somewhat of a beginner in python, i am currently writing a suite of test cases with selenium webdriver using unittest; i have also found a lot of useful answers here, but it's time a ask my first question, i have struggled a lot with this and cannot find a proper answer, so any help is greatly appreciated:
For short, i have a suite of multiple tests cases, and in each case the first step is always ".get('Some URL')"; i have written these test cases for a single environment, but i would like to be able to select the URL on which all tests will be executed. In the example below i called the "access_url" method with a specific environment, but i need to do this for all of my scenarios at once, is it possible to do this from where i execute the .py file (e.g. "python example.py")? or to pass it in the .run() method when i select what suite to run?
import HTMLTestRunner
from selenium import webdriver
import unittest
This is a custom class used to create the 'access_url' method
def MyClass(object):
def __init__(self, driver):
self.driver = driver
def access_url(self, URL):
if URL == 'environment 1':
self.driver.get('https://www.google.com/')
elif URL == 'environment 2':
self.driver.get('https://example.com/')
In the classes i use to write test cases the first step is always 'access URL'
class TestScenario01(unittest.TestCase):
def setUp(self):
[...]
def test_01_access(self):
MyClass(self.driver).access_url(URL='environment 2')
def test_02(self):
[...]
def test_03(self):
[...]
In order to run the tests i place them all in a suite and use .run() on them
tc_scenario01 = unittest.TestLoader().loadTestsFromTestCase(TestScenario01)
test_suite = unittest.TestSuite([tc_scenario01])
HTMLReporterCustom.HTMLTestRunner().run(test_suite)
Finally, in order to execute the script i type the follwoing line in CMD: 'python example_file.py
As i mentioned above, all i want to do is to be able to somehow pass the URL one time to all test cases that call the "access_url()" method. Thanks!
You can maintain environment properties in separate config file.
config.py
DEFAULT_ENVIRONMENT='environment1'
URL = {
'environment1': 'https://www.google.com/',
'environment2': 'https://example.com/'
}
Your Class,
from package import config
def MyClass(object):
def __init__(self, driver):
self.driver = driver
def access_url(self):
self.driver.get(config.URL[config.DEFAULT_ENVIRONMENT])
Then test class will be as expected,
class TestScenario01(unittest.TestCase):
def setUp(self):
[...]
def test_01_access(self):
MyClass(self.driver).access_url()
def test_02(self):
[...]
def test_03(self):
[...]
While running test you can change,
main.py
from package import config
config.DEFAULT_ENVIRONMENT = 'enviroment2'
tc_scenario01 = unittest.TestLoader().loadTestsFromTestCase(TestScenario01)
test_suite = unittest.TestSuite([tc_scenario01])
HTMLReporterCustom.HTMLTestRunner().run(test_suite)
You can also pass the environment name while running python main.py.
main.py
if __name__ == '__main__':
config.DEFAULT_ENVIRONMENT = sys.argv[1] if len(sys.argv) > 2 else 'dev'
tc_scenario01 = unittest.TestLoader().loadTestsFromTestCase(TestScenario01)
test_suite = unittest.TestSuite([tc_scenario01])
HTMLReporterCustom.HTMLTestRunner().run(test_suite)
Below is displayed my test class . This is good test when i call it once - python3 main.py.
There is a problem if I want this test run for example 100 times. How to do that?
When I try call by pytest there are warnings sth like this:
main.py::TestLoginPage::test_login_invalid_credentials
/home/-----/.local/lib/python3.5/site-packages/pytest_repeat.py:31: UserWarning: Repeating unittest class tests not supported
"Repeating unittest class tests not supported")
main.py::TestLoginPage::test_login_valid_credentials
/home/-----/.local/lib/python3.5/site-packages/pytest_repeat.py:31: UserWarning: Repeating unittest class tests not supported
"Repeating unittest class tests not supported")
This is my test class - test_login_page.py
import unittest
from selenium import webdriver
from tests.test_driver import TestDriver
from pages.login_page import LoginPage
class TestLoginPage(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.page_url = LoginPage.login_url
cls.webdriver = webdriver
TestDriver.setUp(cls, cls.page_url)
cls.page = LoginPage(cls.webdriver)
cls.option = 1
def __init_test_method(self):
# [TS001].Check if page is loaded correctly
self.assertTrue(self.page.check_page_loaded_correctly)
# [TS002].Check button contains 'login' text
self.assertEqual(self.page.get_button_text(), 'Login')
# TC001
def test_login_valid_credentials(self):
"""Test login with valid credentials"""
self.__init_test_method()
# [TS003].Clear form, fill with correct data and submit
self.assertEqual(self.page.login_process(self.option), 'Complexes')
# TC002 (invalid password) & TC003 (invalid username)
def test_login_invalid_credentials(self):
"""Test login with invalid credentials"""
for option in range(2, 4):
self.__init_test_method()
self.assertTrue(self.page.login_process(option))
#classmethod
def tearDownClass(cls):
cls.webdriver.quit()
This is main which I run all tests from console - main.py
import unittest
import os
import sys
cwd = os.getcwd()
sys.path.append(cwd)
from tests.test_login_page import TestLoginPage
if __name__ == '__main__':
unittest.main()
If you want to run your tests over and over, I would suggest you use ddt or data driven tests. You can find the documentation for complete usage here.
You will decorate your test class with ddt and your test method with file_data or data to pass in unique test parameters.
This will allow you to feed new test parameters for each iteration of your test and will cycle through and run the same test method for however many times you need.
Here is an example of how I use it:
from ddt import ddt, file_data
#ddt
class MyTestClass(unittest.TestCase):
#file_data('test_parameter_file.json')
def some_test_method1(self, test_package):
Some test method
I have the different test parameters in the test_parameter_file.json file and my test method is run for each parameter in the file.
In a hobby project I intend to use nose for testing, I want to put all tests for specific classes into classes since these tests share setup and other functionality. But I can't seem to get nose to execute the setup methods inside the classes.
Here is an example class that is tested:
class mwe():
def __init__(self):
self.example = ""
def setExample(self, ex):
self.example = ex
The tests work, when I don't use classes:
from nose.tools import ok_
import mwe
exampleList = []
def setUp():
print("setup")
exampleList.append("1")
exampleList.append("2")
exampleList.append("3")
def test_Example():
print("test")
for ex in exampleList:
t = mwe.mwe()
t.setExample(ex)
yield check, t, ex
def check(e, ex):
ok_(e.example == ex)
The output is as expected:
setup
test
...
----------------------------------------------------------------------
Ran 3 tests in 0.004s
OK
When a test class is used, the setup method is not executed and therefore no tests are executed.
from nose.tools import ok_
import mwe
class TestexampleClass(object):
def __init__(self):
print("__init__")
self.exampleList = []
def setup(self):
print("setup class")
self.exampleList.append("1")
self.exampleList.append("2")
self.exampleList.append("3")
def test_ExampleClass(self):
print("test class")
for ex in self.exampleList:
t = mwe.mwe()
t.setExample(ex)
yield self.check, t, ex
def check(self, we, ex):
print("check class")
ok_(we.example == ex)
I'm fairly new to python and a newbie with nose, my question is, why is setup not executed? Where is the error in my code?
__init__
test class
----------------------------------------------------------------------
Ran 0 tests in 0.002s
OK
I will be glad for any feedback.
When I use the code from this Question on SO the setup method is executed, as I would expect it.
SOLUTION: After a lot of desperation I found the following:
Nose executes the the class level setup method before the execution of the yielded function, not when the test_* methods are called, as I expected and as is the case for other test_* methods. This obviously goes against the nose documentation:
Setup and teardown functions may be used with test generators. However, please note that setup and teardown attributes attached to the generator function will execute only once. To execute fixtures for each yielded test, attach the setup and teardown attributes to the function that is yielded, or yield a callable object instance with setup and teardown attributes.
Looking at the bug reports, I found the bug report on github.
A possible workaround is to use class level fixtures:
#classmethod
def setup_class(cls):
#do stuff
pass
Your test class needs to extend TestCase and the setup method needs to be called setUp
from unittest import TestCase
class TestUtils(TestCase):
def setUp(self):
self.x = 1
def test_something(self):
self.assertEqual(1, self.x)
which outputs
.
----------------------------------------------------------------------
Ran 1 test in 0.000s
OK
I cannot figure out how to run single unit tests from within a python script and collect the results.
Scenario: I have a battery of tests that check various methods producing various statistical distributions of different objects. The tests sometimes fail, as they should given that I am basically checking for particular kinds of randomness. I would like to run the tests repeatedly from a script or even from the interpreter and collect results for further analysis.
Suppose I have a module myTest.py with:
class myTest(unittest.TestCase):
def setup(self):
...building objects, etc....
def testTest1(self):
..........
def testTest2(self):
..........
Basically I need to:
run the setup method
run testTest1 (say), 100 times
collect the failures
return the failures
The closest I got to was (using code from a similar question):
from unittest import TextTestRunner, TestSuite
runner = TextTestRunner(verbosity = 2)
tests = ['testTest1']
suite = unittest.TestSuite(map(myTest, tests))
runner.run(suite)
But this does not work, because:
runner.run(suite) does not run the setup method
and
I cannot catch the exception it throws when testTest1 fails
you simply need to add the test that you want to run multiple times to the suite.
Here is a complete code example. You can also see this code running in an interactive Python console to prove that it does actually work.
import unittest
import random
class NullWriter(object):
def write(*_, **__):
pass
def flush(*_, **__):
pass
SETUP_COUNTER = 0
class MyTestCase(unittest.TestCase):
def setUp(self):
global SETUP_COUNTER
SETUP_COUNTER += 1
def test_one(self):
self.assertTrue(random.random() > 0.3)
def test_two(self):
# We just want to make sure this isn't run
self.assertTrue(False, "This should not have been run")
def suite():
tests = []
for _ in range(100):
tests.append('test_one')
return unittest.TestSuite(map(MyTestCase, tests))
results = unittest.TextTestRunner(stream=NullWriter()).run(suite())
print dir(results)
print 'setUp was run', SETUP_COUNTER, 'times'
I am using a custom management command to run code that is +500 lines and contains multiple function definitions(and executes database queries).
management/commands/talk.py
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self):
def hello():
print "Hello!"
def my_god():
print "OMG!"
def main():
hello()
my_god()
I can't access the functions inside the handle method seperately (e.g. by doing Command().handle.hello()) in order to unit test them or am I missing a way to do it?
My solution to this is to put all the code under the handle method in management/handle_command.py and then just import that code and run main under handle in management/commands/talk.py. Then I can just unit test the functions from management/handle_command.py.
Example of a proposed management/commands/talk.py
import my_project.my_app.management.handle_command
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def handle(self):
my_project.my_app.management.handle_command.main()
How should I best deal with testing functions inside the handle method of a custom django-admin command module?
First part:
management/commands/talk.py
You will need to define the functions on the class itself using the self
class Command(BaseCommand):
def hello(self):
print "Hello!"
def my_god(self):
print "OMG!"
def handle(self):
# this was def main():
self.hello()
self.my_god()
Second part:
It's obviously preferable not to have these methods stapled to your Command class if you want to use them elsewhere / add them to unittest. If you're only ever using them in the Command class (other than testing) then your suggested method for testing seems most sensible.