Below is displayed my test class . This is good test when i call it once - python3 main.py.
There is a problem if I want this test run for example 100 times. How to do that?
When I try call by pytest there are warnings sth like this:
main.py::TestLoginPage::test_login_invalid_credentials
/home/-----/.local/lib/python3.5/site-packages/pytest_repeat.py:31: UserWarning: Repeating unittest class tests not supported
"Repeating unittest class tests not supported")
main.py::TestLoginPage::test_login_valid_credentials
/home/-----/.local/lib/python3.5/site-packages/pytest_repeat.py:31: UserWarning: Repeating unittest class tests not supported
"Repeating unittest class tests not supported")
This is my test class - test_login_page.py
import unittest
from selenium import webdriver
from tests.test_driver import TestDriver
from pages.login_page import LoginPage
class TestLoginPage(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.page_url = LoginPage.login_url
cls.webdriver = webdriver
TestDriver.setUp(cls, cls.page_url)
cls.page = LoginPage(cls.webdriver)
cls.option = 1
def __init_test_method(self):
# [TS001].Check if page is loaded correctly
self.assertTrue(self.page.check_page_loaded_correctly)
# [TS002].Check button contains 'login' text
self.assertEqual(self.page.get_button_text(), 'Login')
# TC001
def test_login_valid_credentials(self):
"""Test login with valid credentials"""
self.__init_test_method()
# [TS003].Clear form, fill with correct data and submit
self.assertEqual(self.page.login_process(self.option), 'Complexes')
# TC002 (invalid password) & TC003 (invalid username)
def test_login_invalid_credentials(self):
"""Test login with invalid credentials"""
for option in range(2, 4):
self.__init_test_method()
self.assertTrue(self.page.login_process(option))
#classmethod
def tearDownClass(cls):
cls.webdriver.quit()
This is main which I run all tests from console - main.py
import unittest
import os
import sys
cwd = os.getcwd()
sys.path.append(cwd)
from tests.test_login_page import TestLoginPage
if __name__ == '__main__':
unittest.main()
If you want to run your tests over and over, I would suggest you use ddt or data driven tests. You can find the documentation for complete usage here.
You will decorate your test class with ddt and your test method with file_data or data to pass in unique test parameters.
This will allow you to feed new test parameters for each iteration of your test and will cycle through and run the same test method for however many times you need.
Here is an example of how I use it:
from ddt import ddt, file_data
#ddt
class MyTestClass(unittest.TestCase):
#file_data('test_parameter_file.json')
def some_test_method1(self, test_package):
Some test method
I have the different test parameters in the test_parameter_file.json file and my test method is run for each parameter in the file.
Related
py version:Python 3.6.3
Context:
I am using HtmlTestRunner with unittest to get tidy post-deployment test reports for a REST API.
Issue: When I run the tests, I do get everything separated, but the subtests inherit their name 1:1 from the test.
Example view in html:
Example view in html with errors
Example code :
import unittest
import HtmlTestRunner
class testBadRequest(unittest.TestCase):
def test_serviceOne_bad_request(self):
with self.subTest():
x = endpointOne.getAll(acfg_url, cfg.badHeaders, acfg_host, acfg_params)
self.assertEqual(x["status_code"], 400)
with self.subTest():
x = endpointOne.getByID(cfg.badID, acfg_url, acfg_headers, acfg_host, acfg_params)
self.assertEqual(x["status_code"], 400)
if __name__ == '__main__':
unittest.main(testRunner=HtmlTestRunner.HTMLTestRunner(combine_reports=True))
(answering my own question)
Yes, you add the msg parameter when creating a subTest, so:
class testBadRequest(unittest.TestCase):
def test_serviceOne_bad_request(self):
with self.subTest(msg="getAll"):
I am somewhat of a beginner in python, i am currently writing a suite of test cases with selenium webdriver using unittest; i have also found a lot of useful answers here, but it's time a ask my first question, i have struggled a lot with this and cannot find a proper answer, so any help is greatly appreciated:
For short, i have a suite of multiple tests cases, and in each case the first step is always ".get('Some URL')"; i have written these test cases for a single environment, but i would like to be able to select the URL on which all tests will be executed. In the example below i called the "access_url" method with a specific environment, but i need to do this for all of my scenarios at once, is it possible to do this from where i execute the .py file (e.g. "python example.py")? or to pass it in the .run() method when i select what suite to run?
import HTMLTestRunner
from selenium import webdriver
import unittest
This is a custom class used to create the 'access_url' method
def MyClass(object):
def __init__(self, driver):
self.driver = driver
def access_url(self, URL):
if URL == 'environment 1':
self.driver.get('https://www.google.com/')
elif URL == 'environment 2':
self.driver.get('https://example.com/')
In the classes i use to write test cases the first step is always 'access URL'
class TestScenario01(unittest.TestCase):
def setUp(self):
[...]
def test_01_access(self):
MyClass(self.driver).access_url(URL='environment 2')
def test_02(self):
[...]
def test_03(self):
[...]
In order to run the tests i place them all in a suite and use .run() on them
tc_scenario01 = unittest.TestLoader().loadTestsFromTestCase(TestScenario01)
test_suite = unittest.TestSuite([tc_scenario01])
HTMLReporterCustom.HTMLTestRunner().run(test_suite)
Finally, in order to execute the script i type the follwoing line in CMD: 'python example_file.py
As i mentioned above, all i want to do is to be able to somehow pass the URL one time to all test cases that call the "access_url()" method. Thanks!
You can maintain environment properties in separate config file.
config.py
DEFAULT_ENVIRONMENT='environment1'
URL = {
'environment1': 'https://www.google.com/',
'environment2': 'https://example.com/'
}
Your Class,
from package import config
def MyClass(object):
def __init__(self, driver):
self.driver = driver
def access_url(self):
self.driver.get(config.URL[config.DEFAULT_ENVIRONMENT])
Then test class will be as expected,
class TestScenario01(unittest.TestCase):
def setUp(self):
[...]
def test_01_access(self):
MyClass(self.driver).access_url()
def test_02(self):
[...]
def test_03(self):
[...]
While running test you can change,
main.py
from package import config
config.DEFAULT_ENVIRONMENT = 'enviroment2'
tc_scenario01 = unittest.TestLoader().loadTestsFromTestCase(TestScenario01)
test_suite = unittest.TestSuite([tc_scenario01])
HTMLReporterCustom.HTMLTestRunner().run(test_suite)
You can also pass the environment name while running python main.py.
main.py
if __name__ == '__main__':
config.DEFAULT_ENVIRONMENT = sys.argv[1] if len(sys.argv) > 2 else 'dev'
tc_scenario01 = unittest.TestLoader().loadTestsFromTestCase(TestScenario01)
test_suite = unittest.TestSuite([tc_scenario01])
HTMLReporterCustom.HTMLTestRunner().run(test_suite)
I wrote the following code as below:
#base.py
import sys
import unittest
class BaseClass(unittest.TestCase):
def setUp():
print"in base script"
def tearDown():
print"in teardown"
This is the test script:
#test.py
import sys
import unittest
from base import *
class demo_script(BaseClass):
def setUp(self):
self.flag = False
self.abc = None
super(demo_script, self).setUp()
def test_before(self):
self.abc = 5
## reboot occurs here and system context is saved
def test_after(self):
if self.abc == 5:
print"Pass"
else:
print"fail"
def tearDown(self):
print"clean"
The test is failing as it is unable to access the variable self.abc.
How can the local variable be accessed in both the tests i.e. test_before() and test_after()?
Test runners such as Nose don't ensure test method running sequence. So if you set self.abc in test_before it's not sure that test_after is run after test_before.
Anyway, sharing state between test methods other than defined in setUp is a bad idea -- the tests should be isolated.
So merge the test methods into one.
I have multiple tests inside one test case but I am noticing that it is only running the first test
import unittest
from selenium import webdriver
from selenium.webdriver.common.desired_capabilities import DesiredCapabilities
from selenium.webdriver.support.ui import WebDriverWait
class Test(unittest.TestCase):
def setUp(self):
self.driver = webdriver.Firefox()
self.base_url = "http://www.example.com/"
def test_Google(self):
driver = self.driver
driver.implicitly_wait(10)
driver.get(self.base_url)
def fill_contact(self):
driver.find_element_by_xpath('//a[contains(.,"Contact")]').click()
driver.implicitly_wait(10)
driver.find_element_by_xpath('//input[#type="submit"][#value="Send"]').click()
# def tearDown(self):
# self.driver.quit()
if __name__ == "__main__":
unittest.main()
Whenever I run this it only runs
def test_Google(self)
and nothing after that. am I doing something wrong?
Methods must begin with 'test' to be automatically run.
Per the docs:
A testcase is created by subclassing unittest.TestCase. The three
individual tests are defined with methods whose names start with the
letters test. This naming convention informs the test runner about
which methods represent tests. (my emphasis)
The TestLoader is responsible for loading test and returning them wrapped in a TestSuite. It uses this method to identify tests:
class TestLoader(object):
testMethodPrefix = 'test'
def getTestCaseNames(self, testCaseClass):
"""Return a sorted sequence of method names found within testCaseClass
"""
def isTestMethod(attrname, testCaseClass=testCaseClass,
prefix=self.testMethodPrefix):
return attrname.startswith(prefix) and \
hasattr(getattr(testCaseClass, attrname), '__call__')
testFnNames = filter(isTestMethod, dir(testCaseClass))
...
Thus, the attrname.startswith(prefix) checks if the method name begins with 'test'.
As an alternative to what #unubtu noted:
you can use a nose test runner and mark a method with #istest decorator:
from nose.tools import istest
class Test(unittest.TestCase):
...
#istest
def fill_contact(self):
driver.find_element_by_xpath('//a[contains(.,"Contact")]').click()
driver.implicitly_wait(10)
driver.find_element_by_xpath('//input[#type="submit"][#value="Send"]').click()
Besides, here is a pretty good overview of the unittest test discovery:
How does Python's unittest module detect test cases?
I have a Python project with a bunch of tests that have already been implemented, and I'd like to begin benchmarking them so I can compare performance of the code, servers, etc over time. Locating the files in a manner similar to Nose was no problem because I have "test" in the names of all my test files anyway. However, I'm running into some trouble in attempting to dynamically execute these tests.
As of right now, I'm able to run a script that takes a directory path as an argument and returns a list of filepaths like this:
def getTestFiles(directory):
fileList = []
print "Searching for 'test' in " + directory
if not os.path.isdir(os.path.dirname(directory)):
# throw error
raise InputError(directory, "Not a valid directory")
else:
for root, dirs, files in os.walk(directory):
#print files
for f in files:
if "test" in f and f.endswith(".py"):
fileList.append(os.path.join(root, f))
return fileList
# returns a list like this:
# [ 'C:/Users/myName/Desktop/example1_test.py',
# 'C:/Users/myName/Desktop/example2_test.py',
# 'C:/Users/myName/Desktop/folder1/example3_test.py',
# 'C:/Users/myName/Desktop/folder2/example4_test.py'... ]
The issue is that these files can have different syntax, which I'm trying to figure out how to handle. For example:
TestExampleOne:
import dummy1
import dummy2
import dummy3
class TestExampleOne(unittest.TestCase):
#classmethod
def setUpClass(cls):
# set up
def test_one(self):
# test stuff
def test_two(self):
# test stuff
def test_three(self):
# test stuff
# etc...
TestExampleTwo:
import dummy1
import dummy2
import dummy3
def setup(self):
try:
# config stuff
except Exception as e:
logger.exception(e)
def test_one():
# test stuff
def test_two():
# test stuff
def test_three():
# test stuff
# etc...
TestExampleThree:
import dummy1
import dummy2
import dummy3
def setup(self):
try:
# config stuff
except Exception as e:
logger.exception(e)
class TestExampleTwo(unittest.TestCase):
def test_one(self):
# test stuff
def test_two(self):
# test stuff
# etc...
class TestExampleThree(unittest.TestCase):
def test_one(self):
# test stuff
def test_two(self):
# test stuff
# etc...
# etc...
I would really like to be able to write one module that searches a directory for every file containing "test" in its name, and then executes every unit test in each file, providing execution time for each test. I think something like NodeVisitor is on the right track, but I'm not sure. Even an idea of where to start would be greatly appreciated. Thanks
Using nose test runner would help to discover the tests, setup/teardown functions and methods.
nose-timer plugin would help with benchmarking:
A timer plugin for nosetests that answers the question: how much time
does every test take?
Demo:
imagine you have a package named test_nose with the following scripts inside:
test1.py:
import time
import unittest
class TestExampleOne(unittest.TestCase):
#classmethod
def setUpClass(cls):
cls.value = 1
def test_one(self):
time.sleep(1)
self.assertEqual(1, self.value)
test2.py:
import time
value = None
def setup():
global value
value = 1
def test_one():
time.sleep(2)
assert value == 1
test3.py:
import time
import unittest
value = None
def setup():
global value
value = 1
class TestExampleTwo(unittest.TestCase):
def test_one(self):
time.sleep(3)
self.assertEqual(1, value)
class TestExampleThree(unittest.TestCase):
def test_one(self):
time.sleep(4)
self.assertEqual(1, value)
install nose test runner:
pip install nose
install nose-timer plugin:
pip install nose-timer
run the tests:
$ nosetests test_nose --with-timer
....
test_nose.test3.TestExampleThree.test_one: 4.0003s
test_nose.test3.TestExampleTwo.test_one: 3.0010s
test_nose.test2.test_one: 2.0011s
test_nose.test1.TestExampleOne.test_one: 1.0005s
----------------------------------------------------------------------
Ran 4 tests in 10.006s
OK
The result is actually conveniently highlighted:
The coloring can be controlled by --timer-ok and --timer-warning arguments.
Note that time.sleep(n) calls were added for making the manual slowdowns to see the impact clearly. Also note that value variable is set to 1 in "setup" functions and methods, then in test function and methods the value is asserted to be 1 - this way you can see the work of setup functions.
UPD (running nose with nose-timer from script):
from pprint import pprint
import nose
from nosetimer import plugin
plugin = plugin.TimerPlugin()
plugin.enabled = True
plugin.timer_ok = 1000
plugin.timer_warning = 2000
plugin.timer_no_color = False
nose.run(plugins=[plugin])
result = plugin._timed_tests
pprint(result)
Save it into the test.py script and pass a target directory to it:
python test.py /home/example/dir/tests --with-timer
The result variable would contain:
{'test_nose.test1.TestExampleOne.test_one': 1.0009748935699463,
'test_nose.test2.test_one': 2.0003929138183594,
'test_nose.test3.TestExampleThree.test_one': 4.000233173370361,
'test_nose.test3.TestExampleTwo.test_one': 3.001115083694458}