I am using PyTest framework for writing and running my tests.
I have implemented a concrete logger:
class Logger(object):
class LogFormats:
...
def __init__(self, testname ,setup ,silent=True):
"""
creating concrete logger for pytest.
the logger will create a file for the test in specific test directory in quali FS and will
write to this file all test log output (colored).
:param: testname: test name - recieved from pytest fixtures (command line parameters)
:param: setup: test setup - recieved from pytest fixtures (command line parameters)
:param: silent: log test in silent mode (info only) or not silent mode (everything is logged)
:param: root_password: password for root user
"""
....
...
and in the conftest.py file I wrote the function that will be invoked when this logger will be requested (creating a logger fixture)
#pytest.fixture(scope="module",autouse=True)
def logger(request):
setup = request.config.getoption('--setupname')
logger = Logger(testname=request.node.name, setup=setup)
return logger
Now, my question is how do I make this concrete logger global using pytest?
Meaning I don't want to pass it as argument to the test function like this:
def test_logger(other_fixture,logger):
but still be able to use it inside the test_logger test function (like global variable)
You could do
#pytest.mark.usefixtures("logger")
def test_logger(other_fixture):
Related
I am using pytest and I wish to log some log output that will be captured and displayed should a test fail.
To do this, I wish to add a new log level to the default pytest logger. My rationale for this is that my application outputs to the INFO level anyway, so I wish to avoid capturing those logs during the test. For that reason, I cannot simply change the log level to INFO.
Using the answers to this question I have derived the following solution, but it is not working. The error I see appears to be noted in a comment on accepted answer to the question, but there is no response.
AttributeError: module 'logging' has no attribute 'test_info'
How can I correctly add a custom logging level?
Using pytest fixture and tests
import logging
import pytest
#pytest.fixture(scope='session', autouse=True)
def logging_format(level_name: str='TEST_INFO', level_number: int=31):
def log_test_info(self, message, *args, **kwargs):
if self.isEnabledFor(level_number):
self._log(level_number, message, args, **kwargs)
logging.addLevelName(level_number, level_name)
setattr(logging, level_name, level_number)
setattr(logging.getLoggerClass(), level_name.lower(), log_test_info)
def test():
logging.test_info('Some log data.')
assert True
Not using pytest fixture and tests
To be sure this is not a pytest issue, I it's also possible to replicate the issue without using it.
import logging
level_name = 'TEST_INFO'
level_number = 31
def log_test_info(self, message, *args, **kwargs):
if self.isEnabledFor(level_number):
self._log(level_number, message, args, **kwargs)
logging.addLevelName(level_number, level_name)
setattr(logging, level_name, level_number)
setattr(logging.getLoggerClass(), level_name.lower(), log_test_info)
logging.test_info('Some log data.')
assert True
In this code there is a misunderstanding between a logger instance and the logger module.
To be able to call the test_log method in the original code you need to obtain a logger instance eg:
# After the existing code
logger = logging.getLogger()
logger.test_info("Some log data.")
As you have added the test_info method to the loggerClass it will be available to the logger instance.
To have the method available on the logging module the function needs to be created for that purpose and added to the module:
# After the existing code
def module_test_info(message, *args, **kwargs):
logging.log(level_number, message, *args, **kwargs)
setattr(logging, level_name.lower(), module_test_info)
This will add the module_test_info function to the logging module with the name test_info you will be able to call it as such logging.test_level("Some log data.")
I'm new to python unit test. I learned and did sample unit test where method accepts input and returns output.
But for code as mentioned below, I've some questions.
How I mock configparser of init method in unittest? Path /config/program.cfg is on production server and not exists in dev directory. program.cfg file exists at other location in code directory. Is there a way to handle that in unittest?
How I send or skip something like hardcoded path in unittest e.g. /var/log/info_server.log
If possible, can you please tell me how would you write unittest for below code using pytest module? This will be helpful to understand the flow and I can do that with rest of code.
def __init__(self):
self.setup_logger()
# Read config parameters
config = configparser.ConfigParser()
config.read("/config/program.cfg")
self.host_ip = config.get('default','HostIP')
self.redis_ip = config.get('default','RedisIP')
self.redis_port = config.get('default','RedisPort')
self.info_port = config.get('default','InfoPort')
self.sqlite_db_file = config.get('default','SQLiteDbFile')
self.connect_redis()
def setup_logger(self):
#Initialize logger
self.logger = logging.getLogger(__name__)
self.logger.setLevel(logging.INFO)
# Create a file handler
handler = logging.FileHandler('/var/log/info_server.log')
handler.setLevel(logging.INFO)
# Create a logging format
formatter = logging.Formatter('%(asctime)s - %(name)s - %(levelname)s - %(message)s')
handler.setFormatter(formatter)
# Add the handlers to the logger
self.logger.addHandler(handler)
def connect_redis(self):
# Start up a Redis instance
self.logger.info("Start Redis instance")
self.ad_info = redis.StrictRedis(host=self.redis_ip, port=self.redis_port, db=0)
if self.ad_info is None:
self.logger.error("Failed to start Redis instance")
else:
self.logger.info("Started Redis instance")
Sure, we could mock many things, but a simple re-factoring the class being tested, adding config and log, will make life much easier.
def __init__(self, cfg='/config/program.cfg', log='/var/log/info_server.log'):
In development server where /config/program.cfg is not present, you could just
TheClass(cfg='~/dev.cfg', log='/tmp/dev.log')
self.server_obj.connect_redis() log_mock.logger.info.assert_called_with('Started Redis instance') not working
import logging
logging.getLogger
# <function getLogger at 0x7f3d2ed427b8>
logger = logging.getLogger()
type(logger)
# <class 'logging.RootLogger'>
logger.info
# <bound method Logger.info of <RootLogger root (WARNING)>>
The patch should be sth like this:
with mock.patch.object(logging.RootLogger, 'info') as mock_info:
server_obj = Server(cfg='sth', log='sth')
mock_info.assert_called_with('xxx')
I want to test a class that does logging when initialised and save logs to local file. Therefore, I'm mocking the logging piece of logic in order to avoid file IO when testing. This is pseudo-code representing how I've structured the tests
class TestClass:
def test_1(self, monkeypatch):
monkeypatch.setattr('dotted.path.to.logger', lambda *args: '')
assert True
def test_2(self, monkeypatch):
monkeypatch.setattr('dotted.path.to.logger', lambda *args: '')
assert True
def test_3(self, monkeypatch):
monkeypatch.setattr('dotted.path.to.logger', lambda *args: '')
assert True
Note how monkeypatch.setattr() is copy-pasted across all methods. Considering that:
we know a priori that all call methods will need to be monkey-patched in the same way, and
one might forget to monkeypatch new methods,
I think that monkey-patching should be abstracted at class level. How do we abstract monkeypatching at class level? I would expect the solution to be something similar to what follows:
import pytest
class TestClass:
pytest.monkeypatch.setattr('dotted.path.to.logger', lambda *args: '')
def test_1(self):
assert True
def test_2(self):
assert True
def test_3(self):
assert True
This is where loggers are configured.
def initialise_logger(session_dir: str):
"""If missing, initialise folder "log" to store .log files. Verbosity:
CRITICAL, ERROR, WARNING, INFO, DEBUG, NOTSET."""
os.makedirs(session_dir, exist_ok=True)
logging.basicConfig(filename=os.path.join(session_dir, 'session.log'),
filemode='a',
level=logging.INFO,
datefmt='%Y-%m-%d %H:%M:%S',
format='|'.join(['(%(threadName)s)',
'%(asctime)s.%(msecs)03d',
'%(levelname)s',
'%(filename)s:%(lineno)d',
'%(message)s']))
# Adopt NYSE time zone (aka EST aka UTC -0500 aka US/Eastern). Source:
# https://stackoverflow.com/questions/32402502/how-to-change-the-time-zone-in-python-logging
logging.Formatter.converter = lambda *args: get_now().timetuple()
# Set verbosity in console. Verbosity above logging level is ignored.
console = logging.StreamHandler()
console.setLevel(logging.ERROR)
console.setFormatter(logging.Formatter('|'.join(['(%(threadName)s)',
'%(asctime)s',
'%(levelname)s',
'%(filename)s:%(lineno)d',
'%(message)s'])))
logger = logging.getLogger()
logger.addHandler(console)
class TwsApp:
def __init__(self):
initialise_logger(<directory>)
A cleaner implementation:
# conftest.py
import pytest
#pytest.fixture(autouse=True)
def dont_configure_logging(monkeypatch):
monkeypatch.setattr('twsapp.client.initialise_logger', lambda x: None)
You don't need to mark individual tests with the fixture, nor inject it, this will be applied regardless.
Inject the caplog fixture if you need to assert on records logged. Note that you don't need to configure loggers in order to make logging assertions - the caplog fixture will inject the necessary handlers it needs in order to work correctly. If you want to customise the logging format used for tests, do that in pytest.ini or under a [tool:pytest] section of setup.cfg.
In practice, I've put the fixture in /test/conftest.py. In fact, pytest automatically load fixture from files named conftest.py and can be applied in any module during the testing session.
from _pytest.monkeypatch import MonkeyPatch
#pytest.fixture(scope="class")
def suppress_logger(request):
"""Source: https://github.com/pytest-dev/pytest/issues/363"""
# BEFORE running the test.
monkeypatch = MonkeyPatch()
# Provide dotted path to method or function to be mocked.
monkeypatch.setattr('twsapp.client.initialise_logger', lambda x: None)
# DURING the test.
yield monkeypatch
# AFTER running the test.
monkeypatch.undo()
import pytest
#pytest.mark.usefixtures("suppress_logger")
class TestClass:
def test_1(self):
assert True
def test_2(self):
assert True
def test_3(self):
assert True
EDIT: I ended up using the following in conftest.py
#pytest.fixture(autouse=True)
def suppress_logger(mocker, request):
if 'no_suppress_logging' not in request.keywords:
# If not decorated with: #pytest.mark.no_suppress_logging_error
mocker.patch('logging.error')
mocker.patch('logging.warning')
mocker.patch('logging.debug')
mocker.patch('logging.info')
I have the following code in my unittest TestCase:
class StatsdLogHandler(logging.Handler):
def emit(self, record):
log_entry = self.format(record)
statsd_client.incr(log_entry)
def setup_statsd_logging(logger, level=logging.WARNING):
# set up statsd counts for logging
statsd_format = "pegasus.logger.%(levelname)s.%(name)s"
formatter = logging.Formatter(statsd_format)
statsd_handler = StatsdLogHandler()
statsd_handler.setFormatter(formatter)
statsd_handler.setLevel(level)
logger.addHandler(statsd_handler)
class TestLogging(unittest.TestCase):
#mock.patch('.statsd_client')
def test_logging_to_statsd(self, statsd_mock):
"""Make sure log calls go to statsd"""
root_logger = logging.getLogger()
setup_statsd_logging(root_logger)
root_logger.setLevel(logging.DEBUG)
root_logger.warning('warning')
statsd_mock.incr.assert_called_with('pegasus.logger.WARNING.root')
However, when I run the TestLogging test case, I get
AssertionError: Expected call: incr(u'pegasus.logger.WARNING.root')
Not called
I would expect this call to be successful. Why is this happening?
When I debugged this behavior, I found that root_logger.manager.disable was set to the value of 30, meaning that all logging with a level of WARNING or lower was not being printed.
The logging Manager object isn't documented anywhere that I could easily find, but it is the object that describes and stores all the loggers, including the root logger. It is defined in code here. What I found was that there were calls elsewhere in the test cases to logging.disable(logging.WARNING). This sets the value of root_logger.manager.disable to 30. The manager disable level trumps the effective level of the logger (see this code).
By adding the line logging.disable(logging.NOTSET) to my test case at the beginning of the test case, I ensure that the logging manager disable property was set to the default value of 0, which caused my test cases to pass.
I am using selenium for end to end testing and I can't get how to use setup_class and teardown_class methods.
I need to set up browser in setup_class method, then perform a bunch of tests defined as class methods and finally quit browser in teardown_class method.
But logically it seems like a bad solution, because in fact my tests will not work with class, but with object. I pass self param inside every test method, so I can access objects' vars:
class TestClass:
def setup_class(cls):
pass
def test_buttons(self, data):
# self.$attribute can be used, but not cls.$attribute?
pass
def test_buttons2(self, data):
# self.$attribute can be used, but not cls.$attribute?
pass
def teardown_class(cls):
pass
And it even seems not to be correct to create browser instance for class.. It should be created for every object separately, right?
So, I need to use __init__ and __del__ methods instead of setup_class and teardown_class?
According to Fixture finalization / executing teardown code, the current best practice for setup and teardown is to use yield instead of return:
import pytest
#pytest.fixture()
def resource():
print("setup")
yield "resource"
print("teardown")
class TestResource:
def test_that_depends_on_resource(self, resource):
print("testing {}".format(resource))
Running it results in
$ py.test --capture=no pytest_yield.py
=== test session starts ===
platform darwin -- Python 2.7.10, pytest-3.0.2, py-1.4.31, pluggy-0.3.1
collected 1 items
pytest_yield.py setup
testing resource
.teardown
=== 1 passed in 0.01 seconds ===
Another way to write teardown code is by accepting a request-context object into your fixture function and calling its request.addfinalizer method with a function that performs the teardown one or multiple times:
import pytest
#pytest.fixture()
def resource(request):
print("setup")
def teardown():
print("teardown")
request.addfinalizer(teardown)
return "resource"
class TestResource:
def test_that_depends_on_resource(self, resource):
print("testing {}".format(resource))
When you write "tests defined as class methods", do you really mean class methods (methods which receive its class as first parameter) or just regular methods (methods which receive an instance as first parameter)?
Since your example uses self for the test methods I'm assuming the latter, so you just need to use setup_method instead:
class Test:
def setup_method(self, test_method):
# configure self.attribute
def teardown_method(self, test_method):
# tear down self.attribute
def test_buttons(self):
# use self.attribute for test
The test method instance is passed to setup_method and teardown_method, but can be ignored if your setup/teardown code doesn't need to know the testing context. More information can be found here.
I also recommend that you familiarize yourself with py.test's fixtures, as they are a more powerful concept.
This might help http://docs.pytest.org/en/latest/xunit_setup.html
In my test suite, I group my test cases into classes. For the setup and teardown I need for all the test cases in that class, I use the setup_class(cls) and teardown_class(cls) classmethods.
And for the setup and teardown I need for each of the test case, I use the setup_method(method) and teardown_method(methods)
Example:
lh = <got log handler from logger module>
class TestClass:
#classmethod
def setup_class(cls):
lh.info("starting class: {} execution".format(cls.__name__))
#classmethod
def teardown_class(cls):
lh.info("starting class: {} execution".format(cls.__name__))
def setup_method(self, method):
lh.info("starting execution of tc: {}".format(method.__name__))
def teardown_method(self, method):
lh.info("starting execution of tc: {}".format(method.__name__))
def test_tc1(self):
<tc_content>
assert
def test_tc2(self):
<tc_content>
assert
Now when I run my tests, when the TestClass execution is starting, it logs the details for when it is beginning execution, when it is ending execution and same for the methods..
You can add up other setup and teardown steps you might have in the respective locations.
Hope it helps!
As #Bruno suggested, using pytest fixtures is another solution that is accessible for both test classes or even just simple test functions. Here's an example testing python2.7 functions:
import pytest
#pytest.fixture(scope='function')
def some_resource(request):
stuff_i_setup = ["I setup"]
def some_teardown():
stuff_i_setup[0] += " ... but now I'm torn down..."
print stuff_i_setup[0]
request.addfinalizer(some_teardown)
return stuff_i_setup[0]
def test_1_that_needs_resource(some_resource):
print some_resource + "... and now I'm testing things..."
So, running test_1... produces:
I setup... and now I'm testing things...
I setup ... but now I'm torn down...
Notice that stuff_i_setup is referenced in the fixture, allowing that object to be setup and torn down for the test it's interacting with. You can imagine this could be useful for a persistent object, such as a hypothetical database or some connection, that must be cleared before each test runs to keep them isolated.
Your code should work just as you expect it to if you add #classmethod decorators.
#classmethod
def setup_class(cls):
"Runs once per class"
#classmethod
def teardown_class(cls):
"Runs at end of class"
See http://pythontesting.net/framework/pytest/pytest-xunit-style-fixtures/
import pytest
class Test:
#pytest.fixture()
def setUp(self):
print("setup")
yield "resource"
print("teardown")
def test_that_depends_on_resource(self, setUp):
print("testing {}".format(setUp))
In order to run:
pytest nam_of_the_module.py -v
I'm not sure I got the specifics of using Selenium in your original questions, but in case you were simply asking about how to use a more classical setUp/tearDown style, Pytest supports most unittest features, so you could do something like:
import unittest
class TestHello(unittest.TestCase):
def setUp(self):
print('running setUp')
def test_one(self):
print('running test_one')
def test_two(self):
print('running test_two')
def tearDown(self):
print('running tearDown')
Which produces:
$ pytest -s -v
====================== test session starts =======================
platform linux -- Python 3.8.2, pytest-6.2.4, py-1.10.0, pluggy-0.13.1 -- /gnu/store/nckjv3ccwdi6096j478gvns43ssbls2p-python-wrapper-3.8.2/bin/python
cachedir: .pytest_cache
hypothesis profile 'default' -> database=DirectoryBasedExampleDatabase('/tmp/test/.hypothesis/examples')
rootdir: /tmp/test
plugins: hypothesis-5.4.1
collected 2 items
test_hw.py::TestHello::test_one running setUp
running test_one
running tearDown
PASSED
test_hw.py::TestHello::test_two running setUp
running test_two
running tearDown
PASSED