This test session works properly :
from myapp.models import MyModel
#pytest.fixture(scope='function')
def mymodel():
return G(MyModel)
#pytest.mark.django_db
def test_mymodel_one(mymodel):
assert MyModel.objects.count() > 0
#pytest.mark.django_db
def test_mymodel_two(mymodel):
assert MyModel.objects.count() > 0
and produces this output:
========= test session starts =========
tests/myapp/test_pp.py::test_mymodel_one PASSED
tests/myapp/test_pp.py::test_mymodel_two PASSED
but if I change the scope of my fixture to 'session', test two fails:
========= test session starts =========
tests/myapp/test_pp.py::test_mymodel_one PASSED
tests/myapp/test_pp.py::test_mymodel_two FAILED
========= FAILURES ==============
_________ test_mymodel_two ________
tests/myapp/test_pp.py:85: in test_mymodel_two
assert MyModel.objects.count() > 0
E assert 0 > 0
The object created is correctly returned from fixture (I can access his values) but it isn't stored no more.
How can I use session scope and maintain the storage in my test db?
I tried to replicate the context in my tests package and I've found the same situation you exposed.
First of all, I want to share with you two page of the pytest documentation, where we could find the answer to this question.
In the documentation¹ the organization of the methods is a little bit different, in fact the method which is delegated to create the fixture is inside the conftest.py .
# content of conftest.py
#pytest.fixture(scope="session")
def smtp(...):
# the returned fixture value will be shared for
# all tests needing it
According to your tests settings, you can try moving the mymodel method inside the conftest module. I've tried to move my fixture generator inside the conftest file, but I found several traversal problems due to the required django_db mark, which might conflict with the session scope (I guess?).
I also found another help inside the examples page² of pytest, where is used the session scope widely for different python modules, where is pointed out the impossibility to an internal module test to access to the same session scope defined at the same level of the parents' one.
# content of a/conftest.py
import pytest
class DB:
pass
#pytest.fixture(scope="session")
def db():
return DB()
# content of a/test_db.py
def test_a1(db):
assert 0, db # to show value
# content of a/test_db2.py
def test_a2(db):
assert 0, db # to show value
if
# content of b/test_error.py
def test_root(db): # no db here, will error out
pass
will fail the test, because
The two test modules in the a directory see the same db fixture instance while the one test in the sister-directory b doesn’t see it. We could of course also define a db fixture in that sister directory’s conftest.py file. Note that each fixture is only instantiated if there is a test actually needing it (unless you use “autouse” fixture which are always executed ahead of the first test executing).
In the second example I've noticed that the method for generating the fixture is instantiated for each test and even though the scope is set to session or module, it works like function scope.
Which version of pytest are you using?
Could try moving from the current module to the conftest module your mymodel method?
Related
So, I have fixtures defined in conftest.py file with scope="class" as I want to run them before each test class is invoked. The conftest file is placed inside project root directory for it to be visible to every test module.
Now in one of the test modules, I have another setup function which I want to run once for that module only. But the problem is setup_class() method is called before running fixtures defined in conftest.py. Is this expected? I wanted it to be opposite because I want to use something done in the fixtures defined in conftest. How to do that?
Code -
conftest.py:
#pytest.fixture(scope="class")
def fixture1(request):
#set a
#pytest.fixture(scope="class")
def fixture1(request):
test_1.py:
#pytest.mark.usefixtures("fixture_1", "fixture_2")
class Test1():
#need this to run AFTER the fixture_1 & fixture_2
def setup_class():
#setup
#get a set in fixture_1
def test_1()
.....
I know that I could simply define a fixture in the test file instead of setup_class but then I will have to specify it in arguments of every test method in order it to be invoked by pytest. But suggestions are welcome!
I have exactly the same problem. Only now I have realized that the problem might be taht the setup_class is called before the fixture >-/
I think that this question is similar to this one
Pytest - How to pass an argument to setup_class?
And the problem is mixing the unittest and pytest methods.
I kind of did what they suggested - I ommitted the setup_class and created a new fixture within the particular test file,
calling the fixture in the conftest.py.
It works so far.
M.
The problem is that you can use the result of a fixture only in test function (or method) which is run by pytest. Here I can suggest a workaround. But of course I'm not sure if it suites your needs.
The workaround is to call the function from a test method:
conftest.py
#pytest.fixture(scope='class')
def fixture1():
yield 'MYTEXT'
test_1.py
class Test1:
def setup_class(self, some_var):
print(some_var)
def test_one(self, fixture1):
self.setup_class(fixture1)
Fixtures and setup_class are two different paradigms to initialize test functions (and classes). In this case, mixing the two creates a problem: The class-scoped fixtures run when the individual test functions (methods) run. On the other hand, setup_class runs before they do. Hence, it is not possible to access a fixture value (or fixture-modified state) from setup_class.
One of the solutions is to stop using setup_class entirely and stick with a fixtures-only solution which is the preferred way in pytest nowadays (see the note at the beginning).
# conftest.py or the test file:
#pytest.fixture(scope="class")
def fixture_1(request):
print('fixture_1')
# the test file:
class Test1():
#pytest.fixture(scope="class", autouse=True)
def setup(self, fixture_1, request):
print('Test1.setup')
def test_a(self):
print('Test1.test_a')
def test_b(self):
print('Test1.test_b')
Note that the setup fixture depends on fixture_1 and hence can access it.
Our current project has a project tree with a conftest.py at the root level of the tests like below:
Tests
test_folder_1
test_folder_2
conftest.py
This root level conftest.py imports most of our fixtures at session and class levels. But in test_folder_1, we have another conftest.py that we want to override some of those class level fixtures and turn them function-scoped.
I was under the impression that this would work and has been working, but now I'm getting errors with some of these: You tried to access the 'function' scoped fixture 'new_primary_contact' with a 'class' scoped request object
If this is not how to override fixtures from conftest.py, is there another way? It could be another error, but I really want to know if conftest.py's further in the tree actually override high level ones. Thanks for any help!
EDIT:
Here is an minimal example of my code:
Root level conftest.py:
from automation.Fixtures.ClassScope import config, api_client, new_related_contact, \
new_primary_contact, auto_clean_contacts_helper as contacts_helper, create_contact_payload, document_helper
In my lower level test, in root/feature_1 I have something like this in a conftest.py:
from automation.Fixtures.FunctionScope import new_related_contact, \
new_primary_contact, auto_clean_contacts_helper as contacts_helper, create_contact_payload, document_helper
I was just wanting to override certain fixtures, so my function level tests would not use class scope.
Here is a test example using them in root/feature_1/Tests:
class TestDeleteRelatedContactWorkflow:
def test__remove_related_contact(self, authenticated_chrome_driver, config, new_primary_contact, new_related_contact, new_contact_proposal, contacts_helper):
primary_contact_name = new_primary_contact["display_name"]
related_contact_name = new_related_contact["display_name"]
related_contact_id = new_related_contact["ContactId"]
document_id = new_contact_document["id"]
web_nav = WebNavigation(authenticated_chrome_driver, config)
web_nav.navigate_to_document_view_page(document_id)
...
response = contacts_helper.delete_contact(related_contact_id)
assert response["IsSuccess"] is True, f'Deletion of contact "{related_contact_name}" not successful.'
assert response["StatusCode"] == 200, 'Status code should == 200 when deleting contact.'
The test fails before it logs in and after the webdriver/browser initiates (names changed to protect code)
test setup failed
ScopeMismatch: You tried to access the 'function' scoped fixture 'new_primary_contact' with a 'class' scoped request object, involved factories
root/Fixtures/ClassScope/class_scoped_document_fixtures.py:19: def new_document_payload(config, new_primary_contact)
root/Fixtures/FunctionScope/function_scoped_contacts_fixtures.py:85: def new_primary_contact(contacts_helper, create_contact_payload)
I would add conftest.py to ROOT folder, only if I was 100% sure all my tests are using all imported fixtures there, otherwise you are just slowing down automation execution which is critical.
I have tested the following:
Tests
conftest.py
test_folder_1
conftest.py
test_folder_2
Both conftest.py files had fixture function with the same name.
in test_folder_1/conftest.py - I have set scope="function" and returned different value compared to the same fixture in Tests/conftest.py (which had scope="class").
In a test under test_folder_1,
I was able to use the updated fixture from test_folder_1/conftest.py.
I am not able to apply this fixture on a class, as the scope changed.
Basically, I'm trying to do a test for each iteration of a list of routes to check that the web pages associated with a particular function return a valid status code.
I want something along the lines of this:
import pytest
from flask import url_for
from myflaskapp import get_app
#pytest.yield_fixture
def app():
# App context and specific overrides for test env
yield get_app()
#pytest.yield_fixture
def client(app):
yield app.test_client()
#pytest.yield_fixture
def routes(app):
routes = [
'foo',
'bar',
# There's quite a lot of function names here
]
with app.app_context():
for n, route in enumerate(routes):
routes[n] = url_for(route)
# yield url_for(route) #NOTE: This would be ideal, but not allowed.
# convert the routes from func names to actual routes
yield routes
#pytest.mark.parametrize('route', routes)
def test_page_load(client, route):
assert client.get(route.endpoint).status_code == 200
I read up that you can't mix parametrize with a fixture as an argument due to something along the lines of interpretation/load/execution order, although, how is this solved in terms of 'best practice'?
I saw a solution where you can generate tests from a function directly, and that seems extremely flexible and might be along the lines of what I want Passing pytest fixture in parametrize (Although I can't use call a fixture decorated function directly, so probably not)
Although, I'm new to pytest and I'd love to see more examples of how to generate tests or perform multiple tests in an iteration with little-to-no restrictions while adhering to proper pytest styling and the DRY principle. (I know about conftest.py)
I'd prioritize versatility/practicality over proper styling if that matters. (within reason, maintainability is a high priority too)
I want to be able to reference the solution to this problem to help guide how I tackle structuring my tests in the future, but I seem to keep hitting roadblocks/limitations or being told by pytest I can't do X solution the way I would expect/want too.
Relevant Posts:
DRY: pytest: parameterize fixtures in a DRY way
Generate tests from a function: Passing pytest fixture in parametrize
Very simply solution (doesn't apply to this case): Parametrize pytest fixture
Flask app context in PyTest: Testing code that requires a Flask app or request context
Avoiding edge-cases with multiple list fixtures: Why does Pytest perform a nested loop over fixture parameters
Pytest fixtures themselves can be parameterized, though not with pytest.mark.parametrize. (It looks like this type of question was also answered here.) So:
import pytest
from flask import url_for
from myflaskapp import get_app
#pytest.fixture
def app():
app = get_app()
# app context stuff trimmed out here
return app
#pytest.fixture
def client(app):
client = app.test_client()
return client
#pytest.fixture(params=[
'foo',
'bar'
])
def route(request, app):
'''GET method urls that we want to perform mass testing on'''
with app.app_context():
return url_for(request.param)
def test_page_load(client, route):
assert client.get(route.endpoint).status_code == 200
The documentation explains it this way:
Fixture functions can be parametrized in which case they will be called multiple times, each time executing the set of dependent tests, i. e. the tests that depend on this fixture. Test functions usually do not need to be aware of their re-running. Fixture parametrization helps to write exhaustive functional tests for components which themselves can be configured in multiple ways.
Extending the previous example, we can flag the fixture to create two smtp_connection fixture instances which will cause all tests using the fixture to run twice. The fixture function gets access to each parameter through the special request object:
# content of conftest.py
import pytest import smtplib
#pytest.fixture(scope="module", params=["smtp.gmail.com", "mail.python.org"])
def smtp_connection(request):
smtp_connection = smtplib.SMTP(request.param, 587, timeout=5)
yield smtp_connection
print("finalizing {}".format(smtp_connection))
smtp_connection.close()
The main change is the declaration of params with #pytest.fixture, a list of values for each of which the fixture function will execute and can access a value via request.param. No test function code needs to change.
My current solution that I fumbled my way across is this:
import pytest
from flask import url_for
from myflaskapp import get_app
#pytest.fixture
def app():
app = get_app()
# app context stuff trimmed out here
return app
#pytest.fixture
def client(app):
client = app.test_client()
return client
def routes(app):
'''GET method urls that we want to perform mass testing on'''
routes = ['foo', 'bar']
with app.app_context():
for n, route in enumerate(routes):
routes[n] = url_for(route)
return routes
#pytest.mark.parametrize('route', routes(get_app()))
#NOTE: It'd be really nice if I could use routes as a
# fixture and pytest would handle this for me. I feel like I'm
# breaking the rules here doing it this way. (But I don't think I actually am)
def test_page_load(client, route):
assert client.get(route.endpoint).status_code == 200
My biggest issue with this solution is that, I can't call the fixture directly as a function, and this solution requires either that, or doing all the work my fixture does outside of the fixture, which is not ideal. I want to be able to reference this solution to tackle how I structure my tests in the future.
FOR ANYONE LOOKING TO COPY MY SOLUTION FOR FLASK SPECIFICALLY:
My current solution might be worse for some people than it is for me, I use a singleton structure for my get_app() so it should be fine if get_app() is called many times in my case, because it will call create_app() and store the app itself as a global variable if the global variable isn't already defined, basically emulating the behavior of only calling create_app() once.
I am writing a test framework using pytest. Is there a way to get testcase object in classes other than testcase. For example utility classes.
I want to print the testcase name and some markers for the test in utility classes. Are these information available in some contextmanager?
You cannot directly access pytest test properties if you are not inside a test fixture or a hook function, as there is no fixed test case class as in unittest. Your best bet is probably to get this information in a fixture and store it globally for access from a utility function:
testinfo={}
#pytest.fixture(autouse=True)
def test_info(request):
global testinfo
testinfo['name'] = request.node.name
testinfo['markers'] = [m.name for m in request.node.iter_markers()]
...
yield # the information is stored at test start...
testinfo = {} # ... and removed on test teardown
def utility_func():
if testinfo:
print(f"Name: {testinfo['name']} Markers: {testinfo['markers']}")
...
Or, the same if you use a test class:
class TestSomething:
def setup_method(self):
self.testinfo = {}
#pytest.fixture(autouse=True)
def info(self, request):
self.testinfo['name'] = request.node.name
self.testinfo['markers'] = [m.name for m in
request.node.iter_markers()]
yield # the information is stored at test start...
self.testinfo = {} # ... and removed on test teardown
def utility_func(self):
if self.testinfo:
print(f"Name: {self.testinfo['name']} Markers:"
f" {self.testinfo['markers']}")
#pytest.mark.test_marker
def test_something(self):
self.utility_func()
assert True
This will show the output:
Name: test_something Markers: ['test_marker']
This will work if you call the utility function during test execution - otherwise no value will be set.
Note however that this will only work reliably if you execute the test synchronously. If using pytest-xdist or similar tools for asynchronous test execution, this may not work due to the testinfo variable being overwitten by another test (though that depends on the implementation - it may work, if the variables are copied during a test run). In that case you can to do the logging directly in the fixture or hook function (which may generally be a better idea, depending on your use case).
For more information about available test node properties you can check the documentation of a request node.
I am testing the User message function of a web solution using pytest + selenium. The tests will generate a test message to a test user, and then log in that user to verify that the message indeed is displaying for that user.
I need to generate those messages through an internal API.
In order to be able to access this API, I first have to generate an AUTH token through a different API.
So the test scenario is basically:
At test startup, generate a new AUTH token through an API helper function.
Send a request to another API to set a new message (requires the AUTH token)
Send a request to yet another API to "map" this message to a specified user (requires the AUTH token)
Log in test user and verify that the new message is indeed displaying.
My problem is that I want to avoid creating a new AUTH token every time every test within my test class is run - I want to create a new token once that all tests use within the same test run.
What is the smartest solution to generate one new access token when invoking all tests?
Right now I have come up with something like this, which will generate a new token every time any individual test is run:
import pytest
import helpers.api_access_token_helper as token_helper
import helpers.user_message_generator_api_helper as message_helper
import helpers.login_helper as login_helper
import helpers.popup_helper as popup_helper
class TestStuff(object):
#pytest.yield_fixture(autouse=True)
def run_around_tests(self):
yield token_helper.get_api_access_token()
def test_one(self, run_around_tests):
auth_token = run_around_tests
message_helper.create_new_message(auth_token, some_message_data)
message_helper.map_message_to_user(auth_token, user_one["user_id"])
login_helper.log_in_user(user_one["user_name"], user_one["user_password"])
assert popup_helper.user_message_is_displaying(some_message_data["title"])
def test_two(self, run_around_tests):
auth_token = run_around_tests
message_helper.create_new_message(auth_token, some_other_message_data)
message_helper.map_message_to_user(auth_token, user_two["user_id"])
login_helper.log_in_user(user_two["user_name"], user_two["user_password"])
assert popup_helper.user_message_is_displaying(some_other_message_data["title"])
I have laborated back and forth a bit with the "run-around-tests" fixture but havent been able to find a solution.
You have to adapt the fixture scope to cache its results for all tests in test run (scope='session'), all tests in module (scope='module'), all tests in class (old unittest-style tests only, scope='class'), or for a single test (scope='function'; this the default one). Examples:
fixture function
#pytest.fixture(scope='session')
def token():
return token_helper.get_api_access_token()
class Tests(object):
def test_one(self, token):
...
def test_two(self, token):
...
class OtherTests(object):
def test_one(self, token):
...
The token will be calculated once when first requested and kept in cache throughout the whole test run, so all three tests Tests::test_one, Tests::test_two and OtherTests::test_one will share the same token value.
fixture class method
If you intend to write old-style test classes instead of test functions and want the fixture to be a class method (like it is in your code), note that you can only use the class scope, so that the fixture value is shared only between the tests in the class:
class TestStuff(object):
#pytest.fixture(scope='class')
def token(self):
return token_helper.get_api_access_token()
def test_one(self, token):
...
def test_two(self, token):
...
Stuff aside:
pytest.yield_fixture is deprecated and replaced by pytest.fixture;
you don't need to set autouse=True because you explicitly request the fixture in the test parameters. It will be called anyway.
You can add a scope="module" parameter to the #pytest.fixture.
According to pytest documentation:
Fixtures requiring network access depend on connectivity and are
usually time-expensive to create. Extending the previous example, we
can add a scope="module" parameter to the #pytest.fixture invocation
to cause a smtp_connection fixture function, responsible to create a
connection to a preexisting SMTP server, to only be invoked once per
test module (the default is to invoke once per test function).
Multiple test functions in a test module will thus each receive the
same smtp_connection fixture instance, thus saving time. Possible
values for scope are: function, class, module, package or session.
# content of conftest.py
import pytest
import smtplib
#pytest.fixture(scope="module")
def smtp_connection():
return smtplib.SMTP("smtp.gmail.com", 587, timeout=5)
Fixture scopes
Fixtures are created when first requested by a test, and are destroyed based on their scope:*
function: the default scope, the fixture is destroyed at the end of the test.
class: the fixture is destroyed during teardown of the last test in the class.
module: the fixture is destroyed during teardown of the last test in the module.
package: the fixture is destroyed during teardown of the last test in the package.
session: the fixture is destroyed at the end of the test session.
Note: Pytest only caches one instance of a fixture at a time, which means that when using a parametrized fixture, pytest may invoke a fixture more than once in the given scope.