I have a file celery_test.py which should load the configuration settings for testing(config.test) in Django but instead, it loads configuration of Development(config.development)
from __future__ import absolute_import
import os
from django.conf import settings
from celery import Celery
os.environ.setdefault(
"DJANGO_SETTINGS_MODULE", "config.testing"
)
test_app = Celery('test')
test_app.config_from_object('django.conf:settings')
test_app.autodiscover_tasks()
print("Testing celery")
print(settings)
#test_app.task(bind=True)
def debug_task(self):
print('Request: {0!r}'.format(self.request))
When I am printing settings it printing config.development instead of config.testing. Can anyone help me here how to load config.testing?
Seeing that you use django I would suggest not actually overriding the settings file like this. Instead use the django.test.override_settings decorator to override the specific settings you want to change. This way you're always testing the most current settings file, so you don't have to keep both up-to-date.
More information is also in the documentation: https://docs.djangoproject.com/en/3.0/topics/testing/tools/#django.test.override_settings.
I can't really provide an example, since you don't provide any unittest example. But I hope the linked documentation is clear enough.
Assuming your are using pytest, you could put this in your pytest.ini file:
[pytest]
DJANGO_SETTINGS_MODULE = celery_test.py
Or use:
DJANGO_SETTINGS_MODULE=celery_test.py pytest
for a one-liner.
Related
I am following the testdriven.io Test-Driven Development with FastAPI and Docker tutorial and I am stuck on the Pytest setup step. I have checked over an over again to see what I am missing, and keep coming up short.
The code sample from the tutorial shows that, in conftest.py, you are to ahve the following from statement:
from app import main
from app.config import get_settings, Settings
For starters, Pycharm is telling me that it is unable to import anything from above.
My Folder Structure:
main.py:
import os
from fastapi import FastAPI, Depends
from tortoise.contrib.fastapi import register_tortoise
from .config import get_settings, Settings
app = FastAPI()
register_tortoise(
app,
db_url=os.environ.get("DATABASE_URL"),
modules={"models": ["app.models.tortoise"]},
generate_schemas=False,
add_exception_handlers=True,
)
#app.get("/ping")
async def pong(settings: Settings = Depends(get_settings)):
return {"ping": "pong", "environment": settings.environment, "testing": settings.testing}
conftest.py
import os
import pytest
from starlette.testclient import TestClient
from app import main
from app.config import get_settings, Settings
def get_settings_override():
return Settings(testing=1, database_url=os.environ.get("DATABASE_TEST_URL"))
#pytest.fixture(scope="module")
def test_app():
# set up
main.app.dependency_overrides[get_settings] = get_settings_override
with TestClient(main.app) as test_client:
# testing
yield test_client
# tear down
The tutorial has you run the tests using docker-compose exec web python -m pytest
This is the output I get when running the tests:
Any help would be appreciated. I feel like this is entry level stuff that is causing an extreme headache.
Thanks to #MatsLindh for the help. As he mentioned in his comments above, the tutorial has you running pytest on the entire project instead of just the tests folder. Running directly on tests solved my issue with pytest failing. He also gave good advice on getting imports to work correctly in an IDE by suggesting to look at the pytest documentation for further integration steps.
I have a config.ini file which contains some properties but I want to read the environment variables inside the config file.
[section1]
prop1:(from envrinment variable) or value1
Is this possible or do I have to write a method to take care of that?
I know this is late to the party, but someone might find this handy.
What you are asking after is value interpolation combined with os.environ.
Pyramid?
The logic of doing so is unusual and generally this happens if one has a pyramid server, (which is read by a config file out of the box) while wanting to imitate a django one (which is always set up to read os.environ).
If you are using pyramid, then pyramid.paster.get_app has the argument options: if you pass os.environ as the dictionary you can use %(variable)s in the ini. Not that this is not specific to pyramid.paster.get_app as the next section shows (but my guess is get_app is the case).
app.py
from pyramid.paster import get_app
from waitress import serve
import os
app = get_app('production.ini', 'main', options=os.environ)
serve(app, host='0.0.0.0', port=8000, threads=50)
production.ini:
[app:main]
sqlalchemy.url = %(SQL_URL)s
...
Configparse?
The above is using configparser basic interpolation.
Say I have a file called config.ini with the line
[System]
conda_location = %(CONDA_PYTHON_EXE)
The following will work...
import configparser, os
cfg = configparser.ConfigParser()
cfg.read('config.ini')
print(cfg.get('System', 'conda_location', vars=os.environ))
I think :thinking_face, use .env and in config.py from dotenv import load_dotenv() and in next line load_dotenv() and it will load envs from .env file
To run my Django's tests, I was using the manage.py file.
But I decided to create my own file runtests.py.
As specified in the Django's doc, the manage.py file will execute all methods whose name is starting like "test" and whose class inherits TestCase !
My aim is to use this searching method to display all possible tests (not running them).
So do you know where I could get this "searching method" ?
Thank you !
What you're looking for is called "test discovery", and you don't need to do it manually.
If you want to write a standalone script that runs your tests, you can simply invoke Django's test runner manually from Python code after configuring. The process for this is:
# First you need settings.
# You can do this by setting DJANGO_SETTINGS_MODULE:
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'path.to.your.settings')
# Or you can do it manually:
SETTINGS_DICT = {
# Put all the settings you'll need in here...
}
from django.conf import settings
settings.configure(**SETTINGS_DICT)
import django
django.setup()
import sys
# Now you can run tests:
from django.test.utils import get_runner
TestRunner = get_runner(settings)
test_runner = TestRunner(verbosity=2, interactive=True)
failures = test_runner.run_tests(
['test_module_to_look_in']
)
sys.exit(bool(failures))
I have this python file tasks.py
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'DjangoWebProject.settings')
from django.contrib.auth.models import User
from logreg.models import ActivationCode
import datetime
def remove_users():
print 'hello worldddddddddddddddddddddddddddddddddd'
inactive_users = []
activation_codes = ActivationCode.objects.all()
for activation_code in activation_codes:
if datetime.datetime.date(activation_code.key_expires) < datetime.datetime.date(datetime.datetime.now()):
inactive_users.append(activation_code.user_id)
for inactive_user in inactive_users:
User.objects.filter(id=inactive_user).delete()
But this is in the root folder and when i try to execute it, it gives me the following error
File
"C:\Users\deybala1\AppData\Local\Continuum\Anaconda2\lib\site-packages\dj
ango\apps\registry.py", line 124, in check_apps_ready
raise AppRegistryNotReady("Apps aren't loaded yet.") django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
How do i fix this?
If you're creating any script that is using your django project, it is absolutely necessary to set path to settings of your project before any import from django or your project. And you're importing user model from django in 1st line and model from your project in second.
Also, you will need to call django.setup() first.
To fix that, move import os and setting path to django settings to the very beginning of your script, and put django.setup() just after that (with proper import), like this:
# first, set path to project settings
import os
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'DjangoWebProject.settings')
import django
django.setup()
# now you can import anything else
from django.contrib.auth.models import User
from logreg.models import ActivationCode
import datetime
def remove_users():
print 'hello worldddddddddddddddddddddddddddddddddd'
inactive_users = []
activation_codes = ActivationCode.objects.all()
for activation_code in activation_codes:
if datetime.datetime.date(activation_code.key_expires) < datetime.datetime.date(datetime.datetime.now()):
inactive_users.append(activation_code.user_id)
for inactive_user in inactive_users:
User.objects.filter(id=inactive_user).delete()
Note that you're trying to add a settings module inside a script that already requires it.
Wouldn't it be easier if you add a specific django command? Thanks to it you'd be able to start your task with python manage.py --settings=<path_to_your_settings>.
Another tip:
Move every django import statement below
os.environ.setdefault('DJANGO_SETTINGS_MODULE', 'DjangoWebProject.settings')
I have this folder structure for django
settings/dev.py
settings/prod.py
settings/test.py
Then i have common settings in settings/common.py in which i check the ENV variable like
if PROD:
from settings.prod import *
Based on ENV variable each one of them will be active
I want to use something like this in my code
from myapp import settings
rather than
from myapp.settings import dev
This is the method which I follow. Learnt this from the book Two Scoops of Django.
Have a file, such as, settings/common.py which will contain the properties/settings which are common in dev, prod and test environment. (You already have this.)
The other 3 files should:
Import the common settings from the settings/common.py by adding the line from .common import *
And should contain settings for its own corresponding environment.
The manage.py file decides which settings file to import depending on the OS environment variable, DJANGO_SETTINGS_MODULE. So, for test environment the value of DJANGO_SETTINGS_MODULE should be mysite.settings.test
Links for reference:
Django documentation for django-admin utility - Link
Two Scoops of Django sample project - Link
Preserve your settings folder structure and create __init__.py there.
Please, use code below in your settings/__init__.py:
import os
# DJANGO_SERVER_TYPE
# if 1: Production Server
# else if 2: Test Server
# else: Development Server
server_type = os.getenv('DJANGO_SERVER_TYPE')
if server_type==1:
from prod import *
elif server_type==2:
from test import *
else:
from dev import *
Now you can set environment variable called DJANGO_SERVER_TYPE to choose between Production, Test or Development Server and import settings using:
import settings