I need to perform some fairly simple tasks after my Django environment has been "fully loaded".
More specifically I need to do things like Signal.disconnect() some Django Signals that are setup by my third party library by default and connect my own Signals and I need to do some "monkey patching" to add convenience functions to some Django models from another library.
I've been doing this stuff in my Django app's __init__.py file, which seems to work fine for the monkey patching, but doesn't work for my Signal disconnecting. The problem appears to be one of timing--for whatever reason the Third Party Library always seems to call its Signal.connect() after I try to Signal.disconnect() it.
So two questions:
Do I have any guarantee based on the order of my INSTALLED_APPS the order of when my app's __init__.py module is loaded?
Is there a proper place to put logic that needs to run after Django apps have been fully loaded into memory?
In Django 1.7 Apps can implement the ready() method: https://docs.djangoproject.com/en/dev/ref/applications/#django.apps.AppConfig.ready
My question is a more poorly phrased duplicate of this question: Where To Put Django Startup Code. The answer comes from that question:
Write middleware that does this in init and afterwards raise django.core.exceptions.MiddlewareNotUsed from the init, django will remove it for all requests...
See the Django documentation on writing your own middleware.
I had to do the following monkey patching. I use django 1.5 from github branch. I don't know if that's the proper way to do it, but it works for me.
I couldn't use middleware, because i also wanted the manage.py scripts to be affected.
anyway, here's this rather simple patch:
import django
from django.db.models.loading import AppCache
django_apps_loaded = django.dispatch.Signal()
def populate_with_signal(cls):
ret = cls._populate_orig()
if cls.app_cache_ready():
if not hasattr(cls, '__signal_sent'):
cls.__signal_sent = True
django_apps_loaded.send(sender=None)
return ret
if not hasattr(AppCache, '_populate_orig'):
AppCache._populate_orig = AppCache._populate
AppCache._populate = populate_with_signal
and then you could use this signal like any other:
def django_apps_loaded_receiver(sender, *args, **kwargs):
# put your code here.
django_apps_loaded.connect(django_apps_loaded_receiver)
As far as I know there's no such thing as "fully loaded". Plenty of Django functions include import something right in the function. Those imports will only happen if you actually invoke that function. The only way to do what you want would be to explicitly import the things you want to patch (which you should be able to do anywhere) and then patch them. Thereafter any other imports will re-use them.
Related
Question - What approach or design pattern would make it easy in Django to use stubs for external integrations when running tests? With 'external integrations' I mean a couple of external REST APIs, and NAS file system. The external integrations are already separate modules/classes.
What I do now -
Currently, I disable external dependencies in tests mainly by sprinkling mock.path() statements across my test code.
But this is getting unpractical (needs to be applied per test; also easy to forget especially in more high-level tests), and links too much to the internals of certain modules.
Some details of what I am looking for
I like the concept of 'nullable infrastructure' described at
https://www.jamesshore.com/v2/blog/2018/testing-without-mocks#nullable-infrastructure.
I am especially looking for an approach that integrates well with Django, i.e. considering the settings.py file approach, and running tests via python manage.py test.
I would like to be able to easily:
state that all tests should use the nullable counterpart of an infrastructure class or function
override that behaviour per test, or test class, when needed (e.g. when testing the actual external infrastructure).
I tried the approach outlined in https://medium.com/analytics-vidhya/mocking-external-apis-in-django-4a2b1c9e3025, which basically says to create an interface implementation, a real implementation and a stub implementation. The switching is done using a django setting parameter and a class decorator on the interface class (which returns the chosen class, rather than the interface). But it isn't working out very well: the class decorator in my setup does not work with #override_settings (the decorator applies the settings upon starting of django, not when running the test), and there is really a lot of extra code (which also feels un-pythonic).
I'm using Eclipse with plugged autopep8 and I found it it very helpful. It's saving a lot of my time from fixing code style by hands. But for some coding patterns I don't know how to avoid pep8 rules I don't want to use. For example using Django (1.5.4) I need to connect signals of installed application. I always use import signals at the end of models.py file. But pep8 doesn't allow to use imports at end of file. # noqa comment doesn't helps. I can not put import signals to the top of the models.py file, because in signals I use some models still not defined on that moment.
What can you suggest in this situation? May be there is more appropriate way to connect signals?
Firstly, everything in PEP8 is a recommendation, not a hard-and-fast rule. If your code needs a certain structure, you should feel free to ignore the recommendation.
That said, importing signals at the end of your models file feels a bit strange. Rather, import both models and signals from a separate file that is itself imported at startup. The app's __init__.py file might be a good candidate, or you can use the new AppConfig functionality in 1.7.
If I read the django documentation, only the documents about the template tags mention the potential danger of thread safety.
However, I'm curious what kind of things I have to do / avoid to write thread-safe code in Django...
One example is, I have the following function to config the loggers used in django.
_LOGGER_CONFIGURED = False
def config_logger():
global _LOGGER_CONFIGURED
if _LOGGER_CONFIGURED: return
_LOGGER_CONFIGURED = True
rootlogger = logging.getLogger('')
stderr_handler = StreamHandler(sys.stderr)
rootlogger.addHandler(stderr_handler)
and at the end of my root urlconf, i have the following function call:
config_logger()
The question is:
Is this code threadsafe?
What kind of variables are shared between the django threads?
There's not really a whole lot you can do about django templates and their issues with threading, besides not using them, or at least not using the tags that are sensitive to threading issues. There aren't many django template tags that do have issues, only the stateful ones do, such as cycle.
In the example you have given, you are not doing anything about thread safety, and you shouldn't be anyway: the logging module is already perfectly thread safe, so long as you use it in the normal way, which is to call logging.getLogger in the modules that need it, and LOGGING or LOGGING_CONFIG is set appropriately in your settings.py. No need to be clever with this.
other things you might be concerned about are database integrity in the face of concurrent updates. Don't be, if you are using PostgreSQL or MySQL/INNOdb databases, then you are completely protected from concurrency shennanegans.
I have a main page in Python/Pylons project, which have multiple different blocks (e.g. news/demo/(registration|private zone)/...).
My thought is that each block should be generated in a separate controller.
How can I call another controller method in a main page controller?
What you want to do is HMVC. I'm not sure it is easily doable out of the box with Pylons, since it's MVC.
If you have code that is repeated in multiple controllers, you could move some of this code out of the controller (in the models, or another module).
Also, if you are using Mako templates, you can reuse parts of templates by using inheritance http://www.makotemplates.org/docs/inheritance.html and by using defs http://www.makotemplates.org/docs/defs.html.
This is probably where you start moving chunks of code to library functions, to the /lib part of your Pylons project. "Generated by a separate controller" is probably going too far - you merely need to not repeat yourself. Try using library functions to make sure that the correct data is available, then use Mako's inheritance and namespace features.
I've got some code that runs on every (nearly) every admin request but doesn't have access to the 'request' object.
I need to find the path to Django installation. I could do:
import django
django_path = django.__file__
but that seems rather wasteful in the middle of a request.
Does putting the import at the start of the module waste memory? I'm fairly sure I'm missing an obvious trick here.
So long as Django has already been imported in the Python process (which it has, if your code is, for example, in a view function), importing it again won't do "anything"* — so go nuts, use import django; django.__file__.
Now, if Django hasn't been imported by the current Python process (eg, you're calling os.system("myscript.py") and myscript.py needs to determine Django's path), then import django will be a bit wasteful. But spawning a new process on each request is also fairly wasteful… So if efficiency is important, it might be better import myscript anyway.
*: actually it will set a value in a dictionary… But that's "nothing".