Develop Reference

Is it possible to get django info without running it

I have a django model, the whole code is completed. but I want to access my model info. a code like this to get field names.
for f in myModel._meta.fields:
print(f.get_attname())
is it possible to do it from an external python script without running django server?
other possible automated ways of doing this and saving results to a file are also appreciated.
try1
because Im using docker I ran it up. and from django container I started python shell
>>> from django.conf import settings
>>> settings.configure()
>>> import models
it gave django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
try2
by #Klaus D advice in comments I tried management command. so I created
users/
__init__.py
models.py
management/
__init__.py
commands/
__init__.py
_private.py
modelInfo.py
structure. in modelInfo.py I did
from django.core.management.base import BaseCommand, CommandError
from users import views2
def savelisttxtfile(the_list, path_, type_='w', encoding="utf-8"):
with open(path_, type_, encoding=encoding) as file_handler:
for item in the_list:
file_handler.write("{}\n".format(item))
class Command(BaseCommand):
def handle(self, *args, **options):
dic=[]
for f in views2.ChertModel._meta.fields:
print(f.get_attname())
dic.append(f.get_attname())
savelisttxtfile(dic,"F:\projects\sd.txt")
and from another python file I tried
os.chdir(r'F:\projects\users\management\commands')
from subprocess import run
import sys
run([sys.executable, r'F:\projects\users\management\commands\modelInfo.py'])
and it returned
CompletedProcess(args=['C:\\ProgramData\\Anaconda3\\python.exe', 'F:\projects\users\management\commands\modelInfo.py'], returncode=1)
and the results were not save in sd.txt

thanks to #klaus D and management command documentation I made this structure
users/
__init__.py
models.py
management/
__init__.py
commands/
__init__.py
_private.py
modelInfo.py
and in modelInfo.py I did
from django.core.management.base import BaseCommand, CommandError
from users import views2
def savelisttxtfile(the_list, path_, type_='w', encoding="utf-8"):
with open(path_, type_, encoding=encoding) as file_handler:
for item in the_list:
file_handler.write("{}\n".format(item))
class Command(BaseCommand):
def handle(self, *args, **options):
dic=[]
for f in views2.ChertModel._meta.fields:
print(f.get_attname())
dic.append(f.get_attname())
savelisttxtfile(dic,"F:\projects\sd.txt")
and to run it I went to manage.py location and executed python manage.py modelInfo to launch it.

Regarding your "try1" it seems to be a little bit trickier to start a python shell like python manage.py shell than what you propose there.
Fortunately you can do this:
python manage.py shell < your_script.py
and your script will be executed as if typed directly into the "django shell". Keep in mind that you still need to import your models relative to your project, i.e. from myapp.models import mymodel.

Automatically create django model instances at startup with empty database

My django project requires some model instances to be created at startup if they do not exist.
I currently create the required model instances I need in an app config.
class MyAppConfig(AppConfig):
name = 'my_app'
def ready(self):
create_required_objects()
def create_required_objects():
from my_app.models import MyObject
for name in MyObject.reserved_names:
if not MyObject.objects.filter(name=name).exists():
MyObject.objects.create(name=name, not_editable=True)
This works perfectly when the sqlite database is initialized, however if I clear the database and then try to run the sever, I get the following error:
django.db.utils.OperationalError: no such table: my_app_object
I would like to be able to clear the database (preferably by just removing db.sqlite3) and run the server.

Use post_migrate signal to create a new instance when migrating to the new database:
Like:
from django.db.models.signals import post_migrate
from my_app.models import MyObject
def create_required_objects(sender, **kwargs):
for name in MyObject.reserved_names:
if not MyObject.objects.filter(name=name).exists():
MyObject.objects.create(name=name, not_editable=True)
class MyAppConfig(AppConfig):
name = 'my_app'
def ready(self):
post_migrate.connect(create_required_objects ,sender=self)
This code automatically generates the user after migrating to the database.

You can use model_bakery to populate some temp data. You may need to do makemigrations and migrate to set up all tables in your database, you can follow this workflow.
python manage.py makemigrations
python manage.py migrate
In terms of populating data, you can try the following code:
from model_bakery import baker
from my_app.models import MyObject
baker.make(MyObject)
Add baker.make(MyObject) in your create_required_objects function after installation of model_bakery:
pip install model_bakery

How to access Python class in another folder when building Django management command?

I'm trying to turn a Python script into a Django management command. My script is in an application folder called sites. Folder structure:
project
|--sites
|scanner.py
|--management
|__init__.py
|--commands
|__init__.py
|getdeals.py
I'm trying to have getdeals.py run as a management command. It finds objects in my Site model and then uses them to create an instance of the SiteDeals class, which is in the scanner.py file.
getdeals.py:
from django.core.management.base import BaseCommand
from sites.models import Site
class Command(BaseCommand):
help = "Scans all sites for deals"
def handle(self, *args, **options):
site_set = Site.objects.all()
for site in site_set:
scraper = SiteDeals(site)
When I run python manage.py getdeals it says NameError: name 'SiteDeals' is not defined.
I thought of taking the code from handle and writing it as a main() function in scanner.py, and then accessing it from getdeals.py, but can't work out how to access it that way.
How do I access SiteDeals from the scanner.py file, given that it is in another folder from my management/commands folder, so that I can pass my objects to it?

You need to import SiteDeals, just as you import the Site model. Try:
from sites.scanner import SiteDeals

How to correctly override Django manage.py's command?

I need to override createsuperuser.py's handle method in Django Command class.
I created myapp\management\commands\createsuperuser.py:
import getpass
import sys
import django.contrib.auth.management.commands.createsuperuser as makesuperuser
from django.contrib.auth.management import get_default_username
from django.contrib.auth.password_validation import validate_password
from django.core import exceptions
from django.core.management.base import CommandError
from django.utils.encoding import force_str
from django.utils.text import capfirst
class Command(makesuperuser.Command):
def handle(self, *args, **options):
# the rest of code is copied from Django source and is almost
# standart except few changes related to how info of
# REQUIRED_FIELDS is shown
When I do in terminal ./manage.py createsuperuser I do not see any changes. If I change the name of my file to lets say mycmd.py and do ./manage.py mycmd everything starts to work as I expect.
How to get changes I need using ./manage.py createsuperuser?

Put your application name on top in the INSTALLED_APPS list.

Access django models inside of Scrapy

Is it possible to access my django models inside of a Scrapy pipeline, so that I can save my scraped data straight to my model?
I've seen this, but I don't really get how to set it up?

If anyone else is having the same problem, this is how I solved it.
I added this to my scrapy settings.py file:
def setup_django_env(path):
import imp, os
from django.core.management import setup_environ
f, filename, desc = imp.find_module('settings', [path])
project = imp.load_module('settings', f, filename, desc)
setup_environ(project)
setup_django_env('/path/to/django/project/')
Note: the path above is to your django project folder, not the settings.py file.
Now you will have full access to your django models inside of your scrapy project.

The opposite solution (setup scrapy in a django management command):
# -*- coding: utf-8 -*-
# myapp/management/commands/scrapy.py
from __future__ import absolute_import
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def run_from_argv(self, argv):
self._argv = argv
self.execute()
def handle(self, *args, **options):
from scrapy.cmdline import execute
execute(self._argv[1:])
and in django's settings.py:
import os
os.environ['SCRAPY_SETTINGS_MODULE'] = 'scrapy_project.settings'
Then instead of scrapy foo run ./manage.py scrapy foo.
UPD: fixed the code to bypass django's options parsing.

Add DJANGO_SETTINGS_MODULE env in your scrapy project's settings.py
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'your_django_project.settings'
Now you can use DjangoItem in your scrapy project.
Edit:
You have to make sure that the your_django_project projects settings.py is available in PYTHONPATH.

For Django 1.4, the project layout has changed. Instead of /myproject/settings.py, the settings module is in /myproject/myproject/settings.py.
I also added path's parent directory (/myproject) to sys.path to make it work correctly.
def setup_django_env(path):
import imp, os, sys
from django.core.management import setup_environ
f, filename, desc = imp.find_module('settings', [path])
project = imp.load_module('settings', f, filename, desc)
setup_environ(project)
# Add path's parent directory to sys.path
sys.path.append(os.path.abspath(os.path.join(path, os.path.pardir)))
setup_django_env('/path/to/django/myproject/myproject/')

Check out django-dynamic-scraper, it integrates a Scrapy spider manager into a Django site.
https://github.com/holgerd77/django-dynamic-scraper

Why not create a __init__.py file in the scrapy project folder and hook it up in INSTALLED_APPS? Worked for me. I was able to simply use:
piplines.py
from my_app.models import MyModel
Hope that helps.

setup-environ is deprecated. You may need to do the following in scrapy's settings file for newer versions of django 1.4+
def setup_django_env():
import sys, os, django
sys.path.append('/path/to/django/myapp')
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings'
django.setup()

Minor update to solve KeyError. Python(3)/Django(1.10)/Scrapy(1.2.0)
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = 'Scrapy commands. Accessible from: "Django manage.py". '
def __init__(self, stdout=None, stderr=None, no_color=False):
super().__init__(stdout=None, stderr=None, no_color=False)
# Optional attribute declaration.
self.no_color = no_color
self.stderr = stderr
self.stdout = stdout
# Actual declaration of CLI command
self._argv = None
def run_from_argv(self, argv):
self._argv = argv
self.execute(stdout=None, stderr=None, no_color=False)
def handle(self, *args, **options):
from scrapy.cmdline import execute
execute(self._argv[1:])
The SCRAPY_SETTINGS_MODULE declaration is still required.
os.environ.setdefault('SCRAPY_SETTINGS_MODULE', 'scrapy_project.settings')