I have a django model, the whole code is completed. but I want to access my model info. a code like this to get field names.
for f in myModel._meta.fields:
print(f.get_attname())
is it possible to do it from an external python script without running django server?
other possible automated ways of doing this and saving results to a file are also appreciated.
try1
because Im using docker I ran it up. and from django container I started python shell
>>> from django.conf import settings
>>> settings.configure()
>>> import models
it gave django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
try2
by #Klaus D advice in comments I tried management command. so I created
users/
__init__.py
models.py
management/
__init__.py
commands/
__init__.py
_private.py
modelInfo.py
structure. in modelInfo.py I did
from django.core.management.base import BaseCommand, CommandError
from users import views2
def savelisttxtfile(the_list, path_, type_='w', encoding="utf-8"):
with open(path_, type_, encoding=encoding) as file_handler:
for item in the_list:
file_handler.write("{}\n".format(item))
class Command(BaseCommand):
def handle(self, *args, **options):
dic=[]
for f in views2.ChertModel._meta.fields:
print(f.get_attname())
dic.append(f.get_attname())
savelisttxtfile(dic,"F:\projects\sd.txt")
and from another python file I tried
os.chdir(r'F:\projects\users\management\commands')
from subprocess import run
import sys
run([sys.executable, r'F:\projects\users\management\commands\modelInfo.py'])
and it returned
CompletedProcess(args=['C:\\ProgramData\\Anaconda3\\python.exe', 'F:\projects\users\management\commands\modelInfo.py'], returncode=1)
and the results were not save in sd.txt
thanks to #klaus D and management command documentation I made this structure
users/
__init__.py
models.py
management/
__init__.py
commands/
__init__.py
_private.py
modelInfo.py
and in modelInfo.py I did
from django.core.management.base import BaseCommand, CommandError
from users import views2
def savelisttxtfile(the_list, path_, type_='w', encoding="utf-8"):
with open(path_, type_, encoding=encoding) as file_handler:
for item in the_list:
file_handler.write("{}\n".format(item))
class Command(BaseCommand):
def handle(self, *args, **options):
dic=[]
for f in views2.ChertModel._meta.fields:
print(f.get_attname())
dic.append(f.get_attname())
savelisttxtfile(dic,"F:\projects\sd.txt")
and to run it I went to manage.py location and executed python manage.py modelInfo to launch it.
Regarding your "try1" it seems to be a little bit trickier to start a python shell like python manage.py shell than what you propose there.
Fortunately you can do this:
python manage.py shell < your_script.py
and your script will be executed as if typed directly into the "django shell". Keep in mind that you still need to import your models relative to your project, i.e. from myapp.models import mymodel.
I want to separate my model classes into separate files in a models directory. I would like to have a separate file for:
general (authentication and global classes/tables)
requisitions (tables used for requisitions)
workorders (tables used for workorders)
sales_orders (tables used for sales orders)
...etc
I'm not sure how to structure my project to make that happen.
I've tried putting my main imports into init.py in the directory and them importing those into the individual model files, but I don't know where to put my db.generate_mapping() so that all classes are available. I'm guessing this is a best practice for a large application. I've got about 150 tables in my app at this point.
Any help/pointers would be appreciated.
You can use the following project structure:
# /myproject
# settings.py
# main.py
# /models
# __init__.py
# base.py
# requisitions.py
# workorders.py
# sales_orders.py
settings.py is a file with database settings:
# settings.py
db_params = {'provider': 'sqlite', 'filename': ':memory:'}
main.py is a file when you start application. You put db.generate_mapping here:
# main.py
from pony import orm
import settings
from models import db
from your_favorite_web_framework import App
# orm.set_sql_degug(True)
db.bind(**settings.db_params)
db.generate_mapping(create_tables=True)
with orm.db_session:
orm.select(u for u in db.User).show()
if __name__ == '__main__':
app = App()
app.run()
Note that it is not necessary to implicitly import all models, as they are accessible as attributes of db object (like db.User)
You can put db object in base.py (or general.py), where you define your core models:
# base.py
from pony import orm
db = orm.Database()
class User(db.Entity):
name = Required(str)
orders = Set('Order')
Note that in User model I can refer to Order model defined in another module. You can also write it as
orders = Set(lambda: db.Order)
Unfortunately, IDEs like PyCharm at this moment cannot recognize that db.Order refers to specific Order class. You can import and use Order class directly, but in some cases it will lead to problem with cyclic imports.
In other model files you import db from .base:
# workorders.py
from pony import orm
from .base import db
class Order(db.Entity):
description = Optional(str)
user = Required('User')
In /models/__init__.py you import all modules to ensure that all models classes are defined before generate_mapping is called:
# /models/__init__.py
from .base import db
from . import workorders
...
And then you can write
from models import db
And access models as db attributes like db.User, db.Order, etc.
If you want to refer models directly in your code instead of accessing them as db attributes, you can import all models in __init__.py:
# /models/__init__.py
from .base import db, User
from .workorders import Order
...
And then you can import model classes as:
from models import db, User, Order, ...
I'm trying to turn a Python script into a Django management command. My script is in an application folder called sites. Folder structure:
project
|--sites
|scanner.py
|--management
|__init__.py
|--commands
|__init__.py
|getdeals.py
I'm trying to have getdeals.py run as a management command. It finds objects in my Site model and then uses them to create an instance of the SiteDeals class, which is in the scanner.py file.
getdeals.py:
from django.core.management.base import BaseCommand
from sites.models import Site
class Command(BaseCommand):
help = "Scans all sites for deals"
def handle(self, *args, **options):
site_set = Site.objects.all()
for site in site_set:
scraper = SiteDeals(site)
When I run python manage.py getdeals it says NameError: name 'SiteDeals' is not defined.
I thought of taking the code from handle and writing it as a main() function in scanner.py, and then accessing it from getdeals.py, but can't work out how to access it that way.
How do I access SiteDeals from the scanner.py file, given that it is in another folder from my management/commands folder, so that I can pass my objects to it?
You need to import SiteDeals, just as you import the Site model. Try:
from sites.scanner import SiteDeals
My project:
project_name
|- my_app
|- __init__.py
|- run.py
|- models.py
First example
run.py
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
db = SQLAlchemy(app)
from models import User
db.create_all()
if __name__ == '__main__':
app.run()
models.py
from run import db
class User(db.Model):
#...
__ init.py __ is empty
After running this example I get this error:
ImportError: cannot import name User
This error describe Circular import of app variable in models.py (as I understand).
Second example
run.py
from my_app import app
if __name__ == '__main__':
app.run()
models.py
from my_app import db
class User(db.Model):
#...
__ init.p __
from flask import Flask
from flask_sqlalchemy import SQLAlchemy
app = Flask(__name__)
db = SQLAlchemy(app)
from models import User
db.create_all()
And now all works correct.
And, at this point, I don't understand, why the same code give me different logic?
Where is the magic? Why Circular import in __ init.py __ does't throw an error?
Thank You!
I'm just going to talk about your first example, because you can actually reproduce both scenarios without changing the code:
Example 1: Run run.py directly to see the Import error
Example 2: Open a python repl, and run:
from run import app
app.run()
And your first example now works, for the same reason your second code example did: the imports were moved out of __main__
In example 1, run.py is the top level execution environment, and the code here runs in __main__. The relative import to models.py requires that all references in models.py be resolved (since the class you're importing from models.py may be dependent on other parts of models.py outside of the class itself). The first thing models.py does is go back to __main__ to perform a relative import of db, and since it needs to resolve all references, it does - and one of the references it tries to resolve is the original from models import user statement. Voila, ImportError.
The thing is, at this point run.py hasn't finished executing, and so far it's still trying to import the definition of User. But now models.py is trying to call into it as though it had finished loading, and it expects run.py to already know the definition of User, even though it's right in the middle of trying to figure that exact thing out!
This isn't a circular import - not really. models.py is just trying to import from a file that hasn't finished executing yet, and whose contents aren't available for use.
By moving __main__ elsewhere - into the repl, or by moving your code around so that run.py doesn't define anything - you're giving the code a chance to actually resolve all the dependencies before any code tries to use them. By using from run import app; app.run(), this is the only code that is not done executing. The imports themselves all get to complete their work, resolve their dependencies and make them available to __main__
tl;dr don't accidentally write code that imports from __main__ when __main__ doesn't yet know the things it's being asked for
I have a Django app that requires a settings attribute in the form of:
RELATED_MODELS = ('appname1.modelname1.attribute1',
'appname1.modelname2.attribute2',
'appname2.modelname3.attribute3', ...)
Then hooks their post_save signal to update some other fixed model depending on the attributeN defined.
I would like to test this behaviour and tests should work even if this app is the only one in the project (except for its own dependencies, no other wrapper app need to be installed). How can I create and attach/register/activate mock models just for the test database? (or is it possible at all?)
Solutions that allow me to use test fixtures would be great.
You can put your tests in a tests/ subdirectory of the app (rather than a tests.py file), and include a tests/models.py with the test-only models.
Then provide a test-running script (example) that includes your tests/ "app" in INSTALLED_APPS. (This doesn't work when running app tests from a real project, which won't have the tests app in INSTALLED_APPS, but I rarely find it useful to run reusable app tests from a project, and Django 1.6+ doesn't by default.)
(NOTE: The alternative dynamic method described below only works in Django 1.1+ if your test case subclasses TransactionTestCase - which slows down your tests significantly - and no longer works at all in Django 1.7+. It's left here only for historical interest; don't use it.)
At the beginning of your tests (i.e. in a setUp method, or at the beginning of a set of doctests), you can dynamically add "myapp.tests" to the INSTALLED_APPS setting, and then do this:
from django.core.management import call_command
from django.db.models import loading
loading.cache.loaded = False
call_command('syncdb', verbosity=0)
Then at the end of your tests, you should clean up by restoring the old version of INSTALLED_APPS and clearing the app cache again.
This class encapsulates the pattern so it doesn't clutter up your test code quite as much.
#paluh's answer requires adding unwanted code to a non-test file and in my experience, #carl's solution does not work with django.test.TestCase which is needed to use fixtures. If you want to use django.test.TestCase, you need to make sure you call syncdb before the fixtures get loaded. This requires overriding the _pre_setup method (putting the code in the setUp method is not sufficient). I use my own version of TestCase that lets me add apps with test models. It is defined as follows:
from django.conf import settings
from django.core.management import call_command
from django.db.models import loading
from django import test
class TestCase(test.TestCase):
apps = ()
def _pre_setup(self):
# Add the models to the db.
self._original_installed_apps = list(settings.INSTALLED_APPS)
for app in self.apps:
settings.INSTALLED_APPS.append(app)
loading.cache.loaded = False
call_command('syncdb', interactive=False, verbosity=0)
# Call the original method that does the fixtures etc.
super(TestCase, self)._pre_setup()
def _post_teardown(self):
# Call the original method.
super(TestCase, self)._post_teardown()
# Restore the settings.
settings.INSTALLED_APPS = self._original_installed_apps
loading.cache.loaded = False
I shared my solution that I use in my projects. Maybe it helps someone.
pip install django-fake-model
Two simple steps to create fake model:
1) Define model in any file (I usualy define model in test file near a test case)
from django_fake_model import models as f
class MyFakeModel(f.FakeModel):
name = models.CharField(max_length=100)
2) Add decorator #MyFakeModel.fake_me to your TestCase or to test function.
class MyTest(TestCase):
#MyFakeModel.fake_me
def test_create_model(self):
MyFakeModel.objects.create(name='123')
model = MyFakeModel.objects.get(name='123')
self.assertEqual(model.name, '123')
This decorator creates table in your database before each test and remove the table after test.
Also you may create/delete table manually: MyFakeModel.create_table() / MyFakeModel.delete_table()
I've figured out a way for test-only models for django 1.7+.
The basic idea is, make your tests an app, and add your tests to INSTALLED_APPS.
Here's an example:
$ ls common
__init__.py admin.py apps.py fixtures models.py pagination.py tests validators.py views.py
$ ls common/tests
__init__.py apps.py models.py serializers.py test_filter.py test_pagination.py test_validators.py views.py
And I have different settings for different purposes(ref: splitting up the settings file), namely:
settings/default.py: base settings file
settings/production.py: for production
settings/development.py: for development
settings/testing.py: for testing.
And in settings/testing.py, you can modify INSTALLED_APPS:
settings/testing.py:
from default import *
DEBUG = True
INSTALLED_APPS += ['common', 'common.tests']
And make sure that you have set a proper label for your tests app, namely,
common/tests/apps.py
from django.apps import AppConfig
class CommonTestsConfig(AppConfig):
name = 'common.tests'
label = 'common_tests'
common/tests/__init__.py, set up proper AppConfig(ref: Django Applications).
default_app_config = 'common.tests.apps.CommonTestsConfig'
Then, generate db migration by
python manage.py makemigrations --settings=<your_project_name>.settings.testing tests
Finally, you can run your test with param --settings=<your_project_name>.settings.testing.
If you use py.test, you can even drop a pytest.ini file along with django's manage.py.
py.test
[pytest]
DJANGO_SETTINGS_MODULE=kungfu.settings.testing
Quoting from a related answer:
If you want models defined for testing only then you should check out
Django ticket #7835 in particular comment #24 part of which
is given below:
Apparently you can simply define models directly in your tests.py.
Syncdb never imports tests.py, so those models won't get synced to the
normal db, but they will get synced to the test database, and can be
used in tests.
This solution works only for earlier versions of django (before 1.7). You can check your version easily:
import django
django.VERSION < (1, 7)
Original response:
It's quite strange but form me works very simple pattern:
add tests.py to app which you are going to test,
in this file just define testing models,
below put your testing code (doctest or TestCase definition),
Below I've put some code which defines Article model which is needed only for tests (it exists in someapp/tests.py and I can test it just with: ./manage.py test someapp ):
class Article(models.Model):
title = models.CharField(max_length=128)
description = models.TextField()
document = DocumentTextField(template=lambda i: i.description)
def __unicode__(self):
return self.title
__test__ = {"doctest": """
#smuggling model for tests
>>> from .tests import Article
#testing data
>>> by_two = Article.objects.create(title="divisible by two", description="two four six eight")
>>> by_three = Article.objects.create(title="divisible by three", description="three six nine")
>>> by_four = Article.objects.create(title="divisible by four", description="four four eight")
>>> Article.objects.all().search(document='four')
[<Article: divisible by two>, <Article: divisible by four>]
>>> Article.objects.all().search(document='three')
[<Article: divisible by three>]
"""}
Unit tests also working with such model definition.
I chose a slightly different, albeit more coupled, approach to dynamically creating models just for testing.
I keep all my tests in a tests subdirectory that lives in my files app. The models.py file in the tests subdirectory contains my test-only models. The coupled part comes in here, where I need to add the following to my settings.py file:
# check if we are testing right now
TESTING = 'test' in sys.argv
if TESTING:
# add test packages that have models
INSTALLED_APPS += ['files.tests',]
I also set db_table in my test model, because otherwise Django would have created the table with the name tests_<model_name>, which may have caused a conflict with other test models in another app. Here's my my test model:
class Recipe(models.Model):
'''Test-only model to test out thumbnail registration.'''
dish_image = models.ImageField(upload_to='recipes/')
class Meta:
db_table = 'files_tests_recipe'
Here's the pattern that I'm using to do this.
I've written this method that I use on a subclassed version of TestCase. It goes as follows:
#classmethod
def create_models_from_app(cls, app_name):
"""
Manually create Models (used only for testing) from the specified string app name.
Models are loaded from the module "<app_name>.models"
"""
from django.db import connection, DatabaseError
from django.db.models.loading import load_app
app = load_app(app_name)
from django.core.management import sql
from django.core.management.color import no_style
sql = sql.sql_create(app, no_style(), connection)
cursor = connection.cursor()
for statement in sql:
try:
cursor.execute(statement)
except DatabaseError, excn:
logger.debug(excn.message)
pass
Then, I create a special test-specific models.py file in something like myapp/tests/models.py that's not included in INSTALLED_APPS.
In my setUp method, I call create_models_from_app('myapp.tests') and it creates the proper tables.
The only "gotcha" with this approach is that you don't really want to create the models ever time setUp runs, which is why I catch DatabaseError. I guess the call to this method could go at the top of the test file and that would work a little better.
Combining your answers, specially #slacy's, I did this:
class TestCase(test.TestCase):
initiated = False
#classmethod
def setUpClass(cls, *args, **kwargs):
if not TestCase.initiated:
TestCase.create_models_from_app('myapp.tests')
TestCase.initiated = True
super(TestCase, cls).setUpClass(*args, **kwargs)
#classmethod
def create_models_from_app(cls, app_name):
"""
Manually create Models (used only for testing) from the specified string app name.
Models are loaded from the module "<app_name>.models"
"""
from django.db import connection, DatabaseError
from django.db.models.loading import load_app
app = load_app(app_name)
from django.core.management import sql
from django.core.management.color import no_style
sql = sql.sql_create(app, no_style(), connection)
cursor = connection.cursor()
for statement in sql:
try:
cursor.execute(statement)
except DatabaseError, excn:
logger.debug(excn.message)
With this, you don't try to create db tables more than once, and you don't need to change your INSTALLED_APPS.
If you are writing a reusable django-app, create a minimal test-dedicated app for it!
$ django-admin.py startproject test_myapp_project
$ django-admin.py startapp test_myapp
add both myapp and test_myapp to the INSTALLED_APPS, create your models there and it's good to go!
I have gone through all these answers as well as django ticket 7835, and I finally went for a totally different approach.
I wanted my app (somehow extending queryset.values() ) to be able to be tested in isolation; also, my package does include some models and I wanted a clean distinction between test models and package ones.
That's when I realized it was easier to add a very small django project in the package!
This also allows a much cleaner separation of code IMHO:
In there you can cleanly and without any hack define your models, and you know they will be created when you run your tests from in there!
If you are not writing an independent, reusable app you can still go this way: create a test_myapp app, and add it to your INSTALLED_APPS only in a separate settings_test_myapp.py!
Someone already mentioned Django ticket #7835, but there appears to be a more recent reply that looks much more promising for more recent versions of Django. Specifically #42, which proposes a different TestRunner:
from importlib.util import find_spec
import unittest
from django.apps import apps
from django.conf import settings
from django.test.runner import DiscoverRunner
class TestLoader(unittest.TestLoader):
""" Loader that reports all successful loads to a runner """
def __init__(self, *args, runner, **kwargs):
self.runner = runner
super().__init__(*args, **kwargs)
def loadTestsFromModule(self, module, pattern=None):
suite = super().loadTestsFromModule(module, pattern)
if suite.countTestCases():
self.runner.register_test_module(module)
return suite
class RunnerWithTestModels(DiscoverRunner):
""" Test Runner that will add any test packages with a 'models' module to INSTALLED_APPS.
Allows test only models to be defined within any package that contains tests.
All test models should be set with app_label = 'tests'
"""
def __init__(self, *args, **kwargs):
self.test_packages = set()
self.test_loader = TestLoader(runner=self)
super().__init__(*args, **kwargs)
def register_test_module(self, module):
self.test_packages.add(module.__package__)
def setup_databases(self, **kwargs):
# Look for test models
test_apps = set()
for package in self.test_packages:
if find_spec('.models', package):
test_apps.add(package)
# Add test apps with models to INSTALLED_APPS that aren't already there
new_installed = settings.INSTALLED_APPS + tuple(ta for ta in test_apps if ta not in settings.INSTALLED_APPS)
apps.set_installed_apps(new_installed)
return super().setup_databases(**kwargs)