I have a django model, the whole code is completed. but I want to access my model info. a code like this to get field names.
for f in myModel._meta.fields:
print(f.get_attname())
is it possible to do it from an external python script without running django server?
other possible automated ways of doing this and saving results to a file are also appreciated.
try1
because Im using docker I ran it up. and from django container I started python shell
>>> from django.conf import settings
>>> settings.configure()
>>> import models
it gave django.core.exceptions.AppRegistryNotReady: Apps aren't loaded yet.
try2
by #Klaus D advice in comments I tried management command. so I created
users/
__init__.py
models.py
management/
__init__.py
commands/
__init__.py
_private.py
modelInfo.py
structure. in modelInfo.py I did
from django.core.management.base import BaseCommand, CommandError
from users import views2
def savelisttxtfile(the_list, path_, type_='w', encoding="utf-8"):
with open(path_, type_, encoding=encoding) as file_handler:
for item in the_list:
file_handler.write("{}\n".format(item))
class Command(BaseCommand):
def handle(self, *args, **options):
dic=[]
for f in views2.ChertModel._meta.fields:
print(f.get_attname())
dic.append(f.get_attname())
savelisttxtfile(dic,"F:\projects\sd.txt")
and from another python file I tried
os.chdir(r'F:\projects\users\management\commands')
from subprocess import run
import sys
run([sys.executable, r'F:\projects\users\management\commands\modelInfo.py'])
and it returned
CompletedProcess(args=['C:\\ProgramData\\Anaconda3\\python.exe', 'F:\projects\users\management\commands\modelInfo.py'], returncode=1)
and the results were not save in sd.txt
thanks to #klaus D and management command documentation I made this structure
users/
__init__.py
models.py
management/
__init__.py
commands/
__init__.py
_private.py
modelInfo.py
and in modelInfo.py I did
from django.core.management.base import BaseCommand, CommandError
from users import views2
def savelisttxtfile(the_list, path_, type_='w', encoding="utf-8"):
with open(path_, type_, encoding=encoding) as file_handler:
for item in the_list:
file_handler.write("{}\n".format(item))
class Command(BaseCommand):
def handle(self, *args, **options):
dic=[]
for f in views2.ChertModel._meta.fields:
print(f.get_attname())
dic.append(f.get_attname())
savelisttxtfile(dic,"F:\projects\sd.txt")
and to run it I went to manage.py location and executed python manage.py modelInfo to launch it.
Regarding your "try1" it seems to be a little bit trickier to start a python shell like python manage.py shell than what you propose there.
Fortunately you can do this:
python manage.py shell < your_script.py
and your script will be executed as if typed directly into the "django shell". Keep in mind that you still need to import your models relative to your project, i.e. from myapp.models import mymodel.
Related
I've put the following code at the top of my script file
os.environ.setdefault("DJANGO_SETTINGS_MODULE", 'momsite.conf.local.settings')
django.setup()
Now I can import my django apps and run small snippets (to mainly test stuff)
I'd like to import all the models registered through settings.INSTALLED_APPS
I know https://github.com/django-extensions/django-extensions does this when running manage.py shell_plus it automatically imports all the models and more.
I'm looking at their code. not sure if I'll make sense out of it.
https://github.com/django-extensions/django-extensions/blob/3355332238910f3f30a3921e604641562c79a0a8/django_extensions/management/commands/shell_plus.py#L137
at the moment, I'm doing the following, and I think it is importing models, but not available in the script somehow
from django_extensions.management.shells import import_objects
from django.core.management.base import BaseCommand, CommandError
options = {}
style = BaseCommand().style
import_objects(options, style)
edit.. answer adopted from dirkgroten
import_objects internally calls from importlib import import_module Apparently, we need to populate globals() with imported class
options = {'quiet_load': True}
style = BaseCommand().style
imported_objects = import_objects(options, style)
globals().update(imported_objects)
After you run django.setup(), do this:
from django.apps import apps
for _class in apps.get_models():
if _class.__name__.startswith("Historical"):
continue
globals()[_class.__name__] = _class
That will make all models classes available as globals in your script.
Create a management command. It will automagically load django() and everything.
Then in your command you simply start your command. ./manage.py mytest
#myapp/management/commands/mytest.py
from django.core.management.base import BaseCommand, CommandError
from myapp.sometest import Mycommand
class Command(BaseCommand):
help = 'my test script'
def add_arguments(self, parser):
pass
# parser.add_argument('poll_ids', nargs='+', type=int)
def handle(self, *args, **options):
Mycommand.something(self)
which will call the actuall script:
#sometest.py
from .models import *
class Mycommand():
def something(self):
print('...something')
I'm trying to access my Django database from within a regular Python script. So far what I did is:
import os
import django
from django.db import models
os.environ.setdefault("DJANGO_SETTINGS_MODULE", "m_site.settings")
django.setup()
from django.apps import apps
ap=apps.get_model('py','Post')
q=ap.objects.all()
for a in q:
print(a.title)
There is an application in Django called py where I have many posts (models.py, which contains class Post(models.Model)).
I would like to have the possibility to access and update this database from a regular Python script. So far script above works fine, I can insert, update or query but it looks like it is a separate database and not the database from Django's py application. Am I doing something wrong or missing something? Any help would be very appreciated.
Consider writing your script as a custom management command. This will let you run it via manage.py, with Django all properly wired up for you, e.g.
python manage.py print_post_titles
Something like this should be a good start:
from django.core.management.base import BaseCommand
from py.models import Post
class Command(BaseCommand):
help = 'Prints the titles of all Posts'
def handle(self, *args, **options):
for post in Post.objects.all():
print(a.title)
I'm trying to turn a Python script into a Django management command. My script is in an application folder called sites. Folder structure:
project
|--sites
|scanner.py
|--management
|__init__.py
|--commands
|__init__.py
|getdeals.py
I'm trying to have getdeals.py run as a management command. It finds objects in my Site model and then uses them to create an instance of the SiteDeals class, which is in the scanner.py file.
getdeals.py:
from django.core.management.base import BaseCommand
from sites.models import Site
class Command(BaseCommand):
help = "Scans all sites for deals"
def handle(self, *args, **options):
site_set = Site.objects.all()
for site in site_set:
scraper = SiteDeals(site)
When I run python manage.py getdeals it says NameError: name 'SiteDeals' is not defined.
I thought of taking the code from handle and writing it as a main() function in scanner.py, and then accessing it from getdeals.py, but can't work out how to access it that way.
How do I access SiteDeals from the scanner.py file, given that it is in another folder from my management/commands folder, so that I can pass my objects to it?
You need to import SiteDeals, just as you import the Site model. Try:
from sites.scanner import SiteDeals
I have a flask app with the following directory structure:
myapp/
application.py
__init__.py
models/
__init__.py
user.py
The models use Flask-SQLAlchemy, and therefore, they need to have access to the db object (a SQLAlchemy instance) from application.py
user.py:
import sys,os
sys.path.append('/path/to/application/package')
from testapp import db
class User(db.Model):
id = db.Column(db.Integer,primary_key=True)
username = db.Column(db.String(255),unique=True)
age = db.Column(db.Integer)
def __init__(self,username,age):
self.username = username
self.age = age
def __repr__(self):
return '<User %r>' % self.username
Because any of the models need access to the application's SQLAlchemy instance, the db property, I have to throw this whole package on the path and then import from the main application module. For sanity's sake, I would like to keep the models in separate files. Will I need to put the path code on top of every model? Is there a better way? I'd rather not have the full path input like that, as they may be deployed to different hosts with different directory structures. Ideally there would be some way to internally handle the path so when it is used as another user via mod_wsgi I don't have to manually change the code.
1st approach:
I've ended up with the following structure:
project_root — also holds some configs, .gitignore file, etc
start.py
flask_root
__init__.py
application.py
module_1
__init__.py
models.py
module_2
__init__.py
models.py
Topmost start.py just runs the app:
#! /usr/bin/env python
from flask_root import applicaiton
if __name__ == '__main__':
application.manager.run()
Python searches for packages in the directory you script started from, so now you don't need add them to sys.path (as for me, modification of sys.path looks ugly).
Now you have full-working flask_root python package, and you can import everything from it, from any place of your application:
from flask_root.application import db
2nd approach:
If you start your Flask application from it's directory,
./application.py runserver
the directory you've started from is not be accessible as python package, even if it has __init__.py in it.
Though, with your directory layout you can do the following trick:
models/__init__.py:
from application import db
...
models/user.py:
from . import db
...
The first approach is more clean and universal. The second possibly can be useful when you need to share same blueprints between multiple Flask projects.
Is it possible to access my django models inside of a Scrapy pipeline, so that I can save my scraped data straight to my model?
I've seen this, but I don't really get how to set it up?
If anyone else is having the same problem, this is how I solved it.
I added this to my scrapy settings.py file:
def setup_django_env(path):
import imp, os
from django.core.management import setup_environ
f, filename, desc = imp.find_module('settings', [path])
project = imp.load_module('settings', f, filename, desc)
setup_environ(project)
setup_django_env('/path/to/django/project/')
Note: the path above is to your django project folder, not the settings.py file.
Now you will have full access to your django models inside of your scrapy project.
The opposite solution (setup scrapy in a django management command):
# -*- coding: utf-8 -*-
# myapp/management/commands/scrapy.py
from __future__ import absolute_import
from django.core.management.base import BaseCommand
class Command(BaseCommand):
def run_from_argv(self, argv):
self._argv = argv
self.execute()
def handle(self, *args, **options):
from scrapy.cmdline import execute
execute(self._argv[1:])
and in django's settings.py:
import os
os.environ['SCRAPY_SETTINGS_MODULE'] = 'scrapy_project.settings'
Then instead of scrapy foo run ./manage.py scrapy foo.
UPD: fixed the code to bypass django's options parsing.
Add DJANGO_SETTINGS_MODULE env in your scrapy project's settings.py
import os
os.environ['DJANGO_SETTINGS_MODULE'] = 'your_django_project.settings'
Now you can use DjangoItem in your scrapy project.
Edit:
You have to make sure that the your_django_project projects settings.py is available in PYTHONPATH.
For Django 1.4, the project layout has changed. Instead of /myproject/settings.py, the settings module is in /myproject/myproject/settings.py.
I also added path's parent directory (/myproject) to sys.path to make it work correctly.
def setup_django_env(path):
import imp, os, sys
from django.core.management import setup_environ
f, filename, desc = imp.find_module('settings', [path])
project = imp.load_module('settings', f, filename, desc)
setup_environ(project)
# Add path's parent directory to sys.path
sys.path.append(os.path.abspath(os.path.join(path, os.path.pardir)))
setup_django_env('/path/to/django/myproject/myproject/')
Check out django-dynamic-scraper, it integrates a Scrapy spider manager into a Django site.
https://github.com/holgerd77/django-dynamic-scraper
Why not create a __init__.py file in the scrapy project folder and hook it up in INSTALLED_APPS? Worked for me. I was able to simply use:
piplines.py
from my_app.models import MyModel
Hope that helps.
setup-environ is deprecated. You may need to do the following in scrapy's settings file for newer versions of django 1.4+
def setup_django_env():
import sys, os, django
sys.path.append('/path/to/django/myapp')
os.environ['DJANGO_SETTINGS_MODULE'] = 'myapp.settings'
django.setup()
Minor update to solve KeyError. Python(3)/Django(1.10)/Scrapy(1.2.0)
from django.core.management.base import BaseCommand
class Command(BaseCommand):
help = 'Scrapy commands. Accessible from: "Django manage.py". '
def __init__(self, stdout=None, stderr=None, no_color=False):
super().__init__(stdout=None, stderr=None, no_color=False)
# Optional attribute declaration.
self.no_color = no_color
self.stderr = stderr
self.stdout = stdout
# Actual declaration of CLI command
self._argv = None
def run_from_argv(self, argv):
self._argv = argv
self.execute(stdout=None, stderr=None, no_color=False)
def handle(self, *args, **options):
from scrapy.cmdline import execute
execute(self._argv[1:])
The SCRAPY_SETTINGS_MODULE declaration is still required.
os.environ.setdefault('SCRAPY_SETTINGS_MODULE', 'scrapy_project.settings')