I'm working on a number of Django projects, and in most of them, I have initial data to import.
This data isn't neat JSON, as shown in the fixture examples in the Django docs - it tends to be messy Excel or CSV files. The scripts tend to look like the following snippet, and I just run them as standalone Python, because I don't know any better:
import sys
from django.core.management import setup_environ
sys.path.append(LOCATION OF YOUR DJANGO PROJECTS, EX: 'C:\django')
from mysite import settings
setup_environ(settings)
from PROJECT_NAME.APP_NAME.models import MODEL_NAME
import csv
reader = csv.reader(open(COMPLETE PATH TO YOUR DATA, EX: "C:/django/mysite/restaurants.csv"), dialect='excel')
for row in reader:
COLUMN_ONE_TITLE = row[0]
COLUMN_TWO_TITLE = row[1]
MODEL_NAME.objects.get_or_create(MY_FIRST_COLUMN=COLUMN_ONE_TITLE, MY_SECOND_COLUMN=COLUMN_TWO_TITLE)
I feel that when I commit these projects to GitHub or send them to the client or whatever, I should include the import scripts. They're an important part of the project.
Is there a standard way, in Django, to store the import scripts that I've written to import this data?
Where in the project should they sit? Should I be using management commands?
Thanks!
I like putting them in management commands - that way when someone else works on the project, the commands are where they're expected to be by someone who wasn't familiar with the choices being made during initial development.
Django supports pluggable serialization modules. If you can arrange your CSV formats to work as a deserializer, you can use the regular loaddata command.
What if you put all that code in a utility function? so your views.py would have from utils import csv_import or something like that. That way your views can programmatically use that functionality.
I would first suggest what Martin or Luke suggested.
But if that won't work for you for some reason, then putting the script right in the root of the git project makes sense. Note, I did not suggest putting it in the root of the django project (though, frankly, that isn't a wrong idea either), but I'm suggesting that the django project be a sub folder of the git repo for the very reason that you can put these helper files somewhere that's not clogging up the django project.
Related
Seems like a really simple issue I am having, but for whatever reason I cannot get an arbitrary file i create within a Django app to access my models/compile.
analytics/
analytics/
settings.py
etc...
mapper/
models.py
views.py
filetoDothings.py
I would like "fileToDoThings.py" do things with my models, but when i put even a few simple lines of code in filetodothings, and try to import the models i get the following error: "Requested setting DEFAULT_INDEX_TABLESPACE, but settings are not configured. You must either define the environment variable DJANGO_SETTINGS_MODULE or call settings.configure()"
A lot of searching has led me to find answers similar to this:
sys.path.append("path/toproject")
os.environ["DJANGO_SETTINGS_MODULE"] = "myApp.settings"
from django.contrib.auth.models import ModelName
This worked, but only when i moved the arbitrary file to the outermost "analytics" directory. Even though I got this to work, it never quite seemed to make sense to me/seemed like a lot of work to get a file to access models in the same directory.
So what I am trying to do now is the following, in filetoDothings.py:
from models import Addresses
usaMapPoints = Addresses.objects.filter(countryCode='US').order_by("-frequency")[:25]
print(usaMapPoints)
For "messing around" purposes, I want to call "python filetodothings.py" and have it print the results in the console. But ultimately Im just trying ot get my head around how to create a file that can be automated later on.
I have seen the code for another Django app and someone was able to do exactly this.
Am I missing something really simple?
I would like "fileToDoThings.py" do things with my models
To do arbitrary operations from the command line, write a new Django management command.
You write your ‘lorem_ipsum.py’ module within a particular app, placing it a fooapp/management/commands/lorem_ipsum.py. Write it to define a command as per the documentation above.
Then you can invoke that as a management command: python3 -m manage lorem_ipsum.
Am I missing something really simple?
Simple, yes; obvious, no.
Django is a framework, not a library. The difference has been characterised by saying that with a library, you write code that calls the library; with a framework, you write code that gets called by the framework.
In other words, the way to get things done in a framework is to find the places where your code fits, and write it to fit those places.
looks like you need create command
https://docs.djangoproject.com/en/1.10/howto/custom-management-commands/
I have developed a backend library, say, cool_project. Now I want to build a web interface for it. So I create a Django project. Of course, I want to name it cool_project: my mother told me that Hungarian notation is bad and that the name cool_project is much better than any of cool_project_web etc.
But now I have a collision. As long as I try importing cool_project (the backend one) from django/cool_project/views.py (the views.py of the main Django app) a frontend package is being imported.
Is there any way to import backend project in this case? I tried to add full path to backend package (sys.path.insert(0, "/home/.../...")) but it didn't help.
Or maybe there is some well-known naming convention which helps avoiding such collisions?
You can use relative imports likefrom .. import cool_project as long you modules are in a package. I would suggest you to rename your app to something else though. It would create unnecessary complexity
This happens in Django context, but the question is not Django specific.
I'm trying to split my big urls.py into smaller and more manageable chunks (app includes, third-party modules, search, admin, development and so on). So instead of urls.py, I'm using urls/ directory now, with urls/__init__.py containing code like this:
import admin
urlpatterns += admin.urlpatterns
Now I've got into trouble with third-party modules. For example, I'm overriding a bunch of URLs and views for django-messages, and so I've put all of it into urls/messages.py. It turns out, I can't import django-messages' views then, since
from messages.views import inbox
inside urls/messages.py tries to import from local directory first, finds a messages module, but can't import messages.views then, since urls/messages/views.py is obviously absent. I wouldn't like having to rename my modules to something like urls.messages_urls, but I haven't found a way to explicitely tell Python to import the "other" messages.views. Neither would I like to specify my imported views via strings, even though I can do that in Django.
How can I solve this situation cleanly?
This is the wrong way to do it.
Django has a method for splitting urls.py into separate files, and that is to use include(). Keep the urls for each application in its own urls.py, and include them into the main one.
For those who could not find it, I was hitting ambiguous import errors. For example, in a django project, let's say I have this tree :
tax module
models module
init.py
tax.py
foo.py
In init.py, I want to import tax.models.tax.Tax.
So I write :
from tax.models.tax import Tax
But python does not find the correct import (it looks for models inside tax.models) and throws
ImportError: No module named models
You have indeed understood that it's all about relative imports. python first looks in the current module. If the first part of your import exists in the current module, it fails.
This behaviour has been fixed in Python 2.5 (and may have become default in Python 3.0), you just have to add :
from __future__ import absolute_import
before your absolute imports.
Have a look at Python 2.5 introduces absolute imports for more thorough information.
Have you tried:
from django.contrib.messages.views import inbox
Untested, but I'd think the full import would work as expected.
It is sometimes beneficial to share certain constants between various code files in a django application.
Examples:
- Name or location of dump file used in various modules\commands etc
- Debug mode on\off for the entire app
- Site specific configuration
What would be the elegant\pythonic way of doing this?
There's already a project-wide settings.py file. This is the perfect place to put your own custom setttings.
you can provide settings in your settings.py like
MY_SETTING = 'value'
and in any module you can fetch it like
from django.conf import settings
settings.MY_SETTING
Create a configuration module.
Configuration.py: (in your project/app source directory)
MYCONST1 = 1
MYCONST2 = "rabbit"
Import it from other source files:
from Configuration import MYCONST1,MYCONST2
...
Django apps are meant to be (more or less) pluggable. Therefore, you are not supposed to hack into the code of an app in order to parametrize what you want (it would be quite a mess if you had to do this ! Imagine you want to upgrade an app you downloaded on internet... you would have to re-hack into the code of the new version ?!?).
For this reason you shouldn't add app-level settings at the app level, but rather put them together somewhere in your project-wide settings.py.
I'd like to move my models to a separate directory, similar to the way it's done with Rails to cut down on code clutter. Is there any way to do this easily?
Thanks,
Collin
I assume you're using the basic webkit and not Django or something fancy. In that case just create a subdirectory called models. Put any python files you use for your models in here. Create also one blank file in this folder called __init__.py.
Then in your main.py or "controller" or what have you, put:
import models
at the top.
You just created a python package.
Brandon's answer is what I do. Furthermore, I rather like Rails's custom of one model per file. I don't stick to it completely but that is my basic pattern, especially since Python tends to encourage more-but-simpler lines of code than Ruby.
So what I do is I make models a package too:
models/
models/__init__.py
models/user.py
models/item.py
models/blog_post.py
In the main .py files I put my basic class definition, plus perhaps some helper functions (Python's module system makes it much safer to keep quickie helper functions coupled to the class definition). And my __init__.py stitches them all together:
"""The application models"""
from user import User
from item import Item
from blog_post import BlogPost
It's slightly redundant but I have lots of control of the namespace.