Folder Structure
I'm currently learning Django from the Django book. I have the following directory structure:
.
└── mysite
├── books
│ ├── __init__.py
│ ├── models.py
│ ├── templates
│ │ ├── search_form.html
│ │ └── search_results.html
│ ├── tests.py
│ └── views.py
├── contact
│ ├── forms.py
│ ├── __init__.py
│ ├── templates
│ │ └── contact_form.html
│ └── views.py
├── manage.py
└── mysite
├── __init__.py
├── settings.py
├── templates
│ ├── base.html
│ ├── current_datetime.html
│ └── hours_ahead.html
├── urls.py
├── views.py
└── wsgi.py
Where books, contact, and mysite are folders. I also have a separate templates subfolder for each of those folders to store templates.
Story
Now, inside /mysite/settings.py, I have the following lines:
TEMPLATE_DIRS = (
os.path.join(os.path.dirname(__file__), 'templates').replace('\\', '/'),
)
Inside /contact/views.py, I have the following line (note that render here is from django.shortcuts):
return render(request, 'contact_form.html', {'form': form})
However, the line above gives me a TemplateDoesNotExist exception. This is because the template loader looks for the file contact_form.html under /mysite/templates/ instead of /contact/templates/.
Fair enough, even though I don't understand why that's the case. But, I have another case where it behaves differently.
Recall in the file structure above, I also have books/views.py. In it, I have the following lines (in different logical blocks):
return render(request, 'search_results.html',
{'books': books, 'query': q})
and
return render(request, 'search_form.html', {'errors': errors})
If you look back above, search_results.html and search_form.html are under the /books/templates/ folder, not /mysite/templates/ folder, and yet the template loader managed to find those files. Note that it still works even when I move search_results.html and search_form.html into /mysite/templates/.
So my question is, why is there a difference in where the loader looks for templates?
Files
Anyway, here's what my /contact/views.py looks like:
from django.shortcuts import render
from django.http import HttpResponseRedirect
from django.core.mail import send_mail
from forms import ContactForm
import os
def contact(request):
if request.method == 'POST':
form = ContactForm(request.POST)
if form.is_valid():
cd = form.cleaned_data
send_mail(
cd['subject'],
cd['message'],
cd.get('email', 'noreply#example.com'),
['siteowner#example.com'],
)
return HttpResponseRedirect('/contact/thanks/')
else:
form = ContactForm()
print os.path.join(os.path.dirname(__file__), 'templates')
return render(request, 'contact_form.html', {'form': form})
Here's my /books/views.py:
from django.shortcuts import render
from django.http import HttpResponse
from books.models import Book
def search(request):
errors = []
if 'q' in request.GET:
q = request.GET['q']
if not q:
errors.append('Enter a search term.')
elif len(q) > 20:
errors.append('Please enter at most 20 characters.')
else:
books = Book.objects.filter(title__icontains=q)
return render(request, 'search_results.html',
{'books': books, 'query': q})
return render(request, 'search_form.html', {'errors': errors})
Since you apparently do have the app directories template loader installed, and since it works on your books app, I can only conclude you don't have contact in settings.INSTALLED_APPS.
Related
I am learning to use celery with django through this tutorial. In this tutorial, the person is developing a webscraping tool with django and celery. I am trying to follow the tutorial but I am facing the following error message
TemplateDoesNotExist at /
home.html, scraping/products_list.html
this is how my files are arranged
.
├── celerybeat-schedule.db
├── celerydjangotutorial
│ ├── __init__.py
│ ├── asgi.py
│ ├── celery.py
│ ├── settings.py
│ ├── templates
│ │ ├── base.html
│ │ └── home.html
│ ├── urls.py
│ ├── views.py
│ └── wsgi.py
├── db.sqlite3
├── manage.py
└── scraping
├── __init__.py
├── admin.py
├── apps.py
├── migrations
│ ├── 0001_initial.py
│ ├── 0002_auto_20200827_0735.py
│ ├── __init__.py
│ └── __pycache__
│ ├── 0001_initial.cpython-38.pyc
│ ├── 0002_auto_20200827_0735.cpython-38.pyc
│ └── __init__.cpython-38.pyc
├── models.py
├── tasks.py
├── tests.py
└── views.py
where celerydjangotutorial is the django project and scraping is a django application. In the tutorial, the person places the templates into the project directory. he also creates a views file in the project directory. I followed him as he does in the tutorial. here is my code.
celerydjangotutorial/urls.py
from django.contrib import admin
from django.urls import path, include
from .views import HomePageView
urlpatterns = [
path('', HomePageView.as_view(), name='home'),
path('admin/', admin.site.urls),
]
celerydjangotutorial/views.py
from django.shortcuts import render
from django.views import generic
from scraping.models import Products
class HomePageView(generic.ListView):
template_name = 'home.html'
context_object_name = 'products'
def get_queryset(self):
return Products.objects.all()
scraping/models.py
from django.db import models
class Products(models.Model):
title = models.CharField(max_length=200)
link = models.CharField(max_length=2083, default="", unique=True)
created_at = models.DateTimeField(auto_now_add=True)
updated_at = models.DateTimeField(auto_now=True)
website = models.CharField(max_length=30, default="", blank=True, null=True)
scraping/tasks.py
import time
import json
from datetime import datetime
from celery import Celery
from celery.schedules import crontab
from celery import shared_task
from selenium import webdriver
from selenium.webdriver.support.ui import WebDriverWait
from selenium.webdriver.support import expected_conditions as EC
from selenium.webdriver.common.by import By
from selenium.common.exceptions import TimeoutException, NoSuchElementException
from selenium.webdriver.common.keys import Keys
#shared_task
def scrape():
try:
print('starting the scraping process')
product_list = []
url = f'https://www.amazon.com/s?k=chips&ref=nb_sb_noss_2'
driver = webdriver.Chrome('./chromedriver')
driver.get(url)
driver.implicitly_wait(15)
links = driver.find_elements_by_class_name("a-size-mini")
for link in links:
element = WebDriverWait(driver, 5).until(
EC.presence_of_element_located((By.LINK_TEXT, link.text)))
product_meta = {
'link': element.get_attribute('href'),
'website': 'amazon'
}
product_list.append(product_meta)
print('scraping process completed')
return save_function(product_list)
except Exception as e:
print('scraping failed')
print(e)
#shared_task(serializer='json')
def save_function(product_list):
print('saving extracted data')
new_count = 0
for product in product_list:
try:
Products.objects.create(
title = product['title'],
link = product['link'],
website = product['website']
)
new_count += 1
except Exception as e:
print('failed at latest_product is None')
print(e)
break
return print('finished')
celerydjangotutorial/celery.py
from __future__ import absolute_import
import os
from celery import Celery
from celery.schedules import crontab
os.environ.setdefault('DJANGO_SETTINGS_MODULE','celerydjangotutorial.settings')
app = Celery('celerydjangotutorial')
app.conf.timezone = 'UTC'
app.config_from_object("django.conf:settings", namespace="CELERY")
app.autodiscover_tasks()
settings.py
...
INSTALLED_APPS = [
'django.contrib.admin',
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.messages',
'django.contrib.staticfiles',
'scraping.apps.ScrapingConfig'
]
...
TEMPLATES = [
{
'BACKEND': 'django.template.backends.django.DjangoTemplates',
'DIRS': ['templates'],
'APP_DIRS': True,
'OPTIONS': {
'context_processors': [
'django.template.context_processors.debug',
'django.template.context_processors.request',
'django.contrib.auth.context_processors.auth',
'django.contrib.messages.context_processors.messages',
],
},
},
]
...
CELERY_BROKER_URL = 'amqp://localhost:5672'
CELERY_RESULT_BACKEND = 'amqp://localhost:5672'
CELERY_ACCEPT_CONTENT = ['application/json']
CELERY_TASK_SERIALIZER = 'json'
CELERY_RESULT_SERIALIZER = 'json'
CELERY_TIMEZONE = 'UTC'
I am not sure what I am doing wrong. I followed most of the tutorial. Except for tasks.py the rest of the code is similar to his code. Please help me.
Thanks in advance
[EDIT-1] posting the entire error
Internal Server Error: /
Traceback (most recent call last):
File "/Users/sashaanksekar/anaconda3/lib/python3.8/site-packages/django/core/handlers/exception.py", line 34, in inner
response = get_response(request)
File "/Users/sashaanksekar/anaconda3/lib/python3.8/site-packages/django/core/handlers/base.py", line 145, in _get_response
response = self.process_exception_by_middleware(e, request)
File "/Users/sashaanksekar/anaconda3/lib/python3.8/site-packages/django/core/handlers/base.py", line 143, in _get_response
response = response.render()
File "/Users/sashaanksekar/anaconda3/lib/python3.8/site-packages/django/template/response.py", line 105, in render
self.content = self.rendered_content
File "/Users/sashaanksekar/anaconda3/lib/python3.8/site-packages/django/template/response.py", line 81, in rendered_content
template = self.resolve_template(self.template_name)
File "/Users/sashaanksekar/anaconda3/lib/python3.8/site-packages/django/template/response.py", line 63, in resolve_template
return select_template(template, using=self.using)
File "/Users/sashaanksekar/anaconda3/lib/python3.8/site-packages/django/template/loader.py", line 47, in select_template
raise TemplateDoesNotExist(', '.join(template_name_list), chain=chain)
django.template.exceptions.TemplateDoesNotExist: home.html, scraping/products_list.html
Try adding app name under templates folder and move HTML files into it.
So it should be like
templates/celerydjangotutorial/
base.html
home.html
Check the names of files, folders and path
I am discovering Django (v2.1.1) and want to set up a signin page in which I have 2 emails fields, if the 2 fields are identical, I call form.is_valid().
The project tree :
├── manage.py
├── requirements.txt
├── project
│ ├── __init__.py
│ ├── settings.py
│ ├── urls.py
│ ├── views.py
│ ├── wsgi.py
│ └── templates
│ ├── base.html
│ ├── project
│ │ └── home.html
│ └── registration
│ ├── logged_out.html
│ ├── login.html
│ └── signin.html
└── app
├── __init__.py
├── admin.py
├── apps.py
├── models.py
├── urls.py
├── views.py
├── migrations
└── templates
└── app
└── home.html
Internationalization config in project/settings.py :
# (…)
# Internationalization
LANGUAGE_CODE = 'fr-fr'
USE_I18N = True
USE_L10N = True
# (…)
With a basic django signin view it works well:
project/views.py :
from django.shortcuts import render, redirect
from django.contrib.auth import login, authenticate
from django.contrib.auth.forms import UserCreationForm
def index(request):
return render(request, 'project/home.html', {'context':'project index'})
def signin(request):
# (…)
form = UserCreationForm()
return render(request, 'registration/signin.html', {'form': form})
I have a nice French translated HTML form (except for the submit button) :
Let's add an email field in the form, email is a built-in user fields (as first_name & last_name) so I just add a new class SignInForm inheriting from UserCreationForm :
project/forms.py :
from django.contrib.auth.forms import UserCreationForm
from django.contrib.auth.models import User
class SignInForm(UserCreationForm):
class Meta:
model = User
fields = ('username', 'email', 'password1', 'password2')
I update project/views.py :
# (…)
from project.forms import SignInForm
# (…)
def signin(request):
# (…)
form = SignInForm()
return render(request, 'registration/signin.html', {'form': form})
It works : a nice French translated field is added but it do not have a help_text attribute :
Now I'm stuck…
If I want to set the attribute 'required': True to the email field (it is required and I want to show a translated help_text). The only way I found is overriding the built-in email field, but I loose translation and it do not shows the help_text :
from django.contrib.auth.forms import UserCreationForm
from django.contrib.auth.models import User
from django import forms
class SignInForm(UserCreationForm):
email = forms.EmailField(required=True)
class Meta:
model = User
fields = ('username', 'email', 'password1', 'password2')
If I add a second email2 field, I was hoping copy the original User.email with something like email2 = User.email, but it looks like this is not the good place were email is stored. I can use forms.EmailField() as above, but I'd really like to take benefit of the built-in translation.
Then here is my questions :
How to use an email field with build in attributes (name, required, help_text, …) and get translations ? (When I have an answer I assume that I will found the way to do the same with the submit, first_name & last_name)
Can I duplicate the email field with this constraint? (I do not want to store this second field, just checking before calling form.is_valid())
_
There are a few things at play here.
The UserCreationForm is a ModelForm, which derives its field from a model. In this case, the model is your current AUTH_USER_MODEL setting. When the email form field is created, it inspects the corresponding email field from the model, where it will find the help_text which is wrapped in a gettext (or gettext_lazy) call to allow for i18n to work, as well as any other attribute.
If I understand your question correctly, you want to duplicate the email form field to override some (if any) attributes and to allow you to validate it against the original one. While Django makes it a little harder than what you tried because it is using metaclasses to generate the form class, there has been a public API for doing just that since version 1.8: get_field
Here is an example of what you can do:
class SignInForm(UserCreationForm):
def __init__(self, *args, **kwargs):
super(SignInForm, self).__init__(*args, **kwargs)
email_field = settings.AUTH_USER_MODEL._meta.get_field('email')
self.fields['email2'] = email_field.formfield()
self.fields['email'].required = True
self.fields['email2'].required = True
class Meta:
model = settings.AUTH_USER_MODEL
fields = (...)
A few of remarks:
Using settings.AUTH_USER_MODEL instead of just User will save you from changing the reference to User in your entire codebase should you switch to a custom user model.
get_field also has a plural form (see docs) to retrieve multiple fields at the same time.
In general, any runtime modifications of forms (overriding field attributes, etc.) happens in the __init__ method, since it's where you first get the 'real' form after Django does its black magic with metaclasses, see the required
attribute override.
If you want to override a translatable text, you will have to go through gettext and learn about this mechanism if you want to do it properly. You could of course just hardcode it in French. ;)
Edit: Form validation happens in the validate and validate_{field} form methods, triggered by is_valid. You don't have to have this logic inside the view.
I hope that helps.
I have the following setup for a simple href download page:
urls.py
urlpatterns = [
url(r'^kpis/$', InternalKPIView.as_view(), name='internal_kpis'),
url(r'^tenants/$', TenantListView.as_view(), name='tenant-list'),
url(r'^tenants/(?P<pk>[0-9]+)/$', TenantStatsView.as_view(), name='tenant-stats'),
url(r'^fileformaterror/$', FileFormatErrorView.as_view(), name='file-format-error'),
url(r'^fileformaterror/download/(?P<s3_key>.*)$', FileFormatErrorDownloadView.as_view(), name='file-format-error-download'),
]
template.html:
Download
views.py:
class FileFormatErrorDownloadView(View):
def get(self, request, s3_key):
pass
But when executing I get the following error:
django.urls.exceptions.NoReverseMatch: Reverse for 'file-format-error-download' not found. 'file-format-error-download' is not a valid view function or pattern name.
Tree output of the related files:
$ tree -I "*.pyc|__pycache__"
.
├── apps.py
├── __init__.py
├── migrations
│ └── __init__.py
├── templates
│ └── backoffice
│ ├── file_format_error.html
│ └── internal_kpis.html
├── urls.py
└── views.py
3 directories, 7 files
From what you've provided it seems like the urls.py you are showing belongs to one of the applications within the project. My guess is that URLs of that application are either not included properly or included with a namespace.
why not use django2.0+? then code may as below:
urls.py
path('fileformaterror/download/<s3_key>/', FileFormatErrorDownloadView.as_view(), name='file-format-error-download')
template.html
Download
views.py
from django.shortcuts import HttpResponse
class FileFormatErrorDownloadView(View):
def get(self, request, s3_key):
return HttpResponse('success')
This is my first time creating a project using python and flask. I intend to use SQLAlchemy models along too. and this is a fairly bigger project. As of now, I have divided the project in 2 Blueprints : site and the api. After organizing the project, I am confused as to how can I connnect these models with the database and do I need to re-organize the structure as I am not fully aware of nature of flask.
so this is the directory structure of the dir app/ in my base repository:
`
.
├── Blueprints
│ ├── __init__.py
│ ├── __pycache__
│ │ └── __init__.cpython-36.pyc
│ ├── api
│ │ ├── __init__.py
│ │ ├── __pycache__
│ │ │ ├── __init__.cpython-36.pyc
│ │ │ └── routes.cpython-36.pyc
│ │ └── routes.py
│ ├── config.py
│ └── site
│ ├── __init__.py
│ ├── __pycache__
│ │ ├── __init__.cpython-36.pyc
│ │ └── routes.cpython-36.pyc
│ ├── operations.py
│ ├── routes.py
│ ├── static
│ └── templates
│ ├── about.html
│ ├── contact.html
│ ├── home.html
│ ├── login.html
│ ├── services.html
│ └── stories.html
├── __main__.py
├── __pycache__
│ └── __main__.cpython-36.pyc
└── models
├── Attendance.py
├── Batch.py
├── Course.py
├── Module.py
├── Student.py
├── Test.py
└── __init__.py
`
Please ignore Pycache, as this is auto generated.
now I cannot figure out a way as to how to import and use these models in api and site, neither I can understand as to how am I supposed to fetch the db object created in /Blueprints/__init__.py to all the models.
I understand that this question is not upto the standards of stack overflow questions, BUT I personally feel that organizing a flask project is itself very confusing with each tutorial or forum I see, having their own perspectives of organizing it.
There's several ways to organize a project, but the __init__.py file contained inside the app/ folder is what links a lot of it together. Here's the contents of one of my project's __init__.py file:
from werkzeug.contrib.fixers import ProxyFix
from flask import Flask, session
from app.config import (PERMANENT_SESSION_LIFETIME_MS, Time_Before_Warning,
Min_Ping_Interval)
import datetime
app = Flask(__name__)
app.wsgi_app = ProxyFix(app.wsgi_app)
# Setup the app with the config.py file
app.config.from_pyfile('config.py')
# Setup the logger
from app.logger_setup import logger, log_view
# Setup the database
from flask.ext.sqlalchemy import SQLAlchemy
db = SQLAlchemy(app)
#setup zipcode database
from pyzipcode import ZipCodeDatabase
zdb = ZipCodeDatabase()
# Setup the mail server
from flask.ext.mail import Mail
mail = Mail(app)
# Setup the debug toolbar
#from flask_debugtoolbar import DebugToolbarExtension
#app.config['DEBUG_TB_TEMPLATE_EDITOR_ENABLED'] = False
#app.config['DEBUG_TB_PROFILER_ENABLED'] = False
#toolbar = DebugToolbarExtension(app)
# Setup the password crypting
from flask.ext.bcrypt import Bcrypt
bcrypt = Bcrypt(app)
# Import the views
from app.views import (main, user, error, request, upload, dashboard, org,
msgs, notifications, download, reports,
direct_send,provider,utils)
app.register_blueprint(user.userbp)
app.register_blueprint(request.requestbp)
app.register_blueprint(upload.uploadbp)
app.register_blueprint(dashboard.dashboardbp)
app.register_blueprint(org.orgbp)
app.register_blueprint(msgs.msgbp)
app.register_blueprint(notifications.notificationsbp)
app.register_blueprint(download.downloadbp)
app.register_blueprint(reports.reportsbp)
app.register_blueprint(direct_send.directsendbp)
app.register_blueprint(provider.providerbp)
app.register_blueprint(utils.utilsbp)
# Setup the user login process
from flask.ext.login import LoginManager, current_user
from app.models import User, View
login_manager = LoginManager()
login_manager.init_app(app)
login_manager.login_view = 'userbp.signin'
#login_manager.user_loader
def load_user(email):
return User.query.filter(User.email == email).first()
from flask.ext.principal import identity_loaded, RoleNeed, UserNeed
#identity_loaded.connect_via(app)
def on_identity_loaded(sender, identity):
# Set the identity user object
identity.user = current_user
# Add the UserNeed to the identity
if hasattr(current_user, 'id'):
identity.provides.add(UserNeed(current_user.id))
# Assuming the User model has a list of roles, update the
# identity with the roles that the user provides
if hasattr(current_user, 'roles'):
identity.provides.add(RoleNeed(current_user.roles.type))
from flask.ext.principal import Principal
# load the extension
principals = Principal(app)
# Create a permission with a single Need, in this case a RoleNeed.
#from app import admin
#app.before_request
def make_session_permanent():
session.permanent = True
lt = PERMANENT_SESSION_LIFETIME_MS / (60*1000)
app.permanent_session_lifetime = datetime.timedelta(minutes=lt)
#app.context_processor
def add_session_config():
"""
Add current_app.permanent_session_lifetime converted to milliseconds
to context.
"""
return {
'PERMANENT_SESSION_LIFETIME_MS': PERMANENT_SESSION_LIFETIME_MS,
'Time_Before_Warning': Time_Before_Warning,
'Min_Ping_Interval': Min_Ping_Interval,
}
And then inside one of the blueprints:
from flask import (Blueprint, render_template, redirect, url_for,
abort, flash, request)
from flask.ext.login import login_required, current_user
from app import app, models, db, log_view, config
from app.models import (Groups, Organizations, OrgHasOwner, UserHasGroups,
GroupHasOwner, User, Fax, FavoriteGroups)
from app.forms import org as org_forms
from app.toolbox import email, misc, s3, fax
from sqlalchemy.sql import func
from werkzeug import secure_filename
from uuid import uuid4
import datetime
import string
import os
# Create a user blueprint
orgbp = Blueprint('orgbp', __name__, url_prefix='/org')
#orgbp.route('/invite_user', methods=['GET','POST'])
#login_required
def invite_user():
[stuff goes here]
I would like to access an sqlite3 database from a Flask application (without using Flask-SQLAlchemy, since I require fts4 functionality). I am using Flask blueprints, and I am not sure where to put the following functions (shamelessly copied from a response to this stackoverflow question):
def request_has_connection():
return hasattr(flask.g, 'dbconn')
def get_request_connection():
if not request_has_connection():
flask.g.dbconn = sqlite3.connect(DATABASE)
# Do something to make this connection transactional.
# I'm not familiar enough with SQLite to know what that is.
return flask.g.dbconn
#app.teardown_request
def close_db_connection(ex):
if request_has_connection():
conn = get_request_connection()
# Rollback
# Alternatively, you could automatically commit if ex is None
# and rollback otherwise, but I question the wisdom
# of automatically committing.
conn.close()
My file structure is:
app
├── __init__.py
├── main
│ ├── forms.py
│ ├── __init__.py
│ ├── views.py
├── models.py
├── static
└── templates
├── base.html
├── index.html
└── login.html
I want the request_has_connection() and get_request_connection() functions accessible from all view functions and maybe from models.py as well. Right now, I'm thinking they all belong in my blueprint init.py, which currently contains:
from flask import Blueprint
main = Blueprint('main',__name__)
from . import views
and that my request teardown function would be registered as
#main.teardown_request
def close_db_connection(ex):
<blah-blah-blah>
Is this right?