I'm fair new to Django and I am trying to log visitor for my blog. I'm using generic view for my blog and here is part of code:
#blog/urls.py
urlpatterns = patterns('',
#index
url(r'^(?P<page>\d+)?/?$', PostListView.as_view(
model=Post,
paginate_by=3,
)),
#individual post
url(r'^(?P<pub_date__year>\d{4})/(?P<pub_date__month>\d{1,2})/(?P<slug>[a-zA-Z0-9-]+)/?$',
DetailView.as_view(model=Post,)),
#cat
url(r'^category/(?P<slug>[a-zA-Z0-9]+)/?$', CategoryListView.as_view(
paginate_by=3,
model=Category,
)),
#tag
url(r'^tag/(?P<slug>[a-zA-Z0-9]+)/?$', TagListView.as_view(
paginate_by=3,
model=Tag,
)),
and I wrote a simple model for visitor log:
#tasks/models.py
class Visitor(models.Model):
visit_stamp = models.DateTimeField(auto_now_add=True)
referer = models.CharField(max_length=100, blank=True)
ip = models.IPAddressField(blank=True)
user_agent = models.CharField(max_length=100, blank=True)
page = models.CharField(max_length=100)
and its view:
#tasks/views.py
def log(request, page):
try:
hit = Visitor()
hit.page = page
hit.ip = request.META.get('REMOTE ADDR', '')
hit.last_visit = datetime.now()
hit.referer = request.META.get('HTTP REFERER', '')
hit.user_agent = request.META.get('HTTP_USER_AGENT', '')
hit.save()
except IntegrityError:
pass
def tracking(request, page):
log(request, page)
return render_to_response(page)
My question is how and where can I call this methods so that I can log a user is visiting a specific page. I'd appreciate any advices.
First off, I assume you don't have access to the apache (or whatever host is running your django app) logs and/or you want to eventually add other things and/or you want it available in the database, as otherwise, you can skip a lot of work and just grep the logs.
Anyways, I'd recommend rewriting track to work as a decorator (and adjust log as you need it... note that I believe you can get the URL from the request object versus passing it in as a page value in case you want to know which specific instance was visited). There are also ways you could probably do this with middleware, but this gives you a pretty good mix of simplicity and ability to control which views get logged.
To borrow an example from http://www.djangofoo.com/253/writing-django-decorators
def track(page):
def decorator(func):
def inner_decorator(request, *args, **kwargs):
log(request, page)
return func(request, *args, **kwargs)
return wraps(func)(inner_decorator)
return decorator
And then in your urls (or you can also do #track to decorate function based views)
url(r'^(?P<page>\d+)?/?$', track("index")(PostListView.as_view(
model=Post,
paginate_by=3,
))),
url(r'^someregexp$', track("pagename")(SomeListView.as_view(
model=Post,
paginate_by=3,
))),
Edit: meant to add. Do note that in general, GET requests are supposed to be idempotent; logging is a gray area but the main thing to keep in mind is that some requests may not get logged as you might expect if the page is cached (for Posts this shouldn't be an issue)
Related
I'm currently debugging a weird problem with a Django site where one specific model is triggering a 404 error when creating a new instance or editing an existing one in the admin interface.
Specifically, the error occurs when the form is submitted. I can GET the changeform just fine.
This is only occuring on the live site and only when saving this model. All other models behave as expected, and when I run it locally, everything works as expected. When created programatically, everything is also fine both live and locally.
Here's my model:
class Content(models.Model):
"""Base Content class."""
title = models.CharField(max_length=200)
body = RichTextUploadingField(max_length=30000, blank=True, null=True, config_name='admin')
date_created = models.DateTimeField(auto_now_add=True)
date_updated = models.DateTimeField(auto_now=True)
author = models.ForeignKey(to=User, on_delete=models.CASCADE)
slug = models.SlugField(max_length=100, null=True, default=None)
class Meta:
abstract = True
class ContentPage(Content):
"""Represents a page of generic text content."""
title = models.CharField(max_length=200, unique=True)
has_url = models.BooleanField(default=False, help_text='Sets the page to accessible via a URL.')
banner = models.ImageField(upload_to='myfiles/banners/', blank=True, null=True)
def save(self, *args, **kwargs):
"""Create the slug from the title."""
self.slug = slugify(self.title[:100])
super(ContentPage, self).save(*args, **kwargs)
The ContentPage class is the one triggering the problem in the admin interface. My other class that inherits from Content works fine.
I have stripped back my admin setup to the following and it is still occuring:
class CustomAdminSite(AdminSite):
def get_urls(self):
"""Define custom admin URLs."""
urls = super(CustomAdminSite, self).get_urls()
# Append some new views here...
return urls
admin_site = CustomAdminSite()
admin_site.register(ContentPage)
Here's a basic URL config that reproduces the problem:
from myapp.admin import admin_site
urlpatterns = [
path('mycooladminsite/', admin_site.urls),
path('', include('myapp.urls')),
]
A few other things I've checked:
I have no signals interferring with the save.
I am not performing any actions in the model's save() method.
I have checked for get_object_or_404() calls and can't see any that would affect this.
I've spent a few hours digging through this and I'm currently at a brick wall.
The database engine is mysql.connector.django, with Django version 2.2.11. I can't change the engine or update Django to 3.x yet for this site.
This is a recent problem that I did not notice previously.
Update:
I've narrowed down the problem to the ImageField. When I remove that from the fields displayed, the problem does not occur on save.
I'm using a custom admin form but that doesn't seem to be the problem. I've tried using the default and it still occurs. I've been looking for exceptions in the storage class but haven't found any. I stripped it all the way back and the error remains.
I've examined the local 302 POST and production 404 POST in Firefox Developer Tools and they're virtually identical but production server is Apache and x-powered-by is Phusion Passenger, whereas locally the server is WSGIServer/0.2 CPython/3.7.3.
I've actually noticed a 404 occurring with other multipart/form-data forms in production now, which I never got before.
I don't know what the issue is but if you don't have logs, run it on prod with DEBUG=False, don't use Sentry etc. and it happens only when you save ContentPage admin form, then you can overwrite save_new or save_model method in your ModelAdmin with
def save_model(self, request, obj, form, change):
try:
obj.save()
except Exception:
logging.exception("blah")
(or save_new accordingly)
and check logs.
Or just use Sentry. They have a free tier
I never fully got to the bottom of what was going on, but I did devise a workaround to at least allow the forms to function correctly, as I discussed in my answer to a related question:
Edit _get_response(self, request) in django.core.handlers.base. Change resolver_match = resolver.resolve(request.path_info) to
resolver_match = resolver.resolve(request.path).
Add this middleware (adjust to your exact needs):
class ImageField404Middleware:
def __init__(self, get_response):
self.get_response = get_response
def __call__(self, request):
response = self.get_response(request)
if (request.method == 'POST' and request.user.is_superuser and response.status_code == 302
and request.get_full_path().startswith('/pathtoadmin/')):
post_messages = get_messages(request)
for message in post_messages:
if ('was added successfully' in message.message or 'was changed successfully' in message.message
and message.level == message_levels.SUCCESS):
messages.success(request, message.message)
redirect_url = request.get_full_path()
if '_addanother' in request.POST:
redirect_url = re.sub(r'[^/]*/[^/]*/$', 'add/', redirect_url)
elif '_save' in request.POST:
redirect_url = re.sub(r'[^/]*/[^/]*/(\?.*)?$', '', redirect_url)
if '_changelist_filters' in request.GET:
preserved_filters = parse.parse_qsl(request.GET['_changelist_filters'])
redirect_url += '?' + parse.urlencode(preserved_filters)
elif '_continue' in request.POST:
redirect_url_search = re.search(r'((?<=href=)[^>]*)', message.message)
if redirect_url_search:
redirect_url = redirect_url_search.group(0)
redirect_url = re.sub(r'[\\"]*', '', redirect_url).replace('/pathtoadmin/pathtoadmin/', '/pathtoadmin/')
return HttpResponseRedirect(redirect_url)
return response
Not ideal by any means, but works for me at the moment.
You can read more details here:
https://medium.com/#mnydigital/how-to-resolve-django-admin-404-post-error-966ce0dcd39d
I believe these problems may, in part at least, be caused by ModSecurity Apache. Some problems I had went away after the hosting provider later reconfigured this.
I have a Django/Wagtail/Puput site with this structure:
RootPage
|
|- BlogPage (Puput)
|- InformationPage
I'm trying to display summary info from the Puput blog on the InformationPage. This works with this code, as long as I only have one BlogPage:
class InformationPage(Page):
body = RichTextField(verbose_name=_("body"))
. . .
def get_context(self, request, *args, **kwargs):
context = super(InformationPage, self).get_context(
request, *args, **kwargs)
context['blog_page'] = BlogPage.objects.first()
context['information_page'] = self
return context
But I'm trying to make it work with more than one blog page. It seems like this should work:
class InformationPage(Page):
body = RichTextField(verbose_name=_("body"))
blog_page = models.ForeignKey('wagtailcore.Page', on_delete=models.PROTECT, related_name="information_blog")
content_panels = [
MultiFieldPanel(
[
FieldPanel("title", classname="title"),
FieldPanel("body", classname="full"),
PageChooserPanel('blog_page'),
],
heading=_("Content"),
)]
def get_context(self, request, *args, **kwargs):
context = super(InformationPage, self).get_context(
request, *args, **kwargs)
context['blog_page'] = self.blog_page
context['information_page'] = self
return context
But it doesn't. This was suggested by #gasman here. In other words, if I refer to the blog page properties using context['blog_page'] = BlogPage.objects.first(), everything works fine, but switching it out to use context['blog_page'] = self.blog_page (and selecting the correct blog page in the admin) does not work.
Without switching it out, I think I can only ever have a single instance of BlogPage, because all InformationPages will have to pull from the first instance.
Any thoughts?
You haven't given a description of the problem beyond "it doesn't work", so I'm only guessing here, but you're presumably trying to output fields of the blog page that are part of the BlogPage model. This doesn't work because blog_page is defined as a foreign key to wagtailcore.Page, and so accessing self.blog_page only gives you a basic Page object consisting of the core fields such as title. You can retrieve the full BlogPage object by accessing self.blog_page.specific:
context['blog_page'] = self.blog_page.specific
However, a smarter approach is to change your foreign key to point to BlogPage, since presumably choosing any other page type is not valid here:
blog_page = models.ForeignKey('my_blog_app.BlogPage', on_delete=models.PROTECT, related_name="information_blog")
With this change, self.blog_page will return a BlogPage instance directly, and there's no need for .specific.
I'm using Django Rest Framework to serve an API. I've got a couple tests which work great. To do a post the user needs to be logged in and I also do some checks for the detail view for a logged in user. I do this as follows:
class DeviceTestCase(APITestCase):
USERNAME = "username"
EMAIL = 'a#b.com'
PASSWORD = "password"
def setUp(self):
self.sa_group, _ = Group.objects.get_or_create(name=settings.KEYCLOAK_SA_WRITE_PERMISSION_NAME)
self.authorized_user = User.objects.create_user(self.USERNAME, self.EMAIL, self.PASSWORD)
self.sa_group.user_set.add(self.authorized_user)
def test_post(self):
device = DeviceFactory.build()
url = reverse('device-list')
self.client.force_login(self.authorized_user)
response = self.client.post(url, data={'some': 'test', 'data': 'here'}, format='json')
self.client.logout()
self.assertEqual(status.HTTP_201_CREATED, response.status_code)
# And some more tests here
def test_detail_logged_in(self):
device = DeviceFactory.create()
url = reverse('device-detail', kwargs={'pk': device.pk})
self.client.force_login(self.authorized_user)
response = self.client.get(url)
self.client.logout()
self.assertEqual(status.HTTP_200_OK, response.status_code, 'Wrong response code for {}'.format(url))
# And some more tests here
The first test works great. It posts the new record and all checks pass. The second test fails though. It gives an error saying
AssertionError: 200 != 302 : Wrong response code for /sa/devices/1/
It turns out the list view redirects the user to the login screen. Why does the first test log the user in perfectly, but does the second test redirect the user to the login screen? Am I missing something?
Here is the view:
class APIAuthGroup(InAuthGroup):
"""
A permission to allow all GETS, but only allow a POST if a user is logged in,
and is a member of the slimme apparaten role inside keycloak.
"""
allowed_group_names = [settings.KEYCLOAK_SA_WRITE_PERMISSION_NAME]
def has_permission(self, request, view):
return request.method in SAFE_METHODS \
or super(APIAuthGroup, self).has_permission(request, view)
class DevicesViewSet(DatapuntViewSetWritable):
"""
A view that will return the devices and makes it possible to post new ones
"""
queryset = Device.objects.all().order_by('id')
serializer_class = DeviceSerializer
serializer_detail_class = DeviceSerializer
http_method_names = ['post', 'list', 'get']
permission_classes = [APIAuthGroup]
Here is why you are getting this error.
Dependent Libraries
I did some searching by Class Names to find which libraries you were using so that I can re-create the problem on my machine. The library causing the problem is the one called keycloak_idc. This library installs another library mozilla_django_oidc which would turn out to be the reason you are getting this.
Why This Library Is Causing The Problem
Inside the README file of this library, it gives you instructions on how to set it up. These are found in this file. Inside these instructions, it instructed you to add the AUTHENTICATION_BACKENDS
AUTHENTICATION_BACKENDS = [
'keycloak_oidc.auth.OIDCAuthenticationBackend',
...
]
When you add this authentication backend, all your requests pass through a Middleware defined inside the SessionRefresh class defined inside mozilla_django_oidc/middleware.py. Inside this class, the method process_request() is always called.
The first thing this method does is call the is_refreshable_url() method which always returns False if the request method was POST. Otherwise (when the request method is GET), it will return True.
Now the body of this if condition was as follows.
if not self.is_refreshable_url(request):
LOGGER.debug('request is not refreshable')
return
# lots of stuff in here
return HttpResponseRedirect(redirect_url)
Since this is a middleware, if the request was POST and the return was None, Django would just proceed with actually doing your request. However when the request is GET and the line return HttpResponseRedirect(redirect_url) is triggered instead, Django will not even proceed with calling your view and will return the 302 response immediately.
The Solution
After a couple of hours debugging this, I do not the exact logic behind this middleware or what exactly are you trying to do to provide a concrete solution since this all started based off guess-work but a naive fix can be that you remove the AUTHENTICATION_BACKENDS from your settings file. While I feel that this is not acceptable, maybe you can try using another library that accomplishes what you're trying to do or find an alternative way to do it. Also, maybe you can contact the author and see what they think.
So i guess you have tested this and you get still the same result:
class APIAuthGroup(InAuthGroup):
def has_permission(self, request, view):
return True
Why do you use DeviceFactory.build() in the first test and DeviceFactory.create() in the second?
Maybe a merge of the two can help you:
def test_get(self):
device = DeviceFactory.build()
url = reverse('device-list')
response = self.client.get(url)
self.assertEqual(status.HTTP_200_OK, response.status_code)
Is this a problem with the setUp() method? From what I see, you may be setting self.authorize_user to a user that was already created on the first test.
Instead, I would create the user on each test, making sure that the user doesn't exist already, like so:
user_exists = User.objects.filter(username=self.USERNAME, email=self.EMAIL).exists()
if not user_exists:
self.authorize_user = User.objects.create_user....
That would explain why your first test did pass, why your second didn't, and why #anupam-chaplot's answer didn't reproduce the error.
Your reasoning and code looks ok.
However you are not giving the full code, there must be error you are not seeing.
Suspicious fact
It isn't be default 302 when you are not logged in.
(#login_required, etc redirects but your code doesn't have it)
Your APIAuthGroup permission does allow GET requests for non-logged-in user ( return request.method in SAFE_METHODS), and you are using GET requests (self.client.get(url))
So it means you are not hitting the endpoint that you think you are hitting (your get request is not hitting the DevicesViewSet method)
Or it could be the case you have some global permission / redirect related setting in your settings.py which could be DRF related..
eg :
REST_FRAMEWORK = {
'DEFAULT_PERMISSION_CLASSES': [
'rest_framework.permissions.IsAuthenticated',
]
}
Guess
url = reverse('device-detail', kwargs={'pk': device.pk})
might not point to the url you are thinking..
maybe there's another url (/sa/devices/1/) that overrides the viewset's url. (You might have a django view based url)
And I didn't address why you are getting redirected after force_login.
If it's indeed login related redirect, all I can think of is self.authorized_user.refresh_from_db() or refreshing the request ..
I guess some loggin related property (such as session, or request.user) might point to old instance .. (I have no evidence or fact this can happen, but just a hunch) and you better off not logging out/in for every test case)
You should make a seperate settings file for testing and add to the test command --settings=project_name.test_settings, that's how I was told to do.
I am working on a small site using Wagtail. This site is all about a "mainpage" and several "subpages". So far it is pretty simple! But, depending on what group the user (not admin) is in, the right subpages should show up!
See the following setup (minimized), to get an idea of what I am talking about.
If I set permissions on ToolKitPart (like requiring explicit user-login and group-membership), then the following is happening:
when going to the page using the fully qualified path, the user is requested to login and, in the case of insufficient rights, the user will not see the content!
when going to the ToolkitIndex-Page, all children are displayed, including the ones the user never should see, without the need to be logged in or being a member of a certain group.
class ToolkitIndex(Page):
def get_context(self, request):
# Update context to include only published posts, ordered by reverse-chron
context = super().get_context(request)
blogpages = self.get_children().live().order_by('-first_published_at')
context['pages'] = blogpages
return context
class ToolkitPart(Page):
body = StreamField([
('staff', MpStaff()),
('news', MpNews()),
('stuff', MpStuff()),
('teditor', blocks.RichTextBlock()),
('reditor', blocks.RawHTMLBlock()),
], blank=True)
content_panels = Page.content_panels + [
StreamFieldPanel('body'),
]
class MpNews(blocks.StructBlock):
head = blocks.TextBlock(required=True, help_text='Schlagzeile')
lead = blocks.TextBlock(required=False, help_text='Einleitung')
body = blocks.RichTextBlock(required=True, help_text='Inhalt')
image = ImageChooserBlock(required=False)
type = blocks.ChoiceBlock(
choices=[('default', 'Standard'),
('highlight', 'Hervorgehoben'),
], required=True)
class Meta:
template = 'home/mparts/toolkit_news.html'
icon = 'code'
Any idea how to solve this?
Assuming you've set these permissions up using Wagtail's private pages feature, these are stored in the PageViewRestriction model. Unfortunately Wagtail doesn't currently provide a way to apply these permission checks against anything other than the current page request, so you'd have to recreate this logic yourself to filter a queryset to the user's view permissions. This would be something like (untested):
from django.db.models import Q
class ToolkitIndex(Page):
def get_context(self, request):
context = super().get_context(request)
blogpages = self.get_children().live().order_by('-first_published_at')
if not request.user.is_authenticated:
blogpages = blogpages.public() # only pages with no view restrictions at all
else:
blogpages = blogpages.filter(
# pages with no view restrictions
Q(view_restrictions__isnull=True)
# pages restricted to any logged-in user
| Q(view_restrictions__restriction_type='login')
# pages restricted by group
| Q(view_restrictions__restriction_type='groups', view_restrictions__groups__in=request.user.groups.all())
)
Disclaimers:
This doesn't account for pages that are protected by a shared password
To be fully correct, we'd need to account for the fact that view restrictions propagate down the tree (and so a subpage may still be restricted even if it doesn't have a view restriction record directly attached to it); however, we're only looking at immediate children of the current page (which they evidently do have access to...) so that issue doesn't come up here.
PageViewRestriction is not a public Wagtail API and may change in future releases - in particular, see RFC 32 for a proposed change that may happen in the fairly near future.
I'm making an admin panel for a Django-Mptt tree structure using the FeinCMS TreeEditor interface. This interface provides an 'actions column' per-node for things like adding or moving nodes quickly without using the typical Django admin action select box.
What I am trying to do is add a custom admin action to this collection which passes the pk of the node to a celery task which will then add a collection of nodes as children. Existing functions are simply href links to the URL for that task(add/delete/move), so thus far I have simply mimicked this.
My solution currently involves:
Define the action as a function on the model
Create a view which uses this function and redirects back to the changelist
Add this view to the admin URLs
Super the TreeEditor actions column into the ModelAdmin class
Add an action to the collection which calls this URL
Surely there must be a better method than this? It works, but it feels massively convoluted and un-DRY, and I'm sure it'll break in odd ways.
Unfortunately I'm only a month or two into working with Django so there's probably some obvious functions I could be using. I suspect that I might be able to do something with get_urls() and defining the function directly in the ModelAdmin, or use a codeblock within the injected HTML to call the function directly, though I'm not sure how and whether it's considered a better option.
Code:
I've renamed everything to a simpler library <> books example to remove the unrelated functionality from the above example image.
models.py
class Library(models.Model):
def get_books(self):
# Celery task; file omitted for brevity
get_books_in_library.delay(self.pk)
views.py
def get_books_in_library(request, library_id):
this_library = Library.objects.get(pk=library_id)
this_library.get_books_in_library()
messages.add_message(request, messages.SUCCESS, 'Library "{0}" books requested.'.format(this_library.name))
redirect_url = urlresolvers.reverse('admin:myapp_library_changelist')
return HttpResponseRedirect(redirect_url)
urls.py
urlpatterns = [
url(r'^admin/myapp/library/(?P<library_id>[0-9]+)/get_books/$', get_books_in_library, name='get books in library'),
url(r'^admin/', include(admin.site.urls)),
]
admin.py
class LibraryAdmin(TreeEditor):
model = Library
def _actions_column(self, obj):
actions = super(LibraryAdmin, self)._actions_column(obj)
actions.insert(
0, u'<a title="{0}" href="{1}/get_books"><img src="{2}admin/img/icon_addlink.gif" alt="{0}" /></a>'.format(
_('Get Books'),
obj.pk,
settings.STATIC_URL
)
)
return actions
Note that I may have broken something in renaming things and removing the extraneous cruft if you try to execute this code, I think it should adequately illustrate what I'm trying to do here however.
After digging around today and simply trying various other solutions, I've put together one that uses get_urls and a view defined directly into the admin interface which feels tidier though it's effectively just moving the code from multiple django files into the admin interface - though it does make use of the admin wrapper to stop unauthenticated users, which is an improvement.
I'll leave a copy of the working code here for anyone who finds this in future, as I've seen very few examples of TreeEditor et al. being used in newer versions of Django.
class NodeAdmin(TreeEditor):
model = Node
# < ... > Other details removed for brevity
def get_urls(self):
urls = super(NodeAdmin, self).get_urls()
my_urls = [
url(r'^(?P<node_id>[0-9]+)/get_suggestions/$', self.admin_site.admin_view(self.get_suggestions)),
]
return my_urls + urls
def get_suggestions(self, request, node_id):
this_node = Node.objects.get(pk=node_id)
get_suggestions(this_node.pk)
messages.add_message(request, messages.SUCCESS, 'Requested suggestions for {0}'.format(this_node.term))
redirect_url = urlresolvers.reverse('admin:trinket_node_changelist')
return HttpResponseRedirect(redirect_url)
def _actions_column(self, obj):
actions = super(NodeAdmin, self)._actions_column(obj)
# Adds an 'get suggestions' action to the Node editor using a search icon
actions.insert(
0, u'<a title="{0}" href="{1}/get_suggestions"><img src="{2}admin/img/selector-search.gif" alt="{0}" /></a>'.format(
_('Get Suggestions'),
obj.pk,
settings.STATIC_URL,
)
)
# Adds an 'add child' action to the Node editor using a plus icon
actions.insert(
0, u'<a title="{0}" href="add/?{1}={2}"><img src="{3}admin/img/icon_addlink.gif" alt="{0}" /></a>'.format(
_('Add child'),
getattr(self.model._meta,'parent_attr', 'parent'),
obj.pk,
settings.STATIC_URL
)
)
return actions