Python37 and Django 2 TextField not a string? - python

We decided to migrate our django 1.10.5 project with python 2.7.15 to a newer version of python and django. Now we are using python 3.7 and django 2. After some problems with our email sending script I found something strange. When sending automated mails we took text from a TextField in the database and inserted it into our Mail.
content = content.replace('##ENTRY##', entry.text)
The text field of entry is a django models.TextField.
Now with python 37 it doesn't allow me to use it like that. I have to wrap the entry.text into a str() cast. But shouldn't the TextField be a string?
content = content.replace('##ENTRY##', str(entry.text))
With that it works, but it lets my tummy hurt when doing it like that, since i don't understand why.
Edit:
def send_task_entry_new(entry):
content = Email.load_email_template('tasks.watcher.info.entry.new')
content = content.replace('##TAG##', entry.task.get_tag())
content = content.replace('##ENTRY##', entry.text)
content = content.replace('##SUBJECT##', entry.task.get_subject())
subject = entry.task.get_tag() + " " + entry.task.get_subject()
for watcher in entry.task.taskwatcher_set.all():
if watcher.user.last_login:
content = content.replace('##LINK##', settings.BASE_URL + entry.task.get_absolute_url())
# If User has never logged in (Dummy for external task user)
else:
content = content.replace('##LINK##', settings.BASE_URL + reverse('tasks_public', args=[watcher.token]))
Email.send(watcher.user.email, '', '', subject, content)
Entry is an TaskEntry object which gets created as soon as I create a Task. The TaskEntry model looks like this. At least the attributes of a TaskEntry:
task = models.ForeignKey(Task, null=True, on_delete=models.SET_NULL)
creator = models.ForeignKey(User, on_delete=models.SET_NULL, null=True, blank=True)
external = models.EmailField(null=True, blank=True)
text = models.TextField(verbose_name="Text")
date_created = models.DateTimeField(auto_now_add=True)
Edit Flow:
Email comes in and get processed:
text = get_decoded_email_body(response_part[1])
which fires:
def get_decoded_email_body(message_body):
msg = email.message_from_bytes(message_body)
if msg.is_multipart():
# Walk through all parts of this email
for part in msg.walk():
# print "%s => %s" % (part.get_content_type(), part.get_content_charset())
charset = part.get_content_charset()
#TODO umstellung auf Python3 enfernen und testen
if part.get_payload(decode=True):
if part.get_content_type() == 'text/plain':
if charset:
return str(part.get_payload(decode=True), charset, 'ignore')
else:
return to_unicode(part.get_payload(decode=True))
if part.get_content_type() == 'text/html':
if charset:
return str(part.get_payload(decode=True), charset, 'ignore')
else:
return to_unicode(part.get_payload(decode=True))
return "Email has no text/plain or text/html part"
else:
return to_unicode(msg.get_payload(decode=True))
After that we got our text extracted and create a now task:
user = User.objects.get_or_create(email=sender)[0]
task = Task.objects.create(subject=subject, creator=user)
task.create_entry(text, user) #Here it crashes
Email.send_task_operators_new_task(task, text, sender)
Now into the models.py and we go into create_entry:
def create_entry(self, text, creator):
entry = TaskEntry()
entry.text = text
entry.task = self
entry.creator = creator
entry.save()
if creator:
TaskWatcher.objects.get_or_create(task=self, user=creator)
Next step is the .save() method:
def save(self, *args, **kwargs):
if self.pk == None:
created = True
else:
created = False
super(TaskEntry, self).save(*args, **kwargs)
if created:
# If this entry is the first one inform creator and owner if given
if self.task.taskentry_set.all().count() == 1:
Email.send_task_created_to_creator(self.task)
# If a owner has been given how is not the creator
if self.task.owner and self.task.owner != self.task.creator:
Email.send_task_created_to_owner(self.task)
# This is not the first entry, so inform all watcher about the new entry
if self.task.taskentry_set.all().count() > 1:
Email.send_task_entry_new(self)
Now it executes the Email.send_task_entry_new(self) and throws the replace() error.

Thanks to Daniel Roseman I found the issue. Since I take the payload from the mail as bytes and never convert it into a string, it is saved as bytes into the db. As soon as I request the data from the db it is bytes and I have to convert them into string as I need it.

Related

How to integrate python script in a django project

So I am extremely new to programming, and am stuck at this issue, I am using python with Django and Mongodb for database. I need to write a service that assigns an ID (not the one assigned by mongodb) upon each user form submission. for example entry 1's ID will be [Prefix entered by user] 2101, entry 2's ID will be [Prefix entered by user] 2102, so its basically adding in the number 2100.
I have no idea how and where to integrate this logic in my code. I have tried a few solutions on the internet but nothing seems to work.
my code:
Model.py
class Writeups(Document):
blog_id = 2100
title = fields.StringField(max_length=120)
date_created = fields.DateField(blank=True, null=True)
date_modified = fields.DateField(blank=True, null=True)
version_number = fields.DecimalField(null= True , max_digits=1000, decimal_places=2)
storage_path = fields.StringField(max_length=120)
STRIKE_READY_BRIEF = 'SRB'
STRIKE_READY_THREAT_REPORT = 'SRTR'
PREFIX_CHOICES = [
(STRIKE_READY_BRIEF, 'SRB'),
(STRIKE_READY_THREAT_REPORT, 'SRTR'),
]
prefix = fields.StringField(
max_length=4,
choices=PREFIX_CHOICES,
null=False,
blank=False,
)
views.py:
#csrf_exempt
def writeups_request(request):
"""
Writeup Request
"""
if request.method == 'GET':
try:
data = {
'form-TOTAL_FORMS': '1',
'form-INITIAL_FORMS': '0',
}
writeups = WriteupsFormset(data)
# print(writeup)
return render(request, "writeups/writeups.html", {'writeups_forms': writeups})
except Exception as e:
print(e)
response = {"error": "Error occurred"}
return JsonResponse(response, safe=False)
if request.method == 'POST':
writeup_data = WriteupsFormset(request.POST)
if writeup_data.is_valid():
flag = False
logs = []
for writeups_data in writeup_data:
print(writeups_data)
if writeups_data.cleaned_data.get('DELETE'): # and malware_data._should_delete_form(form):
continue
title = writeups_data.cleaned_data.get('title')
date_created = writeups_data.cleaned_data.get('date_created')
date_modified = writeups_data.cleaned_data.get('date_modified')
version_number = writeups_data.cleaned_data.get('version_number')
storage_path = writeups_data.cleaned_data.get('storage_path')
prefix = writeups_data.cleaned_data.get('prefix')
try:
writeups = Writeups(),
title=title,
date_created=date_created,
date_modified=date_modified,
version_number=version_number,
storage_path=storage_path,
prefix=prefix)
writeups.save()
In order to implement your custom script at all submissions, Django's Signals are the solution for your use case. Look into post_save and pre-save signals and use them as per your problem.
https://docs.djangoproject.com/en/3.2/ref/signals/
If you have an existing database and you want a script to iterate through it and update the dataset, you can take a look at Management Commands
https://docs.djangoproject.com/en/4.0/howto/custom-management-commands/

How do I assign specific users to a user-uploaded file so they can modify it/delete it (Django + Apache)

Im using django 1.10 + Apache in Linux.
I've created a small webapp to upload documents (with dropzone.js) and want to implement the ability for a user to specify who can view/modify/delete a specific file but i can't figure out a way how. I attempted using a ManyToManyField but maybe im not understading the Field itself correctly.
The "Document" model is this:
Model
class Document(models.Model):
file = models.FileField(upload_to = 'files/')
#validators=[validate_file_type])
uploaded_at = models.DateTimeField(auto_now_add = True)
extension = models.CharField(max_length = 30, blank = True)
thumbnail = models.ImageField(blank = True, null = True)
is_public = models.BooleanField(default = False)
accesible_by = models.ManyToManyField(User) #This is my attempt at doing this task.
def clean(self):
self.extension = self.file.name.split('/')[-1].split('.')[-1]
if self.extension == 'xlsx' or self.extension == 'xls':
self.thumbnail = 'xlsx.png'
elif self.extension == 'pptx' or self.extension == 'ppt':
self.thumbnail = 'pptx.png'
elif self.extension == 'docx' or self.extension == 'doc':
self.thumbnail = 'docx.png'
def delete(self, *args, **kwargs):
#delete file from /media/files
self.file.delete(save = False)
#call parent delete method.
super().delete(*args, **kwargs)
#Redirect to file list page.
def get_absolute_url(self):
return reverse('dashby-files:files')
def __str__(self):
return self.file.name.split('/')[-1]
class Meta():
ordering = ['-uploaded_at']
My View to handle the creation of documents:
View
class DocumentCreate(CreateView):
model = Document
fields = ['file', 'is_public']
def form_valid(self, form):
self.object = form.save(commit = False)
## I guess here i would Add the (self.request.user) to the accesible_by Field.
self.object.save()
data = {'status': 'success'}
response = JSONResponse(data, mimetype =
response_mimetype(self.request))
return response
Thanks in advance to anyone for any ideas or suggestions...
You have a model and a view that hopefully works for adding new documents, you still have a number of steps to go.
You'll need a place to assign users that can view/modify/delete your files. If you need to store access levels (view/delete...), your accessible_by will not suffice and you'll do well with a through table to add more information like access level.
You need to write views for various actions like view, delete... that users will request and here you ensure users have the right privileges. An implementation would be to get the request.user and the document id, look up if the user has the permission for what she's doing, return an http unauthorized exception or allow the action to proceed.
Edit: My question is about how can I assign user-permissions to each
individual file
If we're keeping this to access control from the django level, using the document model you already have, and you've taken some steps and for every document, you can assign users (accessible_by). Something like this can get you started:
from django.core.exceptions import PermissionDenied
def view_document(request, doc_pk):
doc = get_object_or_404(Document, pk=doc_pk)
if not doc.accessible_by.filter(username=request.user.username):
raise PermissionDenied
#perform rest of action
Or do you mean to use the permissions framework itself?

Django: Transferring/Accessing forms full error message

On a website that I'm working on for my school, the user enters their school email and password, and if they have registered they log in. If not, a second part of the log in is revealed asking for a pen name and to confirm the password. Because of this, and my convoluted amateur Django programming, I have a list of errors named er. For instants, when the program tests whether the email is a school one, it might add to the er list "school email only". I am also using two form classes as well. The page uses ajax to call this function, which uses plain html instead of JSON because of the sites small size.
In the forms.py file I have:
class log_in(forms.Form):
username = forms.EmailField(error_messages= {'required': "Email Field is required,", 'invalid' : "Invalid Email Address."})
password = forms.CharField(help_text = 'Password Invalid')
class new_user(forms.Form):
username = forms.EmailField(error_messages = {'required': "Email Field is required,", 'invalid' : "Invalid Email Address."})
password = forms.CharField(required=True)
password2 = forms.CharField(required=True)
pen_name = forms.CharField(max_length=30, min_length=3, error_messages = {'required': "Pen Name is required", 'max_length': "Pen Name must be less than 30 characters", 'min_length': "Pen Name must be more than 3 characters"})
The problem is that I want to transfer the full error message that I specified in the error_message argument to the er list.
This is my views.py file
def user_log_in(request):
er = []
user_pass = log_in(request.POST)
if user_pass.is_valid(): # If it is valid at all
cleaned_info = user_pass.cleaned_data
email_bbn = cleaned_info['username'].split("#")
if 'bbns.org' in email_bbn: # Check if BBN email address
user_object = User.objects.filter(email = cleaned_info['username'])
if user_object.exists():
logged_in_user = auth.authenticate(username=cleaned_info['username'], password=cleaned_info['password'])
#add in is_active
if logged_in_user is not None: #If password is right
if user_object[0].get_profile().activated:
auth.login(request, logged_in_user)
return HttpResponseRedirect("")
else:
return HttpResponse("not_act")
else:
er.append("Incorrect Password")
else: # If new user
new_user_pass = new_user(request.POST)
if new_user_pass.is_valid():
cleaned_info_new = new_user_pass.cleaned_data
if cleaned_info_new['password'] == cleaned_info_new['password2']:
msg = "In order to activate your account at Knights of the Round Table, please click on this link:"
try:
send_mail('Activate', msg, 'michaelrgoldfine#gmail.com', [cleaned_info_new['username']], fail_silently=False)
new_user_object = User.objects.create_user(
username=cleaned_info_new['username'],
password=cleaned_info_new['password'],
email=cleaned_info_new['username']
)
new_user_profile = new_user_object.get_profile()
new_user_profile.pen_name = cleaned_info_new['pen_name']
new_user_profile.activated = False;
new_user_profile.save()
return HttpResponse("not_act")
except:
er.append("Error Sending Email")
else:
er.append('Passwords are not the same')
elif "TN" in request.POST: #If open but not filled in
print "TN"
er.append(new_user_pass.pen_name.error_messages)
else: # if new user field
print "n_usr"
return HttpResponse('n_usr')
else:
er.append("BBN email addresses only")
else:
for e in user_pass.errors:
er.append(e)
errors_template = Template("{% for e in errors %}<li>{{ e }}</li> {% endfor %}")
errors_html = errors_template.render(Context({'errors':er}))
return HttpResponse(errors_html)
I try to accsess the errors twice. Once, on the else you see right at the end with a for loop, and two elses up from that on elif 'TN'... The last one just returns the field thats invalid (so i get user_name or pen_name). The other one says that the form has no object pen_name or whatever I use it for.
It would be better to add errors to the actual form. Forms have an _errors dict attached to them that contain all the errors generated by the form. "Non-field errors" (errors that don't directly relate to a particular field or that relate to multiple fields) go in form._errors['__all__']. All field-specific errors go into the key of the field's name. So, errors for a foo field would go in form._errors['foo'].
Now, the list of errors for each item in the _errors dict is actually an ErrorList type, not a standard list. So, to add errors to the form you do:
from django.forms.util import ErrorList
form._errors.setdefault('foo', ErrorList()).append('Some error here')
Or, to add the error to non-field errors:
form._errors.setdefault('__all__', ErrorList()).append('Some error here')
Then, when your form renders, the errors will all fall naturally where they should, just like any normal validation error.
The array probably looks like error[i].field[i].error, so you're just calling the fieldname and not the error message. Call e.error in your Template() function.

need help creating permalinks in google app engine

So I am trying to create a unique permalink each time that a person posts on my webpage and I want it to be relatively search engine friendly so I have made a little code to change the title to a good search engine title and it is working but then my handler cannot accept it. At least that is what I think is happening because the webpage just gives me a 404 error. The HTML works fine because when I redirect to a static page it all goes through. Here is the applicable code:
def post(self):
subject = self.request.get('subject')
content = self.request.get('content')
if subject and content:
p = Post(parent = blog_key(), subject = subject, content = content)
p.put()
id=str(p.key().id())
subject = str(subject)
subject = subject.replace(' ', '25fdsa67ggggsd5')
subject = ''.join(e for e in subject if e.isalnum())
subject = subject.replace('25fdsa67ggggsd5', '-')
subject = subject.lower()
url = '/blog/%s/%s' % (id, subject)
self.redirect('/blog/%s/%s' % (id, subject))
class PostPage(BlogHandler):
def get(self, post_id):
key = db.Key.from_path('PersonalPost', int(post_id), parent=blog_key())
post = db.get(key)
if not post:
self.error(404)
return
self.render("permalink.html", post = post)
class PersonalPost(db.Model):
subject = db.StringProperty(required = True)
content = db.TextProperty(required = True)
created = db.DateTimeProperty(auto_now_add = True)
last_modified = db.DateTimeProperty(auto_now = True)
user_id = db.StringProperty(required = True)
def render(self):
self._render_text = self.content.replace('\n', '<br>')
return render_str("post.html", p = self)
def blog_key(name = 'default'):
return db.Key.from_path('blogs', name)
app = webapp2.WSGIApplication([('/blog/([0-9]+)/([.*]+)', PostPage)]
And again it works when I just have it redirect to the main page and list them but not when I try to direct to the new SEO page.
UPDATE:
The test url I am using is setting
subject = "test-url"
id = "1234"
The app then directs me to www.url.com/blog/1234/test-url but it gives me a 404 error.
You define two groups in ('/blog/([0-9]+)/([.*]+) but your PostPage.get() only takes one.
Change it to def get(self, post_id, subject) or remove the second group ('/blog/([0-9]+)/[.*]+
I think you should have a look at the quotes on ur handler mapping, it seems inconsistent.
yours: app = webapp2.WSGIApplication([('/blog/([0-9]+)/([.*]+)', PostPage)]
try : app = webapp2.WSGIApplication(['/blog/([0-9]+)/([.*]+)', PostPage)]

Using tastypie resource in view

my first question here :
So I'm using tastypie to have api's for my app.
I want to be able to use tastypie to render json and then include that in a django view so that I can bootstrap my app's data.
There is an example of this in django tastypie cookbook here : http://django-tastypie.readthedocs.org/en/latest/cookbook.html#using-your-resource-in-regular-views
The problem is that I CANNOT get this to work, I've tried variants from simpler to more complex and I just cant get it, here some code for my models :
class ChatMessage(models.Model):
content = models.TextField()
added = models.DateTimeField(auto_now_add=True)
author = models.ForeignKey(ChatUser, related_name="messages")
chat_session = models.ForeignKey(ChatSession, related_name="messages")
answer_to = models.ForeignKey('self', blank=True, null=True)
flagged = models.BooleanField(blank=True,default=False)
mododeleted = models.BooleanField(blank=True,default=False)
mododeleted_by = models.ForeignKey(ChatUser,blank=True,null=True,default=None)
mododeleted_at = models.DateTimeField(blank=True,null=True,default=None)
[...]
class ChatSession (models.Model):
title = models.CharField(max_length=200)
link_title = models.CharField(max_length=200)
description = tinymce_models.HTMLField()
date = models.DateTimeField()
online = models.BooleanField(default=False)
next_session = models.BooleanField(default=False)
meps = models.ManyToManyField(ChatMep)
uid_newsupdate = models.CharField(max_length=200,blank=True,null=True,default="")
[...]
and my resources :
class ChatMessageResource(MyModelResource):
chat_session = fields.ForeignKey(ChatSessionResource, 'chat_session')
def renderOne(self,request,pkval):
data = self.obj_get(None,pk=pkval)
dbundle = self.build_bundle(obj=data,request=request)
return self.serialize(None,self.full_dehydrate(dbundle),'application/json')
def dehydrate(self, bundle):
bundle.data['likes'] = bundle.obj.get_likes()
bundle.data['likes_count'] = len(bundle.data['likes'])
return bundle
class Meta:
authentication = Authentication()
authorization = Authorization()
queryset = ChatMessage.objects.all()
resource_name = 'message'
fields = ('content', 'added', 'flagged', 'mododeleted','author','answer_to','chat_session')
filtering = {
'chat_session': ALL_WITH_RELATIONS,
}
and my view index :
def index(request):
cur_sess = get_current_chat_session()
data1= ChatMessageResource().renderOne(request,723)
return render_to_response('test.html',
{
'all_data' : data1
},
context_instance=RequestContext(request))
What I want is my renderOne() function to give me the json of ONE ChatMessageResource
And also I'd like a renderAll() function to gice me ALL (or filtered) ChatMessageResources in json.
And I want to use tastypie internals, I KNOW i could serialize it by myself but that's not the point..
Right now the error is :
NoReverseMatch at /live/
Reverse for 'api_dispatch_detail' with arguments '()' and keyword arguments '{'pk': 14L, 'resource_name': 'session'}' not found.
I'm just getting crazy, I've been trying for hours.
So please, how to get ONE/ALL resource as JSON by code using tastypie in a django view !
If It's not clear or I need to clarify, please just ask, thanks
Really what I want to do is to be able to get the JSON returned by an API url I created, but from code, not by visiting the url .. So If I have /api/v1/messages/?chat_session=14 which return a list of messages, I want to be able to do the same by code (and not by fetching the url with curl or something please).
Note :
definition of ModelResource.obj_get from https://github.com/toastdriven/django-tastypie/blob/master/tastypie/resources.py
def obj_get(self, request=None, **kwargs):
"""
A ORM-specific implementation of ``obj_get``.
Takes optional ``kwargs``, which are used to narrow the query to find
the instance.
"""
try:
base_object_list = self.get_object_list(request).filter(**kwargs)
object_list = self.apply_authorization_limits(request, base_object_list)
stringified_kwargs = ', '.join(["%s=%s" % (k, v) for k, v in kwargs.items()])
if len(object_list) <= 0:
raise self._meta.object_class.DoesNotExist("Couldn't find an instance of '%s' which matched '%s'." % (self._meta.object_class.__name__, stringified_kwargs))
elif len(object_list) > 1:
raise MultipleObjectsReturned("More than '%s' matched '%s'." % (self._meta.object_class.__name__, stringified_kwargs))
return object_list[0]
except ValueError:
raise NotFound("Invalid resource lookup data provided (mismatched type).")
So here I found the solution, the problem was with url resolving ... I needed to add
def get_resource_uri(self, bundle_or_obj):
return '/api/v1/%s/%s/' % (self._meta.resource_name,bundle_or_obj.obj.id)
to the related object (session here) in order for it to work (don't ask why!)
So here is my working solution for renderDetail and renderList :
def renderDetail(self,pkval):
request = HttpRequest()
request.GET = {'format': 'json'}
resp = self.get_detail(request, pk=pkval)
return resp.content
def renderList(self,options={}):
request = HttpRequest()
request.GET = {'format': 'json'}
if len(options) > 0:
request.GET.update(options)
resp = self.get_list(request)
return resp.content
And here is an example usage :
cmr = ChatMessageResource()
dataOne= cmr.renderDetail("723")
dataAll = cmr.renderList({'limit':'0','chat_session':cur_sess.pk})
https://github.com/toastdriven/django-tastypie/issues/962
I've found that obj_get method needs a bundled request object. See the link.
def user_detail(request, username):
ur = UserResource()
# Add this request bundle to the obj_get() method as shown.
req_bundle = ur.build_bundle(request=request)
user = ur.obj_get(req_bundle, username=username)
....
Your problem seems to be here:
data = self.obj_get(None,pk=pkval)
The parameters to obj_get should be kwargs that can be passed directly to a standard get. None should not be in there.

Categories

Resources