What I'm trying to do is saving multiple scraped data(actor names) in my Actor table inside Django models.
So far I've written the for loop below to achieve this goal but it only saves the last object.
class Actor(models.Model):
name = models.CharField(max_length=50)
url = models.URLField()
def __str__(self):
return self.name
def save(self, *args, **kwargs):
i = 0
for num in range(5):
title, year, actor_list = scraper(self.url)
self.name = actor_list[i]
super().save(*args, **kwargs)
i += 1
I get the URL from users within a form, then save the form and send it to my models and the scraping begins.
I scrape 5 actor names inside the scraper() function and append them to actor_list.
But when I try to save the actor_list under the save function, it only saves the 5th actor which I think overwrites the previously saved objects.
Is there something wrong with my for loop? or should I completely change my approach for this purpose?
I'd also prefer to save the objects using Actor.objects.get_or_create() to skip the already existing objects, but I don't know how and where.
I would appreciate it if someone could help me with this.
Yes, you should use a completely different approach. A Model is just a code representation of your database table.
You are also executing the same scraping 5 times on the same url, but that is another point.
Doing scraping and creating multiple actors should not be done within the model. Bear in mind that the model gets instantiated as an actor object (a single record in the actor table), so every time you do a super().save() it overwrites the same record.
The standard way is to do it in the view you use to print the form and have that form submit to its own URL (i.e. the form action should point to the same view you use to print it).
Something like this:
forms.py
from django import forms
class MyForm(forms.Form):
url = forms.CharField(label='Your name', max_length=255)
views.py
from my_app.forms import MyForm
from my_app.models import Actor
from django.shortcuts import render
def my_view(request):
if request.method == "POST":
form = MyForm(request.POST)
if form.is_valid():
title, year, actor_list = scraper(form.cleaned_data['url'])
for actor in actor_list:
# Here you are creating new instances of Actor (new records)
# at each iteration, unless they already exist.
Actor.objects.get_or_create(name=actor)
else:
form = MyForm()
return render(request, 'your_template.html', {'form': form})
The save() method acts on a single Actor object. It's wrong to try to save multiple actors there.
Do this in a view:
title, year, actor_list = scraper(self.url)
for i in range(5):
actor = Actor()
actor.name = actor_list[i]
actor.save()
And delete the save() method in the model
Related
I have a program that lets users upload data files and lookup tables (both which are ID'd to a specific company) and map them together. One page lets users choose which company they want to map data for by looking at which companies have both data files and lookup tables, which I use a queryset/model manager for. The problem is if I load a new data file and hierarchy the queryset doesn't pick them up until the server restarts. The queryset returns all the companies that have a data file and hierarchies at the time the server starts, but not anything that's added afterwards. I think this must be because the queryset is defined at startup, but I'm not sure. Is there a way to work around this?
forms.py
class CompanySelectionForm(forms.Form):
companies = RawData.objects.get_companyNames(source="inRDandH")
companiesTuple = makeTuple(companies)
print(companiesTuple)
company = forms.ChoiceField(widget=forms.Select(attrs={'class': 'form-select'}), choices=companiesTuple)
managers.py
class RawDataManager(models.Manager):
def get_queryset(self):
return RawDataQuerySet(self.model, using=self._db)
def get_companyNames(self, source):
return self.get_queryset().get_companyNames(source)
class RawDataQuerySet(models.QuerySet):
def get_companyNames(self, source):
if (source == 'inRDandH'):
distinct_companiesRD = self.filter(fileType=0).values_list('companyName', flat=True).distinct()
distinct_companiesH = self.filter(fileType=1).values_list('companyName', flat=True).distinct()
distinct_companies = set(distinct_companiesRD).intersection(set(distinct_companiesH))
else:
distinct_companies = self.values_list('companyName', flat=True).distinct()
return distinct_companies
The problem is that this code runs only once, when the code is initialised on server start, because it is part of your form class definition:
companies = RawData.objects.get_companyNames(source="inRDandH")
The solution is to make choices a callable, which is run every time the form is instantiated. define that field dynamically, in the __init__ method of the form:
def get_companies_tuple():
companies = RawData.objects.get_companyNames(source="inRDandH")
return makeTuple(companies)
class CompanySelectionForm(forms.Form):
company = forms.ChoiceField(
widget=forms.Select(attrs={'class': 'form-select'}),
choices=get_companies_tuple
)
This will now fetch the data from the database every time the form is initialised, rather than only once during startup.
I have a django form that has a multiple choice field. The field should be dynamic, that is, only the records associated with the user who is currently logged in should be displayed. I've managed to put this together so far;
forms.py
class myForm(forms.ModelForm):
def __init__(self, someUser, *args, **kwargs):
super(myForm, self).__init__(*args, **kwargs)
someRecords = models.SomeModel.objects.filter(someUser = someUser)
#The line above gets records associated with a specific user
displayNames = []
for i in someRecords:
displayNames.append((i.someField, i.someOtherField + ' ' + i.someOtherField2))
#I am basically making a list of tuples, containing a field and a concatnation of two other fields. The latter will be what is displayed in the select box
self.fields['theSelectField'] = forms.ChoiceField(choices = displayNames)
class Meta:
#I defined model, fields and labels here
views.py
def myFormPage(request):
someUser = request.user.someextensionofuser.someUser
form = forms.myForm(someUser)
context = {'form': form}
if request.method == 'POST':
form = forms.myForm(someUser, data = request.POST)
if form.is_valid():
#Do Stuff if form is valid. However,this stuff doesn't get done, the page refreshes instead
So I've managed to make the select options dynamic. However, now I can't submit data.
EDIT: One of the comments helped me solve the previously stated problem. I've updated the views.py code. However, now I'm running into this error;
Cannot assign "'someString'": "someModel.someField" must be a
"someForeignModel" instance
The option values seem to be strings instead of references to objects. How do I solve this?
This limits the possible options of your select field:
self.fields['theSelectField'].queryset = SomeModel.objects.filter(someUser = someUser)
In your views you might want to use a Class Based View, because it handles a lot of stuff automatically and saves you time. Take a look here: https://ccbv.co.uk/
I firgured it out. Since my main problem was with how the options are displayed to a user, I decided to go with changing my str method in models.py to;
class someModel(models.Model):
#my fields
def __str__(self):
return self.someField + ' ' + self.someOtherField
Then in my forms.py, I went with #dmoe's answer;
self.fields['theSelectField'] = forms.ModelChoiceField(queryset = models.SomeModel.objects.filter(someUser = someUser))
So now both problems are solved. My options have custom labels, and I can submit my data without running into valueError.
I have a model which creates Memo objects. I would like to use a custom Model Manager's posted method to return the total number of Memo objects - then use this number within a template. I am trying to keep as much of my code as possible within my Models and Model Managers and less within my Views as I read that this was a best practice in 'Two Scoops of Django'.
In the shell I can get the number of memos as such:
>>> from memos.models import Memo
>>> Memo.objects.all()
<QuerySet [<Memo: Test Memo 2>, <Memo: Test Memo 1>]>
>>> Memo.objects.all().count()
2
This is what my Model and Model Manager look like:
class MemoManager(models.Manager):
use_for_related_fields = True
def posted(self):
return self.count()
class Memo(models.Model):
title = models.CharField(max_length=100)
content = models.TextField()
date_time = models.DateTimeField(default=timezone.now)
author = models.ForeignKey(User, on_delete=models.CASCADE)
objects = MemoManager()
def __str__(self):
return self.title
def get_absolute_url(self):
return reverse('memos-detail', kwargs={'pk': self.pk})
I know this is clearly the wrong way to do it but I have confused myself here. So how do I use my Model Manager to get the count of objects and use it in a template like: {{ objects.all.count }}?
P.S. I see other posts that show how to do this within the view but as stated I am trying not to use the view. Is using the view required? I also understand my posted method is written incorrectly.
I'm sorry but you have misinterpreted what was written in TSD. The Lean View Fat Model is meant to keep code which pertains to 'business logic' out of the views, and certain model specific things. A request should be handled by a view. So when you want to load a template, you must first have a GET request to your app.
A view function should be written such that Validation of POST data or the Creation of a new object in DB or Querying/Filtering for GET requests should be handled in the corresponding serializer/model/model manager.
What should be happening while you want to load your template.
Have a url for the template that you have created and a view function mapped for it
In the view function you should render said template and pass the necessary data inside the context.
To keep in line with the Lean View Fat Model style, if you want to get a Queryset of of Memo's but only those which have their is_deleted fields set to False, you can overwrite the model manager get_queryset() method for Memo model.
If you want to create a new Memo with a POST request, you can handle
the creation using a ModelForm!
Hope this clears things up!
EDIT:
How to pass a context to a template, in your case the memo count.
def random_memo_view(request):
context = {'memo_count': Memo.posted()}
return render(request, 'template.html', context=context)
RE-EDIT
I just checked that you were using DetailView. In this case follow this from the django docs.
Class Based Views: Adding Extra Context
For example.
class One(models.Model):
text=models.CharField(max_length=100)
class Two(models.Model):
test = models.Integer()
many = models.ManyToManyField(One, blank=True)
When I try save my object in admin panel, I take error such as:
"'Two' instance needs to have a primary key value before a many-to-many relationship can be used."
I use django 1.3. I tried add AutoField to Two class, but it's not work too.
This is my code.
from django.http import HttpResponse, HttpResponseRedirect
from django.shortcuts import render_to_response, redirect
from django.template import RequestContext
from django.core.urlresolvers import reverse
from project.foo.forms import FooForm
from project.foo.models import Foo
from project.fooTwo.views import fooTwoView
def foo(request, template_name="foo_form.html"):
if request.method == 'POST':
form = FooForm(data=request.POST)
if form.is_valid():
foo = Foo()
foo.name = request.POST.get("name")
foo.count_people = request.POST.get("count_people")
foo.date_time = request.POST.get("date_time")
foo.save()
return fooTwoView(request)
else:
form = FooForm()
return render_to_response(template_name, RequestContext(request, {
"form": form,
}))
P.S. I find my fail. It is in model. I used many-to-many in save method. I add checking before using, but it's not help.
class Foo(models.Model):
name = models.CharField(max_length=100, null=False, blank=False)
count_people = models.PositiveSmallIntegerField()
menu = models.ManyToManyField(Product, blank=True, null=True)
count_people = models.Integer()
full_cost = models.IntegerField(blank=True)
def save(self, *args, **kwargs):
if(hasattr(self,'menu')):
self.full_cost = self.calculate_full_cost()
super(Foo, self).save(*args, **kwargs)
def calculate_full_cost(self):
cost_from_products = sum([product.price for product in self.menu.all()])
percent = cost_from_products * 0.1
return cost_from_products + percent
I try hack in save method such as
if(hasattr(self,Two)):
self.full_cost = self.calculate_full_cost()
This is help me, but i dont think that is the django way. What is interesting, that is without this checking admin panel show error, but create object. Now, if i select item from Two and save, my object does not have full_cost, but when i view my object, admin panel remember my choice and show me my Two item, what i select... I dont know why.
How do i save this?
There are quite a few problems with your code. The most obvious one are
1/ in your view, using a form for user inputs validation/sanitization/conversion then ignoring the santized/converted data and getting unsanitized inputs directly from the request. Use form.cleaned_data instead of request.POST to get your data, or even better use a ModelForm which will take care of creating a fully populated Foo instance for you.
2/ there's NO implicit "this" (or "self" or whatever) pointer in Python methods, you have to explicitely use "self" to get at the instance attributes. Here's what your model's "save" method really do:
def save(self, *args, **kwargs):
# test the truth value of the builtin "id" function
if(id):
# create a local variable "full_cost"
full_cost = self.calculate_full_cost()
# call on super with a wrong base class
super(Banquet, self).save(*args, **kwargs)
# and exit, discarding the value of "full_cost"
Now with regard to your question: Foo.save is obviously not the right place to compute someting based on m2m related objects. Either write a distinct method that run the computation AND update Foo AND save it and call it after the m2m are saved (hint : a ModelForm will take care of saveing the m2m related objects for you), or just use the m2m_changed signal.
This being said, I strongly suggest you spend a few hours learning Python and Django - it will save you a lot of time.
Why not use "OneToOneField" instead of Many-to-Many
I am using FormWizard to complete a set of operation in my app, I have two models Employee and Person, Employee class inherits Person, and all the fields of Person are available for Employee object.
Now I am creating a set of forms using FormWizard, I just wanted to know that. If a user starts entering the data in the forms and fills upto 2 forms out of 4 and is willing to fill the rest of the forms afterwards. So is this possible that the data for the two forms which he filled can be saved in the database.
And the next time he comes can complete the operation form the 3rd form.
If anyone knows that then plz help me out, it would be a great help. Thank You!
what you can do is every step, save out the form state to some serialised object in db ForeignKeyed to the user.
then when hooking up the formwizard, wrap the formwizard view in a custom view which checks if the user has a saved form and if so deserialises and redirects to the appropriate step.
Edit: seems formwizard saves state in POST. only need to save postdata.
models.py:
class SavedForm(Model):
user = ForeignKey(User)
postdata = TextField()
views.py:
import pickle
class MyWizard(FormWizard):
def done(self, request, form_list):
SavedForm.objects.get(user=request.user).delete() # clear state!!
return render_to_response('done.html',)
formwizard = MyWizard([Form1, Form2]) <- class name, not instance name
def formwizard_proxy(request, step):
if not request.POST: #if first visit, get stored data
try:
prev_data = SavedForm.objects.get(user=request.user)
request.POST = pickle.loads(prev_data.postdata)
except:
pass
else: # otherwise save statet:
try:
data = SavedForm.objects.get(user=request.user)
except:
data = SavedForm(user=request.user)
data.postdata=pickle.dumps(request.POST)
data.save()
return formwizard(request)
edit: changed formwizard constructor