django orm search with postgres - python

For small searches in PostgreSQL, http://django-orm.readthedocs.org/en/latest/orm-pg-fulltext.html can be used easily, as shown in the docs.
Following are the steps, I used to implement it -
'''call the libraries'''
from djorm_pgfulltext.models import SearchManager
from djorm_pgfulltext.fields import VectorField
class Notes(models.Model):
title = models.CharField()
description = models.TextField()
# create a vector field
search_index = VectorField()
objects = models.Manager()
search_manager = SearchManager(
fields=('title', 'description'),
config='pg_catalog.english',
search_field='search_index',
auto_update_search_field=True
)
Ran the migration, and all the changes are being reflected in the database.
Last step -
In my postgresql database, I did the following -
sudo -u postgres psql postgres // login as root
CREATE EXTENSION unaccent;
ALTER FUNCTION unaccent(text) IMMUTABLE;
All this done, now I open my shell
from myapp.models import Notes
In [2]: Note.search_manager.search("p")
Out[3]: []
Any idea, why I am getting no result??
What is missing?

Related

How to create an object for a model from the console similar to how Django made the createsuperuser command

I'm trying to create an object from the console but not sure how to set that up.
This is my modelManager:
class MajorManager(models.Manager):
def __str__(self):
return self.name
def createMajor(self, name):
try:
name = name.lower()
major = self.create(name=name)
except IntegrityError:
print("This major has already been created")
And here is the model:
class Majors(models.Model):
name = models.CharField(max_length=30, unique=True)
objects = MajorManager()
Any help would be much appreciated.
You can go this route using Django's API - checkout the docs
First create a shell:
python manage.py shell
Then you can import your models and do basic CRUD on them.
>>> from polls.models import Choice, Question # Import the model classes we just wrote.
# No questions are in the system yet.
>>> Question.objects.all()
<QuerySet []>
# Create a new Question.
# Support for time zones is enabled in the default settings file, so
# Django expects a datetime with tzinfo for pub_date. Use timezone.now()
# instead of datetime.datetime.now() and it will do the right thing.
>>> from django.utils import timezone
>>> q = Question(question_text="What's new?", pub_date=timezone.now())
# Save the object into the database. You have to call save() explicitly.
>>> q.save()
Or, alternatively you can try the dbshell route, here's the documentation.
This command assumes the programs are on your PATH so that a simple
call to the program name (psql, mysql, sqlite3, sqlplus) will find the
program in the right place. There’s no way to specify the location of
the program manually.
You can't use the Django's ORM though, it's pure SQL, so it would be instructions like:
CREATE TABLE user (
Id Int,
Name Varchar
);

Django loaddata with natural keys and many-to-many-relation to self

I have a model with ManyToManyField to "self". Now I want to copy data from my local to the production environment. In order to do that, I'd like to dump all entries from my database to a fixture and load that on the production server. Because that server already contains data, I export them using natural keys. The export runs fine, but I get errors during loaddata if one instance references a relation to another instance of the same model that is defined below that one in the fixture.
What confuses me: Everything goes well, if I don't use natural keys. But this clashes with data already present on production.
Please consider the following example:
from django.db import models
class PersonManager(models.Manager):
def get_by_natural_key(self, name):
return self.get(name=name)
class Person(models.Model):
objects = PersonManager()
name = models.CharField(max_length=150, unique=True)
friends = models.ManyToManyField("self", blank=True)
def natural_key(self):
return self.name,
To reproduce:
>>> python manage.py shell
>>> from relations.models import *
>>> person1 = Person.objects.create(name='foo')
>>> person2 = Person.objects.create(name='bar')
>>> person3 = Person.objects.create(name='baz')
>>> person2.friends.add(person2)
>>> person2.save()
>>> exit()
>>>
>>> python manage.py dumpdata relations.Person --natural-primary --natural-foreign > relations/fixtures/people.json
# change the database url to an empty database and run migrations
>>> python3 manage.py loaddata people
raises relations.models.DoesNotExist: Person matching query does not exist.
If I omit the --natural-primary and -natural-foreign but follow the same steps, loading into a fresh db works.
Can somebody explain why the different behaviour occurs?

Django raw raises relation does not exist

i am getting a relation does not exist and I cant find a solution.
error:relation "sales_Oeslshstsql" does not exist
LINE 1: SELECT * FROM "sales_Oeslshstsql
(app name is sales)
model:
class Oeslshstsql(models.Model):
hst_prd = models.SmallIntegerField()
hst_year = models.SmallIntegerField()
cus_no = models.CharField(max_length=12)
item_no = models.CharField(max_length=15)
.....
a4glidentity = models.IntegerField(db_column='A4GLIdentity', primary_key = True)
class Meta:
managed = False
db_table = 'OESLSHST_SQL'
def __str__(self):
return (self.hst_year)
View:
def sales(request):
#sales_list = Oeslshstsql.objects.all().order_by('hst_year','hst_prd').reverse()
s = Oeslshstsql.objects.raw('SELECT * FROM "sales_Oeslshstsql"')
sales_list = s
return render(request,'saleslist.html',{'sales_list':sales_list})
The error is raised when s is evaluated. I tried switching cases in the select and messed with migrations no luck.
I am migrating an existing app to Django using a postgres backend, any help would be appreciated.
try:
s = Oeslshstsql.objects.raw('SELECT a4glidentity as id, ... , FROM "OESLSHST_SQL"')
http://docs.djangoproject.com/en/1.8/ref/models/options/#db-table seems your tablename in query is wrong
edit: you should add the primary key as id see https://docs.djangoproject.com/en/1.8/topics/db/sql/#mapping-query-fields-to-model-fields
Hi I had the same issue migrating an existing app to 1.11. The only solution I found was ....
Clear all all files from the app's migrations dir leaving only the init.py file
Make sure that the admin.py file is empty
Run manage.py makemigrations
Run manage.py sqlmigrate <app_label> 0001
copy the sql output
using pgAdminIII select "Execute arbitrary SQL queries"
Paste and execute the SQL statements in pgAdminIII
This was the only solution I could find, bit of a hack true, but worked. Hope it helps.This would also work via psql terminal I suppose, but I used pgAdmin

Getting missing column error whenever a model is saved

I'm having trouble creating a model in django. I wrote a model like this:
from django.db import models
class FooModel(models.Model):
name = models.CharField(max_length = 255)
I run
manage.py syncdb
But when I'm in the shell, I can't save an instance. Every time I call it, it tells me it's missing a column
manage.py shell
>>from app.models import FooModel
>>foo = FooModel()
>>foo.name = 'foo'
>>foo.save()
DatabaseError: column "name" of relation "ecommerce_foomodel" does not exist
LINE 1: INSERT INTO "ecommerce_foomodel" ("name") VALUES (E'123123as...
We're using postgres.
The database table was created before you added the corresponding fields.
So, you can recreate all of the tables of that app (in case you don't have any useful data) using:
python manage.py reset ecommerce
Or, you should migrate the database to the latest version using South.

UUID field added after data already in database. Is there any way to populate the UUID field for existing data?

I've added a UUID field to some of my models and then migrated with South. Any new objects I create have the UUID field populated correctly. However the UUID fields on all my older data is null.
Is there any way to populate UUID data for existing data?
For the following sample class:
from django_extensions.db.fields import UUIDField
def MyClass:
uuid = UUIDField(editable=False, blank=True)
name = models.CharField()
If you're using South, create a data migration:
python ./manage.py datamigration <appname> --auto
And then use the following code to update the migration with the specific logic to add a UUID:
from django_extensions.utils import uuid
def forwards(self, orm):
for item in orm['mypp.myclass'].objects.all():
if not item.uuid:
item.uuid = uuid.uuid4() #creates a random GUID
item.save()
def backwards(self, orm):
for item in orm['mypp.myclass'].objects.all():
if item.uuid:
item.uuid = None
item.save()
You can create different types of UUIDs, each generated differently. the uuid.py module in Django-extensions has the complete list of the types of UUIDs you can create.
It's important to note that if you run this migration in an environment with a lot of objects, it has the potential to time out (for instance, if using fabric to deploy). An alternative method of filling in already existing fields will be required for production environments.
It's possible to run out of memory while trying to do this to a large number of objects (we found ourselves running out of memory and having the deployment fail with 17,000+ objects).
To get around this, you need to create a custom iterator in your migration (or stick it where it's really useful, and refer to it in your migration). It would look something like this:
def queryset_iterator(queryset, chunksize=1000):
import gc
pk = 0
last_pk = queryset.order_by('-pk')[0].pk
queryset=queryset.order_by('pk')
if queryset.count() < 1
return []
while pk < last_pk:
for row in queryset.filter(pk__gt=pk)[:chunksize]:
pk = row.pk
yield row
gc.collect()
And then your migrations would change to look like this:
class Migration(DataMigration):
def forwards(self, orm):
for item in queryset_iterator(orm['myapp.myclass'].objects.all()):
if not item.uuid:
item.uuid = uuid.uuid1()
item.save()
def backwards(self, orm):
for item in queryset_iterator(orm['myapp.myclass'].objects.all()):
if item.uuid:
item.uuid = None
item.save()
To add UUID values to all existing records first you will need to make sure your model has the UUID filed with blank=True, null=True
Then Run the schemamigration command with south and then open up the resulting migration file.
And then Edit your migration file with the following as shown in this post
Quote:
You'll need to import the following import uuid
At the end of the forwards() function add the following def forwards(self, orm):
...
for a in MyModel.objects.all():
a.uuid = u'' + str(uuid.uuid1().hex)
a.save()
As stated that will loop through existing instances and add a uuid to it as part of the migration.
There is now an excellent, updated answer for Django 1.9 to this exact question in the Django docs.
Saved me a lot of time!

Categories

Resources