Develop Reference

Can not delete document object using mongoengine

I am working with mongoDB 4.2 using python 3.7 and library mongoengine 0.18.2.
So, I want to delete a document but I got lookup error.
ODM:
from mongoengine import Document
from mongoengine.fields import *
class Myparent(Document):
fieldfoo = IntField()
fieldbar = IntField()
class Mychild(Document):
fieldfoo = StringField()
myparent = ReferenceField('Myparent')
Now, when I want to delete a child:
item = Mychild.objects.get(id=123456)
item.delete()
I got this error:
site-packages/mongoengine/queryset/transform.py", line 60, in query
fields = _doc_cls._lookup_field(parts)
site-packages/mongoengine/base/document.py", line 1032, in _lookup_field
raise LookUpError('Cannot resolve field "%s"' % field_name)
mongoengine.errors.LookUpError: Cannot resolve field "myparent"
Any clue? Thanks

The id you are providing is not a valid id. Mongo supports 12 byte binary BSON type format which is definitely not 123456. If your id is correct then you can do the following :
item=Mychild.objects(pk='some_id')
Whatever is returned in item will be a list but id is unique so you can write it as:
item=Mychild.objects(pk='some_id').first()
item.delete()

python, mongo and marshmallow: datetime struggles

I'm trying to do something pretty simple: get the current time, validate my object with marshmallow, store it in mongo
python 3.7
requirements:
datetime==4.3
marshmallow==3.5.1
pymongo==3.10.1
schema.py
from marshmallow import Schema, fields
...
class MySchema(Schema):
user_id = fields.Str(required=True)
user_name = fields.Str()
date = fields.DateTime()
account_type = fields.Str()
object = fields.Raw()
preapredata.py
from datetime import datetime
from schema.py import Myschema
...
dt = datetime.now()
x = dt.isoformat()
data = {
"user_id": '123123123',
"user_name": 'my cool name',
"date": x,
"account_type": 'another sting',
"trade": {'some':'dict'}
}
# validate the schema for storage
validator = MySchema().load(data)
if 'errors' in validator:
log.info('validator.errors')
log.info(validator.errors)
...
res = MyService().create(
data
)
myservice.py
def create(self, data):
log.info("in creating data service")
log.info(data)
self.repo.create(data)
return MySchema().dump(data)
connector to mongo is fine, am saving other data that has no datetime with no issue.
I seem to have gone through a hundred different variations of formatting the datetime before passing it to the date key, as well as specifying the 'format' option in the schema field both inline and in the meta class, example:
#class Meta:
# datetimeformat = '%Y-%m-%dT%H:%M:%S+03:00'
Most variations I try result in:
{'date': ['Not a valid datetime.']}
i've finally managing to pass validation going in by using simply
x = dt.isoformat()
and leaving the field schema as default ( date = fields.DateTime() )
but when i dump back through marshmallow i get
AttributeError: 'str' object has no attribute 'isoformat'
the record is created in mongo DB fine, but the field type is string, ideally I'd like to leverage the native mongo date field
if i try and pass
datetime.now()
to the date, it fails with
{'date': ['Not a valid datetime.']}
same for
datetime.utcnow()
Any guidance really appreciated.
Edit: when bypassing marshmallow, and using either
datetime.now(pytz.utc)
or
datetime.utcnow()
field data stored in mongo as expected as date, so the issue i think can be stated more succinctly as: how can i have marshmallow fields.DateTime() validate either of these formats?
Edit 2:
so we have already begun refactoring thanks to Jérôme's insightful answer below.
for anyone who wants to 'twist' marshmallow to behave like the original question stated, we ended up going with:
date = fields.DateTime(
#dump_only=True,
default=lambda: datetime.utcnow(),
missing=lambda: datetime.utcnow(),
allow_none=False
)
i.e. skip passing date at all, have marshmallow generate it from missing, which was satisfying our use case.

The point of marshmallow is to load data from serialized (say, JSON, isoformat string, etc.) into actual Python objects (int, datetime,...). And conversely to dump it from object to a serialized string.
Marshmallow also provides validation on load, and only on load. When dumping, the data comes from the application and shouldn't need validation.
It is useful in an API to load and validate data from the outside world before using it in an application. And to serialize it back to the outside world.
If your data is in serialized form, which is the case when you call isoformat() on your datetime, then marshmallow can load it, and you get a Python object, with a real datetime in it. This is what you should feed pymongo.
# load/validate the schema for storage
try:
loaded_data = MySchema().load(data)
except ValidationError as exc:
log.info('validator.errors')
log.info(exc.errors)
...
# Store object in database
res = MyService().create(loaded_data)
Since marshmallow 3, load always returns deserialized content and you need to try/catch validation errors.
If your data does not come to your application in deserialized form (if it is in object form already), then maybe marshmallow is not the right tool for the job, because it does not perform validation on deserialized objects (see https://github.com/marshmallow-code/marshmallow/issues/1415).
Or maybe it is. You could use an Object-Document Mapper (ODM) to manage the validation and database management. This is an extra layer other pymongo. umongo is a marshmallow-based mongoDB ODM. There are other ODMs out there: mongoengine, pymodm.
BTW, what is this
datetime==4.3
Did you install DateTime? You don't need this.
Disclaimer: marshmallow and umongo maintainer speaking.

In Django, how do I serialize a QuerySet that returns a dict object (created with .values())

In Django 1.10 I am trying to serialize an QuerySet that I obtained from the following query:
events_in_period = Event.objects \
.filter(timestamp__gte=start_day,
timestamp__lte=end_day,
request__isnull=False,
name=EventType.COMPLETED) \
.annotate(date=TruncDay('timestamp')) \
.values('date') \
.annotate(completed_count=Count('id')) \
.order_by('date')
The main thing is the .values() statement that makes this statement return a QuerySet that contains a dict rather than a Django model instance.
Therefore the following calls to serialize it
from django.core import serializers
output = serializers.serialize('json', result)
fail with the following error:
AttributeError: 'dict' object has no attribute '_meta'
Any suggestions on serialization without omitting the .values() as I need them for brevity.

If you want to serialize the dict object into json, then you could import json,
import json
data = json.dumps(result)
The Django serializers are for Django model objects and QuerySets. That's why it's looking for a _meta field. 

Serializing Django queryset to JSON: getting AttributeError 'tuple' object has no attribute '_meta'

I'm using Django 1.8. This is my view:
from django.http import JsonResponse
...
query = "SELECT * FROM frontend_chemical WHERE id LIKE %s"
cursor.execute(query, (code + "%",))
data = cursor.fetchall()
print data
return JsonResponse(serializers.serialize('json', data), safe=False)
But this gives me:
AttributeError at /api/1.0/bnf_code: 'tuple' object has no attribute '_meta'
At the line:
return JsonResponse(serializers.serialize('json', data), safe=True)
In the console, I can see that my data has returned OK:
[(u'0210', u'Stable angina, acute coronary syndromes, and fibrinolysis'), (u'0211', u'Antifibrinolytic drugs and haemostatics'), (u'0212', u'Lipid-regulating drugs'), (u'0213', u'Local sclerosants')]
Any idea what might be going wrong?
UPDATE: I've also been trying to get the data as a dictionary, using data = dictfetchall(cursor) as described here.
However, this just fails in the same place with a slightly different error:
AttributeError: 'dict' object has no attribute '_meta'

Serializers are used to serialize the django models only. To serialize simple python data use the built-in json module:
import json
return JsonResponse(json.dumps(data), safe=True)

Converting a django ValuesQuerySet to a json object

I'm trying to use the ValuesQuerySet feature in Django to limit the number of fields returned from query to only those I need. I would like to serialize this data set a JSON object However, Django keeps throwing an error. Below I've included my code and the error I receive:
objectList = ConventionCard.objects.values('fileName','id').filter(ownerUser = user)
data = serializers.serialize('json', objectList)
return HttpResponse(data, mimetype='application/javascript')
The Error:
Exception Type: AttributeError
Exception Value: 'dict' object has no attribute '_meta'
Exception Location: C:\Python27\lib\site-packages\django\core\serializers\base.py in serialize, line 41
Thanks !

Cast the ValuesQuerySet to a list first:
query_set = ConventionCard.objects.values('fileName','id').filter(ownerUser = user)
list(query_set)
Removing the values call as suggested by ars causes the manager to pull all columns from the table, instead of only the two you need.

Try subsetting the fields in your values list through the serialize method using a QuerySet instead:
from django.core import serializers
objectQuerySet = ConventionCard.objects.filter(ownerUser = user)
data = serializers.serialize('json', objectQuerySet, fields=('fileName','id'))

I continued to get a dict object has no attribute _meta error when using the list() method above. However I found this snippet that does the trick
def ValuesQuerySetToDict(vqs):
return [item for item in vqs]
# Usage
data = MyModel.objects.values('id','title','...','...')
data_dict = ValuesQuerySetToDict(data)
data_json = simplejson.dumps(data_dict)

Just to add a few details I've found:
When I tried #ars answer specifying the fields, like:
s_logs = serializers.serialize("json", logs, fields=('user', 'action', 'time'))
I get this:
[{"pk": 520, "model": "audit.auditlog", "fields": {"user": 3, "action": "create", "time":"2012-12-16T12:13:45.540"}}, ... ]
Which was not a simple serialization of the values as I wanted it.
So I tried the solution proposed by #Aaron, converting the valuesqueryset to a list, which didn't work the first time because the default encoder cannot deal with floats or datetime objects.
So I used #Aaron solution but using the JSON encoder that is used by django's serializer (DjangoJSONEncoder) by passing it as a kwarg to simplejson.dumps(), like this:
s_logs = list(logs.values('user', 'ip', 'object_name', 'object_type', 'action', 'time'))
return HttpResponse(simplejson.dumps( s_logs, cls=DjangoJSONEncoder ), mimetype='application/javascript')