Develop Reference

Perform $gte and $lt on the same field _id in MongoDB

db.comments.find({"_id" : {"$gte": ObjectId("6225f932a7bce76715a9f3bd"), "$lt":ObjectId("6225f932a7bce76715a9f3bd")}}).sort({"created_datetime":1}).limit(10).pretty()
I am using this query which should give me the current "6225f932a7bce76715a9f3bd" doc, 4 docs inserted before this and 5 docs inserted after this. But currently when i run this query, i get null result. Where am i going wrong ??

I had no other option but to seperate my queries in order to achieve my expectation.
query = request.args.to_dict()
find_query = {}
find_query["_id"] = {"$lt": ObjectId(query["comment_id"])}
previous_comments = list(db.comments.find(find_query))
find_query["_id"] = {"$gte": ObjectId(query["comment_id"])}
next_comments = list(db.comments.find(find_query))
previous_comments.extend(next_comments)
return {"comments":previous_comments}

Updating a MongoDB document if field doesn't exist

Whenever I updated my insert_one with a new field to use, I had to always delete the old posts in the collection. I know there are manual methods of updating such fields using update_many but I know it's inefficient.
For example:
posts.insert_one({
"id": random.randint(1,10000)
"value1": "value1",
"value2": "value2"
})
I use the following code to check if the document exists or not. How would this work for a field?
if posts.find({'id': 12312}).count() > 0:
I know I can easily overwrite the previous data but I know people won't enjoy having their data wiped every other month.
Is there a way to add the field to a document in Python?

How would this work for a field?
You can use $exists to check whether a field exists in a doc.
In your case, you can combine this with find
find({ 'id':1, "fieldToCheck":{$exists:"true"}})
It will return the doc if it exists with id = 1, fieldToCheck is present in doc with id = 1
You can skip id=1, in that case, it will return all docs where fieldToCheck exists
Is there a way to add the field to a document in Python?
You could use update with new field, it will update if it is present else it will insert.
update({"_id":1}, {field:"x"})
If field is present, it will set to x else it will add with field:x
Beware of update options like multi, upsert

Yes you can you use update command in mongoDB shell to do that. check here
This is the command to use...
db.collection.update({},{$set : {"newfield":1}},false,true)
The above will work in the mongoDB shell. It will add newfield in all the documents, if it is not present.
If you want to use Python, use pymongo.
For python, following command should work
db.collection.update({},{"$set" : {"newfield":1}},False, True)

Thanks to john's answer I have made an entire solution that automatically updates documents without the need to run a task meaning you don't update inactive documents.
import datetime
import pymongo
database = pymongo.MongoClient("mongodb://localhost:27017") # Mongodb connection
db = database.maindb # Database
posts = db.items # Collection within a database
# A schema equivalent function that returns the object
def user_details(name, dob):
return {
"username": name, # a username/id
"dob": dob, # some data
"level": 0, # some other data
"latest_update": datetime.datetime.fromtimestamp(1615640176)
# Must be kept to ensure you aren't doing it that often
}
# The first schema changed for example after adding a new feature
def user_details2(name, dob, cake):
return {
"username": name, # a username/id
"dob": dob, # Some data
"level": 0, # Some other data
"cake": cake, # Some new data that isn't in the document
"latest_update": datetime.datetime.utcnow() # Must be kept to ensure you aren't doing it that often
}
def check_if_update(find, main_document,
collection): # parameters: What you find a document with, the schema dictionary, then the mongodb collection
if collection.count_documents(find) > 0: # How many documents match, only proceed if it exists
fields = {} # Init a dictionary
for x in collection.find(find): # You only want one for this to work
fields = x
if "latest_update" in fields: # Just in case it doesn't exist yet
last_time = fields["latest_update"] # Get the time that it was last updated
time_diff = datetime.datetime.utcnow() - last_time # Get the time difference between the utc time now and the time it was last updated
if time_diff.total_seconds() < 3600: # If the total seconds of the difference is smaller than an hour
print("return")
return
db_schema = main_document # Better naming
db_schema["_id"] = 0 # Adds the _id schema_key into the dictionary
if db_schema.keys() != fields:
print("in")
for schema_key, schema_value in db_schema.items():
if schema_key not in fields.keys(): # Main key for example if cake is added and doesn't exist in db fetched fields
collection.update_one(find, {"$set": {schema_key: schema_value}})
else: # Everything exists and you want to check for if a dictionary within that dictionary is changed
try:
sub_dict = dict(schema_value) # Make the value of it a dictionary
# It exists in the schema dictionary but not in the db fetched document
for key2, value2 in sub_dict.items():
if key2 not in fields[schema_key].keys():
new_value = schema_value
new_value[
key2] = value2 # Adding the key and value from the schema dictionary that was added
collection.update_one(find,
{"$set": {schema_key: new_value}})
# It exists in the db fetched document but not in the schema dictionary
for key2, value2 in fields[schema_key].items():
if key2 not in sub_dict.keys():
new_dict = {} # Get all values, filter then so that only the schema existent ones are passed back
for item in sub_dict:
if item != key2:
new_dict[item] = sub_dict.get(item)
collection.update_one(find, {"$set": {schema_key: new_dict}})
except: # Wasn't a dict
pass
# You removed a value from the schema dictionary and want to update it in the db
for key2, value2 in fields.items():
if key2 not in db_schema:
collection.update_one(find, {"$unset": {key2: 1}})
else:
collection.insert_one(main_document) # Insert it because it doesn't exist yet
print("start")
print(posts.find_one({"username": "john"}))
check_if_update({"username": "john"}, user_details("john", "13/03/2021"), posts)
print("inserted")
print(posts.find_one({"username": "john"}))
check_if_update({"username": "john"}, user_details2("john", "13/03/2021", "Lemon drizzle"), posts)
print("Results:")
print(posts.find_one({"username": "john"}))
It is available as a gist

Why python dictionary only update the last value appended?

I have a mongodb database and I retrieve some information from it. And I try to append it to a python dictionary using a for loop, but it only appends the last row.
here's my code:
import pymongo
import datetime
# #app.route("/bemployees", methods=["POST", "GET"])
def retrieve_all_documents():
client = pymongo.MongoClient(
"<url-removed>"
)
# database
db = client.cdap
# collection (table)
collection = db.predicted_values
cursor = collection.find({})
documents_ = {}
for document in cursor:
documents_.update(document)
print(document) # prints all the records
print(documents_) # only prints the last record
return documents_
can someone please help me?

I assume that each document is a dict that has all the same keys. I think you need a list to store.
documents_ = []
for i in cursor:
dic = {}
dic.update(i)
documents_.append(dic)
print(documents_)
Please share your response data in the question that will helpful for answer this question thank you

Cosmos DB - Delete Document with Python

In this SO question I had learnt that I cannot delete a Cosmos DB document using SQL.
Using Python, I believe I need the DeleteDocument() method. This is how I'm getting the document ID's that are required (I believe) to then call the DeleteDocument() method.
# set up the client
client = document_client.DocumentClient()
# use a SQL based query to get a bunch of documents
query = { 'query': 'SELECT * FROM server s' }
result_iterable = client.QueryDocuments('dbs/DB/colls/coll', query, options)
results = list(result_iterable);
for x in range(0, len (results)):
docID = results[x]['id']
Now, at this stage I want to call DeleteDocument().
The inputs into which are document_link and options.
I can define document_link as something like
document_link = 'dbs/DB/colls/coll/docs/'+docID
And successfully call ReadAttachments() for example, which has the same inputs as DeleteDocument().
When I do however, I get an error...
The partition key supplied in x-ms-partitionkey header has fewer
components than defined in the the collection
...and now I'm totally lost
UPDATE
Following on from Jay's help, I believe I'm missing the partitonKey element in the options.
In this example, I've created a testing database, it looks like this
So I think my partition key is /testPART
When I include the partitionKey in the options however, no results are returned, (and so print len(results) outputs 0).
Removing partitionKey means that results are returned, but the delete attempt fails as before.
# Query them in SQL
query = { 'query': 'SELECT * FROM c' }
options = {}
options['enableCrossPartitionQuery'] = True
options['maxItemCount'] = 2
options['partitionKey'] = '/testPART'
result_iterable = client.QueryDocuments('dbs/testDB/colls/testCOLL', query, options)
results = list(result_iterable)
# should be > 0
print len(results)
for x in range(0, len (results)):
docID = results[x]['id']
print docID
client.DeleteDocument('dbs/testDB/colls/testCOLL/docs/'+docID, options=options)
print 'deleted', docID

According to your description, I tried to use pydocument module to delete document in my azure document db and it works for me.
Here is my code:
import pydocumentdb;
import pydocumentdb.document_client as document_client
config = {
'ENDPOINT': 'Your url',
'MASTERKEY': 'Your master key',
'DOCUMENTDB_DATABASE': 'familydb',
'DOCUMENTDB_COLLECTION': 'familycoll'
};
# Initialize the Python DocumentDB client
client = document_client.DocumentClient(config['ENDPOINT'], {'masterKey': config['MASTERKEY']})
# use a SQL based query to get a bunch of documents
query = { 'query': 'SELECT * FROM server s' }
options = {}
options['enableCrossPartitionQuery'] = True
options['maxItemCount'] = 2
result_iterable = client.QueryDocuments('dbs/familydb/colls/familycoll', query, options)
results = list(result_iterable);
print(results)
client.DeleteDocument('dbs/familydb/colls/familycoll/docs/id1',options)
print 'delete success'
Console Result:
[{u'_self': u'dbs/hitPAA==/colls/hitPAL3OLgA=/docs/hitPAL3OLgABAAAAAAAAAA==/', u'myJsonArray': [{u'subId': u'sub1', u'val': u'value1'}, {u'subId': u'sub2', u'val': u'value2'}], u'_ts': 1507687788, u'_rid': u'hitPAL3OLgABAAAAAAAAAA==', u'_attachments': u'attachments/', u'_etag': u'"00002100-0000-0000-0000-59dd7d6c0000"', u'id': u'id1'}, {u'_self': u'dbs/hitPAA==/colls/hitPAL3OLgA=/docs/hitPAL3OLgACAAAAAAAAAA==/', u'myJsonArray': [{u'subId': u'sub3', u'val': u'value3'}, {u'subId': u'sub4', u'val': u'value4'}], u'_ts': 1507687809, u'_rid': u'hitPAL3OLgACAAAAAAAAAA==', u'_attachments': u'attachments/', u'_etag': u'"00002200-0000-0000-0000-59dd7d810000"', u'id': u'id2'}]
delete success
Please notice that you need to set the enableCrossPartitionQuery property to True in options if your documents are cross-partitioned.
Must be set to true for any query that requires to be executed across
more than one partition. This is an explicit flag to enable you to
make conscious performance tradeoffs during development time.
You could find above description from here.
Update Answer:
I think you misunderstand the meaning of partitionkey property in the options[].
For example , my container is created like this:
My documents as below :
{
"id": "1",
"name": "jay"
}
{
"id": "2",
"name": "jay2"
}
My partitionkey is 'name', so here I have two paritions : 'jay' and 'jay1'.
So, here you should set the partitionkey property to 'jay' or 'jay2',not 'name'.
Please modify your code as below:
options = {}
options['enableCrossPartitionQuery'] = True
options['maxItemCount'] = 2
options['partitionKey'] = 'jay' (please change here in your code)
result_iterable = client.QueryDocuments('dbs/db/colls/testcoll', query, options)
results = list(result_iterable);
print(results)
Hope it helps you.

Using the azure.cosmos library:
install and import azure cosmos package:
from azure.cosmos import exceptions, CosmosClient, PartitionKey
define delete items function - in this case using the partition key in query:
def deleteItems(deviceid):
client = CosmosClient(config.cosmos.endpoint, config.cosmos.primarykey)
# Create a database if not exists
database = client.create_database_if_not_exists(id=azure-cosmos-db-name)
# Create a container
# Using a good partition key improves the performance of database operations.
container = database.create_container_if_not_exists(id=container-name, partition_key=PartitionKey(path='/your-pattition-path'), offer_throughput=400)
#fetch items
query = f"SELECT * FROM c WHERE c.device.deviceid IN ('{deviceid}')"
items = list(container.query_items(query=query, enable_cross_partition_query=False))
for item in items:
container.delete_item(item, 'partition-key')
usage:
deviceid=10
deleteItems(items)
github full example here: https://github.com/eladtpro/python-iothub-cosmos

How come I can't insert into Mongo? It used to work

con = pymongo.Connection(MONGO_DOC_HOST, MONGO_DOC_PORT)
db = con.testdb
datasets = db.datasets
for post in db.datasets.find({"test_set":"xyz"}).sort("num_favs",pymongo.DESCENDING).limit(2):
print post #this works, and it prints fine!
post = {"hi":"abc"}
mongo_id = datasets.insert(post)
When I do datasets.insert, and print out the mongo_id. The id prints!
However, when I do: db.datasets.find().count() in the mongo console, the count is still the same...
Weird. When I do this in console..I get this error:
> db.datasets.insert({"gooder":"absdlk"})
E11000 duplicate key error index: fabletest.datasets.$flickr_original_1 dup key: { : null }
That's weird, I didn't index "gooder" at all.

are you definitely hitting the same database ("testdb") in both cases?
the default db in the shell is "test"