I have a local MongoDB database with multiple collections.
I use pymongo in jupyter notebook, what I would like to do is run a query FULLTEXT looking for the data on all the collections present.
is it possible to do this? if so how could i proceed?
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017/')
client.list_database_names()
out: ['admin', 'config', 'local']
In local I have a more collection:
this is what I do with just one collection
db = client["local"]
firstdb = db["firstdb"]
result = db.firstdb.find({"email": {"$regex":"test","$options": 'i'}})
for item in result:
print(item['email'],item['log'])
in essence I would like to perform an email query also on secondb, thirdb, fourthdb, etc. etc.
no one can help me? basically I have to do a FULLTEXT query on all collections.
the only solution I found is this:
result = db.firstdb.find({"email": {"$regex":"test","$options": 'i'}})
result1 = db.secondb.find({"email": {"$regex":"test","$options": 'i'}})
result2 = db.thirdb.find({"email": {"$regex":"test","$options": 'i'}})
for item in result:
print(item['email'],item['log'])
for item in result1:
print(item['email'],item['date'])
for item in result2:
print(item['email'],item['account'])
but I'm not sure I'm going on the right track!
I thank anyone who can help me!
PS: I would not like to change the structure of the collections, the problem could occur in other Database
Related
I have the following:
from pymongo import MongoClient
client = MongoClient()
db=client.localhost
collection=db['accounts']
db.collection.remove({})
cursor = collection.find({})
for document in cursor:
print(document)
This second part is to just print all the documents in the collection. However, the collection isn't clearing every time I rerun the program. Does anyone know why?
Just do this
db.accounts.drop()
Instead of
db.collection.remove({})
you can try this
collection.delete_many({})
Hope that solves your problem.
Instead of doing this
db.collection.remove({})
do this
db.accounts.remove({})
Also you won't need this line collection=db['accounts']
If you want dynamic collection name, you can do the following:
collection_name = 'accounts'
getattr(db, collection_name).remove({})
I'm trying to insert documents in bulk, I have created a unique index in my collection and want to skip documents which are duplicate while doing bulk insertion. This can be accomplished with native mongodb function:
db.collection.insert(
<document or array of documents>,
{
ordered: <boolean>
}
)
I want to accomplish this with mongoengine, If anybody knows how to achieve this, please answer the question, thanks.
If you have a class like this:
class Foo(db.Document):
bar= db.StringField()
meta = {'indexes': [{'fields': ['bar'], 'unique': True}]}
And having a list with Foo instances foos=[Foo('a'), Foo('a'), Foo('a')]
and trying Foo.objects.insert(foos) you will get mongoengine.errors.NotUniqueError
1st woraround would be delete index from mongodb, insert duplicates, and than ensure index with {unique : true, dropDups : true}
2nd workaround would be using underlying pymongo API for bulk ops: https://docs.mongodb.com/manual/reference/method/db.collection.initializeOrderedBulkOp/#db.collection.initializeOrderedBulkOp
For now I am using raw pymongo from mongoengine as a workaround for this. This is the 2nd workaround that #Alexey Smirnov mentioned. So for a mongoengine Document class DocClass you will access the underlying pymongo collection and execute query like below:
from pymongo.errors import BulkWriteError
try:
doc_list = [doc.to_mongo() for doc in me_doc_list] # Convert ME objects to what pymongo can understand
DocClass._get_collection().insert_many(doc_list, ordered=False)
except BulkWriteError as bwe:
print("Batch Inserted with some errors. May be some duplicates were found and are skipped.")
print(f"Count is {DocClass.objects.count()}.")
except Exception as e:
print( { 'error': str(e) })
I want to write a function to return all the documents contained in mycollection in mongodb
from pymongo import MongoClient
if __name__ == '__main__':
client = MongoClient("localhost", 27017, maxPoolSize=50)
db=client.mydatabase
collection=db['mycollection']
cursor = collection.find({})
for document in cursor:
print(document)
However, the function returns: Process finished with exit code 0
Here is the sample code which works fine when you run from command prompt.
from pymongo import MongoClient
if __name__ == '__main__':
client = MongoClient("localhost", 27017, maxPoolSize=50)
db = client.localhost
collection = db['chain']
cursor = collection.find({})
for document in cursor:
print(document)
Please check the collection name.
pymongo creates a cursor. Hence you'll get the object 'under' the cursor. To get all objects in general try:
list(db.collection.find({}))
This will force the cursor to iterate over each object and put it in a list()
Have fun...
I think this will work fine in your program.
cursor = db.mycollection # choosing the collection you need
for document in cursor.find():
print (document)
it works fine for me,try checking the exact database name and collection name.
and try changing from db=client.mydatabase to db=client['mydatabase'] .
If your database name is such that using attribute style access won’t work (like test-database), you can use dictionary style access instead.
source !
I am migrating some data from other databases , so i am using raw sql queries for inserting data into database . But i don't know how to get last inserted id from raw sql queries in django. I have tried this
affected_count1=cursor2.execute("table')")
and
SELECT IDENT_CURRENT(‘MyTable’)
but it gives me the error of "(1305, 'FUNCTION pydev.SCOPE_IDENTITY does not exist')"
So please tell me how can i get the last inserted id in raw sq l queries in django
You can get latest create obj like this:
obj = Foo.objects.latest('id')
more info here
Try this
LastInsertId = (TableName.objects.last()).id
In Django 1.6
obj = Foo.objects.latest('id')
obj = Foo.objects.earliest('id')
How to check in PyMongo if collection exists and if exists empty (remove all from collection)?
I have tried like
collection.remove()
or
collection.remove({})
but it doesn't delete collection. How to do that ?
Sample code in Pymongo with comment as explanation:
from pymongo import MongoClient
connection = MongoClient('localhost', 27017) #Connect to mongodb
print(connection.database_names()) #Return a list of db, equal to: > show dbs
db = connection['testdb1'] #equal to: > use testdb1
print(db.list_collection_names()) #Return a list of collections in 'testdb1'
print("posts" in db.list_collection_names()) #Check if collection "posts"
# exists in db (testdb1)
collection = db['posts']
print(collection.count() == 0) #Check if collection named 'posts' is empty
collection.drop() #Delete(drop) collection named 'posts' from db and all documents contained.
You should use .drop() instead of .remove(), see documentation for detail: http://api.mongodb.org/python/current/api/pymongo/collection.html#pymongo.collection.Collection.drop
=====
Sorry for misunderstanding your question.
To check if a collection exists, use method collection_names on database:
>>> collection_name in database.list_collection_names()
To check if a collection is empty, use:
>>> collection.count() == 0
both will return True or False in result.
Have you tried this:
db.collection.remove();