I have a mongodb database and I retrieve some information from it. And I try to append it to a python dictionary using a for loop, but it only appends the last row.
here's my code:
import pymongo
import datetime
# #app.route("/bemployees", methods=["POST", "GET"])
def retrieve_all_documents():
client = pymongo.MongoClient(
"<url-removed>"
)
# database
db = client.cdap
# collection (table)
collection = db.predicted_values
cursor = collection.find({})
documents_ = {}
for document in cursor:
documents_.update(document)
print(document) # prints all the records
print(documents_) # only prints the last record
return documents_
can someone please help me?
I assume that each document is a dict that has all the same keys. I think you need a list to store.
documents_ = []
for i in cursor:
dic = {}
dic.update(i)
documents_.append(dic)
print(documents_)
Please share your response data in the question that will helpful for answer this question thank you
Related
Is it possible to update with the scan() method from Python's elasticsearch-dsl library?
For example:
search = Search(index=INDEX).query(query).params(request_timeout=6000)
print('Scanning Query and updating.')
for hit in search.scan():
_id = hit.meta.id
# sql query to get the record from th database
record = get_record_by_id(_id)
hit.column1 = record['column1']
hit.column2 = record['column2']
# not sure what use to update here
This can be done by using the elasticsearch client:
for hit in search.scan():
_id = hit.meta.id
print(_id)
# get data from database
record = get_record_by_id(_id)
body = dict(record)
body.pop('_id')
# update the elastichsearch client
elastic_client.update(
index=INDEX,
id=_id,
body={"doc": body}
)
Context: I'm trying to query data from SQL Server via Pyodbc & use looping logic to read the results into the variables in the query block below.
Question(s): Can someone please help me modify the code block below so that it properly populates the variables via the looping logic? I suspect because I'm using the fetchall() method on the query cursor that each result row transforms into a tuple within a list -- which then seems to make the looping logic below it useless. Can someone please suggest an alternative solution?
from constantcontact import ConstantContact
from constantcontact import Contact
import requests
import json
import pyodbc
username = 'REDACTED'
password = 'REDACTED'
sample_contact_connection = pyodbc.connect("Driver={ODBC Driver 13 for SQL Server};""Server=PC;""Database=leadgen_sandbox;""Username="+username+";""Password="+password+";""Trusted_Connection=yes;")
sample_contact_cursor = sample_contact_connection.cursor()
sample_contact_query = "SELECT first_name,last_name,title,company_name,email_address FROM leadgen_sandbox.dbo.sample_contacts"
sample_contact_connection.autocommit = True
sample_contact_cursor.execute(sample_contact_query)
print(sample_contact_cursor.fetchall())
constantcontact = ConstantContact('REDACTED','REDACTED')
list_id = '1816761971'
for [first_name, last_name, title, company_name, email_address] in sample_contact_cursor.fetchall():
new_contact = Contact()
new_contact.set_first_name(''+first_name+'')
new_contact.set_last_name(''+last_name+'')
new_contact.set_job_title(''+title+'')
new_contact.set_company_name(''+company_name+'')
new_contact.add_list_id(''+list_id+'')
new_contact.set_email_address(''+email_address+'')
response = constantcontact.post_contacts(new_contact)
response_text = json.dumps(response, indent = 4, sort_keys = True)
print(response_text)
sample_contact_connection.close()
When you called sample_contact_cursor.fetchall() , you already exhausted the contents of the cursor. So, it's content would no longer be available for the loop. Removing the print before the loop would fix this. Also, the .fetchall() is redundant in the loop as each row would be read in the for loop one by one anyways. You could write this way as well:
for [first_name, last_name, title, company_name, email_address] in sample_contact_cursor:
new_contact = Contact()
new_contact.set_first_name(''+first_name+'')
new_contact.set_last_name(''+last_name+'')
#Write your remaining code
If you do need to first print the cursor, and then run the loop, you have to execute the cursor once agin before the loop like this:
sample_contact_cursor.execute(sample_contact_query)
print(sample_contact_cursor.fetchall())
constantcontact = ConstantContact('REDACTED','REDACTED')
list_id = '1816761971'
sample_contact_cursor.execute(sample_contact_query)
for [first_name, last_name, title, company_name, email_address] in sample_contact_cursor:
new_contact = Contact()
new_contact.set_first_name(''+first_name+'')
new_contact.set_last_name(''+last_name+'')
#Write your remaining code
I'm facing a problem with continuation when querying items from CosmosDB.
I've already tried the following solution but with no success. I'm only able to query the first 10 results of a page even though I get a token that is not NULL.
The token has a size of 10733 bytes and looks like this.
{"token":"+RID:gtQwAJ9KbavOAAAAAAAAAA==#RT:1#TRC:10#FPP:AggAAAAAAAAAAGoAAAAAKAAAAAAAAAAAAADCBc6AEoAGgAqADoASgAaACoAOgBKABoAKgA6AE4AHgAuAD4ASgAeACoAPgBOAB4ALgA+AE4AHgAqAD4ASgAeAC4APgBOAB4ALgA+AE4AIgA2AEYAFgAmADYARgAaACYAPgBKABYAKgA6AE4AHgAuAD4ATgAeAC4APgBOAB4ALgA+AE4AIgAuAD4ATgAeAC4APgBOACIAMgA+AFIAIgAyAD4AUgAmADIAQgAWACIALgBCABIAIgAyAEIAEgAiADIAQgAOACYANgBKAB4AJgA6AEYAGgAqADoATgAeAC4APgB....etc...etc","range":{"min":"","max":"05C1BF3FB3CFC0"}}
Code looks like this. Function QueryDocuments did not work. Instead I had to use QueryItems.
options = {}
options['enableCrossPartitionQuery'] = True
options['maxItemCount'] = 10
q = client.QueryItems(collection_link, query, options)
results_1 = q._fetch_function(options)
#this is a string representing a JSON object
token = results_1[1]['x-ms-continuation']
data = list(q._fetch_function({'maxItemCount':10,'enableCrossPartitionQuery':True, 'continuation':token}))
Is there a solution to this? Thanks for your help.
Please use pydocumentdb package and refer to below sample code.
from pydocumentdb import document_client
endpoint = "https://***.documents.azure.com:443/";
primaryKey = "***";
client = document_client.DocumentClient(endpoint, {'masterKey': primaryKey})
collection_link = "dbs/db/colls/coll"
query = "select c.id from c"
query_with_optional_parameters = [];
q = client.QueryDocuments(collection_link, query, {'maxItemCount': 2})
results_1 = q._fetch_function({'maxItemCount': 2})
print(results_1)
token = results_1[1]['x-ms-continuation']
results_2 = q._fetch_function({'maxItemCount': 2, 'continuation': token})
print(results_2)
Output:
I have a dictionary to be inserted dynamically in MongoDB.
Current MongoDB document-
"Customers":{
"Payment":{
"CustomerDetails":{
"Source":"Visa Card",
"Name" :"John",
}
}
}
The document that I am trying to insert into this through python dictionary object -
final= {"CustomerPayable":["Month":"Feb-1FN-2018","Details":
["Code":"ghg23","AmtPayable": "123.00"]]}
The query I am trying -
db.collection.update({"UserID":UserID},{ '$push':{
'Customers.Payment.Output':final}})
I wanted the dynamic field of "Output" to be created through the above query. Expected output-
"Customers":{
"Payment":{
"CustomerDetails":{
"Source":"Visa Card",
"Name" :"John",
},
"Output":{"CustomerPayable":["Month":"Feb-1FN-2018",Details:
["Code":"ghg23","AmtPayable": "123.00"]]}
}
}
Any help is great.Thanks in advance
The following code should achieve your desired results.
from pymongo import MongoClient
client = MongoClient()
db = client.stackoverflow
collection = db.stackoverflow
a = {"Customers":{ "Payment":{ "CustomerDetails":{ "Source":"Visa Card", "Name" :"John"}}}}
collection.insert(a)
# Prints object before update.
cur = collection.find_one({"Customers.Payment.CustomerDetails.Name":"John"})
print(cur)
final = {"CustomerPayable":{"Month":"Feb-1FN-2018","Details":
{"Code":"ghg23","AmtPayable": "123.00"}}}
collection.update({"Customers.Payment.CustomerDetails.Name":"John"},
{'$push':{'Customers.Payment.Output':final}})
# Prints object after update.
cur = collection.find_one({"Customers.Payment.CustomerDetails.Name":"John"})
print(cur)
A couple things wrong with your code are:
In you final declaration you tried to use dictionary syntax inside of a list.
In your update query you don't have a field called UserID for I changed it to query on Name
Anyways, I hope this helps.
In short, how do i
var="TableName"
models.var.query.all()
explanation
My goal is to allow the user to change the order of list of items.
I set up an ajax call that sends an array of id's to the api below.
It works if i hard code the query, and make a api view per table.
my problem is that i want "table" to fill in this line
models.table.query.filter_by(id=item).first()
to complete the query.
here is the view api which gives me an error "no attribute 'table'"
#app.route('/order/<table>')
def order(table):
# import pdb;pdb.set_trace()
sortedItems = request.args.listvalues()[0]
o=1
import pdb;pdb.set_trace()
for item in sortedItems:
grab = models.table.query.filter_by(id=item).first()
grab.order=o
o=o+1
db.session.commit()
return jsonify(result=sortedItems)
You can use getattr():
>>> var = 'table_name'
>>> table = getattr(models, var)
>>> table.query.filter_by(id=item).first()
getattr() will raise an AttributeError if the attribute your trying to get does not exist.
Example for your order()-function:
#app.route('/order/<table>')
def order(table):
sortedItems = request.args.listvalues()[0]
o=1
table = getattr(models, table)
for item in sortedItems:
grab = table.query.filter_by(id=item).first()
grab.order=o
o=o+1
db.session.commit()
return jsonify(result=sortedItems)