Develop Reference

constructing a message format from the fetchall result in python

*New to Programming
Question: I need to use the below "Data" (two rows as arrays) queried from sql and use it to create the message structure below.
data from sql using fetchall()
Data = [[100,1,4,5],[101,1,4,6]]
##expected message structure
message = {
"name":"Tom",
"Job":"IT",
"info": [
{
"id_1":"100",
"id_2":"1",
"id_3":"4",
"id_4":"5"
},
{
"id_1":"101",
"id_2":"1",
"id_3":"4",
"id_4":"6"
},
]
}
I tried to create below method to iterate over the rows and then input the values, this is was just a starting, but this was also not working
def create_message(data)
for row in data:
{
"id_1":str(data[0][0],
"id_2":str(data[0][1],
"id_3":str(data[0][2],
"id_4":str(data[0][3],
}
Latest Code
def create_info(data):
info = []
for row in data:
temp_dict = {"id_1_tom":"","id_2_hell":"","id_3_trip":"","id_4_clap":""}
for i in range(0,1):
temp_dict["id_1_tom"] = str(row[i])
temp_dict["id_2_hell"] = str(row[i+1])
temp_dict["id_3_trip"] = str(row[i+2])
temp_dict["id_4_clap"] = str(row[i+3])
info.append(temp_dict)
return info

Edit: Updated answer based on updates to the question and comment by original poster.
This function might work for the example you've given to get the desired output, based on the attempt you've provided:
def create_info(data):
info = []
for row in data:
temp_dict = {}
temp_dict['id_1_tom'] = str(row[0])
temp_dict['id_2_hell'] = str(row[1])
temp_dict['id_3_trip'] = str(row[2])
temp_dict['id_4_clap'] = str(row[3])
info.append(temp_dict)
return info
For the input:
[[100, 1, 4, 5],[101,1,4,6]]
This function will return a list of dictionaries:
[{"id_1_tom":"100","id_2_hell":"1","id_3_trip":"4","id_4_clap":"5"},
{"id_1_tom":"101","id_2_hell":"1","id_3_trip":"4","id_4_clap":"6"}]
This can serve as the value for the key info in your dictionary message. Note that you would still have to construct the message dictionary.

I want to firstly create a database and then update it as per the values in mongodb

I want to update the value of Entry1 using upsert. I have a sensor that returns the value of Entry1. If sensor is blocked, the value is true. If sensor is not blocked then the value is False.
machineOne = None
oneIn = 1
while True:
global machineOneId
global userId
try:
if Entry1.get_value() and oneIn < 2:
machineOne = Entry1.get_value()
print('entered looopp ONeeeE', machineOne)
machine1 = {
'Entry1': Entry1.get_value(),
'Exit1': Exit1.get_value(),
'id': 'test'
}
result = Machine1.insert_one(machine1)
myquery = {"Entry1": 'true'}
newvalues = {"$set": {"id": result.inserted_id}}
#result = Machine1.insert_one(machine1)
Machine1.update_one(myquery, newvalues)
userId = result.inserted_id
oneIn += 1
print('added', result.inserted_id, oneIn)
elif machineOne:
print('entered looopp', userId)
myquery = {"id": userId}
newvalues = {"$set": {"id": Entry1.get_value()}}
upsert = True
#result = Machine1.insert_one(machine1)
Machine1.update_one(myquery, newvalues)
if Exit1.get_value():
print('added',)
finally:
print('nothings happened', machineOne)
what is expected: i should be able to update the Entry1 from true to false in the same log, displayed in robo3t

Good Afternoon #digs10 ,
I read your post and I think the error is how you locate the document that you want to update.
I remember that MongoDB document primary key is "_id" instead of "id". You could take a look here MongoDB Documents
For what I see in the code (I don't know Python but it is readable), you are referring to the document "Entry1" using the field "id" instead of "_id.
Try modifing the line myquery = {"id": userId} for myquery = {"_id": userId}.
I hope this answer can help you.
Best Regards,
JB
P.S: I saw this question in my email and I took a quick read of it, If I misunderstood it, please let me know.

PyMongo query not returning results although the same query returns results in mongoDB shell

import pymongo
uri = 'mongodb://127.0.0.1:27017'
client = pymongo.MongoClient(uri)
db = client.TeamCity
students = db.students.find({})
for student in students:
print (student)
Python Result:
Blank
MongoDB: Results
db.students.find({})
{ "_id" : ObjectId("5788483d0e5b9ea516d4b66c"), "name" : "Jose", "mark" : 99 }
{ "_id" : ObjectId("57884cb3f7edc1fd01c3511e"), "name" : "Jordan", "mark" : 100
}
import pymongo
uri = 'mongodb://127.0.0.1:27017'
client = pymongo.MongoClient(uri)
db = client.TeamCity
students = db.students.find({})
print (students.count())
Python Result:
0
mongoDB Results
db.students.find({}).count()
2
What am I missing?
For
import pymongo
uri = 'mongodb://127.0.0.1:27017'
client = pymongo.MongoClient(uri)
db = client.TeamCity
students = db.students.find({})
print (students)
Python Result :
So I think it is able to connect to the db successfully but not returning results

Try your pymongo code like so, i.e. changing TeamCity to Teamcity
Print all students:
import pymongo
uri = 'mongodb://127.0.0.1:27017'
client = pymongo.MongoClient(uri)
db = client.Teamcity
students = db.students.find({})
for student in students:
print (student)
Count all students:
import pymongo
uri = 'mongodb://127.0.0.1:27017'
client = pymongo.MongoClient(uri)
db = client.Teamcity
students = db.students.find({})
print (students.count())

I know this question has been answered long ago, but I ran into the same kind of problem today and it happened to have a different reason, so I'm adding an answer here.
Code working on shell:
> db.customers.find({"cust_id": 2345}, {"pending_questions": 1, _id: 0})
{ "pending_questions" : [ 1, 5, 47, 89 ] }
Code not working in PyMongo (cust_id set through a web form):
db.customers.find({"cust_id": cust_id}, {"pending_questions": 1, "_id": 0})
It turned out that the numbers in the shell are being interpreted as ints, whereas the numbers used in Python code are being interpreted by PyMongo as floats, and hence return no matches. This proves the point:
cust_id = int(request.args.get('cust_id'))
db.customers.find({"cust_id": cust_id}, {"pending_questions": 1, "_id": 0})
which produces the result:
[1.0, 5.0, 47.0, 89.0]
The simple solution was to typecast everything to int in the python code. In conclusion, the data type inferred by the shell may be different from the data type inferred by PyMongo and this may be one reason that a find query that returns results on the shell doesn't return anything when run in PyMongo.

pymongo $set on array of subdocuments

I have a pymongo collection in the form of:
{
"_id" : "R_123456789",
"supplier_ids" : [
{
"id" : "S_987654321",
"file_version" : ISODate("2016-03-15T00:00:00Z"),
"latest" : false
},
{
"id" : "S_101010101",
"file_version" : ISODate("2016-03-29T00:00:00Z"),
"latest" : true
}
]
}
when I get new supplier data, if the supplier ID has changed, I want to capture that by setting latest on the previous 'latest' to False and the $push the new record.
$set is not working as I am trying to employ it (commented code after 'else'):
import pymongo
from dateutil.parser import parse
new_id = 'S_323232323'
new_date = parse('20160331')
with pymongo.MongoClient() as client:
db = client.transactions
collection_ids = db.ids
try:
collection_ids.insert_one({"_id": "R_123456789",
"supplier_ids": ({"id": "S_987654321",
"file_version": parse('20160315'),
"latest": False},
{"id": "S_101010101",
"file_version": parse('20160329'),
"latest": True})})
except pymongo.errors.DuplicateKeyError:
print('record already exists')
record = collection_ids.find_one({'_id':'R_123456789'})
for supplier_id in record['supplier_ids']:
print(supplier_id)
if supplier_id['latest']:
print(supplier_id['id'], 'is the latest')
if supplier_id['id'] == new_id:
print(new_id, ' is already the latest version')
else:
# print('setting', supplier_id['id'], 'latest flag to False')
# <<< THIS FAILS >>>
# collection_ids.update_one({'_id':record['_id']},
# {'$set':{'supplier_ids.latest':False}})
print('appending', new_id)
data_to_append = {"id" : new_id,
"file_version": new_date,
"latest": True}
collection_ids.update_one({'_id':record['_id']},
{'$push':{'supplier_ids':data_to_append}})
any and all help is much appreciated.
This whole process seems unnaturally verbose - should I be using a more streamlined approach?
Thanks!

You can try with positional operators.
collection_ids.update_one(
{'_id':record['_id'], "supplier_ids.latest": true},
{'$set':{'supplier_ids.$.latest': false}}
)
This query will update supplier_ids.latest = false, if it's true in document and matches other conditions.
The catch is you have to include field array as part of condition too.
For more information see Update

How to get a dictionary's data from List of Dictionaries

Consider the following List of Dictionaries(retrieved from database):
education = [{
"schools" : "Nelson Mandela Metropolitan University",
"studied_from" : new Date("16/1/2012 02:00:00"),
"studied_to" : new Date("25/1/2055 02:00:00"),
"qualifications" : " National Diploma",
"fields_of_study" : "Industrial engineering"
}, {
"schools" : "Massachusetts Institute of Technology",
"studied_from" : new Date("16/1/2009 02:00:00"),
"studied_to" : new Date("25/1/2020 02:00:00"),
"qualifications" : "B Tech",
"fields_of_study" : "Information Technology"
}]
I am trying to get the data from one dict to be returned.
I have tried the following:
for edu in context.education:
studied_to.append(edu.studied_to)
qualification = edu.qualifications
fields_of_study = edu.fields_of_study
school = edu.schools
recent = max(studied_to)
if recent:
print recent
print qualification
print fields_of_study
This is not working as it always returns the data for the data in the last dict,
I also tried putting the if-statement inside the for loop, but this did not work either as then it runs for all entries.
I ultimately want to use these values to generate on an HTML page.

Your code unfortunately makes no sense at all. You seem to be appending some variables to lists, and redefining other variables with the latest version, without ever actually returning the dictionary you want.
It seems however that what you're actually after is the dict with the maximum "studied_to" value. So, you should just replace all that code with max with a key:
most_recent = max(education, key=lambda s: s['studied_to'])
(Also, do please try and post valid syntax for the language you're using. The first snippet appears to be Javascript, not Python.)

I think you are trying to get the data for the most recent education. You should use a variable to hold the most recent date so far and compare that to the entries in your list. See this as a reference:
max_date = None
for edu in education:
# this line is for initialization
if max_date is None:
max_date = edu["studied_to"]
# check if we have a recent entry at hand
if max_date <= edu["studied_to"]:
max_date = edu["studied_to"]
qualification = edu["qualifications"]
fields_of_study = edu["fields_of_study"]
print max_date
print qualification
print fields_of_study

With some correction of syntax:
from operator import itemgetter
from datetime import datetime
education = [{
"schools" : "Nelson Mandela Metropolitan University",
"studied_from" : datetime.strptime("16/1/2012 02:00:00", "%d/%m/%Y %H:%M:%S"),
"studied_to" : datetime.strptime("25/1/2055 02:00:00", "%d/%m/%Y %H:%M:%S"),
"qualifications" : " National Diploma",
"fields_of_study" : "Industrial engineering"
}, {
"schools" : "Massachusetts Institute of Technology",
"studied_from" : datetime.strptime("16/1/2009 02:00:00", "%d/%m/%Y %H:%M:%S"),
"studied_to" : datetime.strptime("25/1/2020 02:00:00", "%d/%m/%Y %H:%M:%S"),
"qualifications" : "B Tech",
"fields_of_study" : "Information Technology"
}]
most_recent = max(education, key=itemgetter('studied_to'))

Thanks to all, your answers were really helpful. I took Daniel's answer and modified it:
edu = sorted(education, key=lambda studied_to: studied_to)
Now I can call edu[0] to get the value of the most recent education:
>>> edu[0]["studied_to"]
datetime.datetime(2055, 1, 25, 2, 0)
>>> edu[-1]["studied_to"]
datetime.datetime(2020, 1, 25, 2, 0)
No matter what ed[0] will always be the latest education.