While writing python script for the api in mongodb..
We have..
new_posts = [{ 'name': 'A', 'age': 17, 'marks': 97, 'school': 'School1' },
{ 'name': 'B', 'age': 18, 'marks': 95, 'school': 'School2' },
{ 'name': 'C', 'age': 19, 'marks': 97, 'school': 'School2' }]
db.posts.insert( new_posts )
We create indexes as follows..
db.posts.create_index([('name',1),('school',1)],unique=True)
Now we perform two operations..
db.posts.update({ 'name':'A', 'age': 17, 'school': 'School3' },
{ 'name':'D', 'age': 17, 'marks': 70, 'school': 'School1' },
upsert=True )
db.posts.update({ 'name':'A', 'age': 17, 'school': 'School1' },
{ 'name':'A', 'age': 17, 'marks': 60, 'school': 'School1' },
upsert=True )
What does the update() returns here? How can we find out weather the document is inserted into the db or existing document is updated?
Can we do something like..
post1 = db.posts.update({ 'name':'A', 'age': 17, 'school': 'School3' },
{ 'name':'D', 'age': 17, 'marks': 70, 'school': 'School1' },
upsert=True )
post2 = db.posts.update({ 'name':'A', 'age': 17, 'school': 'School1' },
{ 'name':'A', 'age': 17, 'marks': 60, 'school': 'School1' },
upsert=True )
print post1
print post2
As the docs for update say, the method returns:
A document (dict) describing the effect of the update or None if write acknowledgement is disabled.
Just try it and print the return value to see what's available. You'll see something like:
{u'syncMillis': 0, u'ok': 1.0, u'err': None, u'writtenTo': None,
u'connectionId': 190, u'n': 1, u'updatedExisting': True}
The updatedExisting field is what you're looking for.
Related
I have this list of dictionary and I would like to get those with the same exact value of 'name' and 'school' into a new list and also getting their 'age' merged into a list as well and the rest of the dictionary that is not identical to just add into the list as per usual..
Here is an example of the list of dictionary
[{'name': 'Jane', 'age':12, 'school': 'SIT'}, {'name': 'John', 'age':13, 'school': 'SMU'},{'name': 'Jane', 'age':14, 'school': 'SIT'}, {'name': 'Jane', 'age':16, 'school': 'SIT'}, {'name': 'John', 'age':13, 'school': 'NUS'}]
and I would like it to make it into something like this..
[{'name': 'Jane', 'age': [12,14,16], 'school': 'SIT'}, {'name': 'John', 'age': 13, 'school': 'SMU'}, {'name': 'John', 'age':13, 'school': 'NUS'}]
using Python.. please help!
tried using counter, loops but still can't get it to work..
You could use itertools.groupby().
Example:
import itertools
from pprint import pprint
data = [{'name': 'Jane', 'age':12, 'school': 'SIT'}, {'name': 'John', 'age':13, 'school': 'SMU'},{'name': 'Jane', 'age':14, 'school': 'SIT'}, {'name': 'Jane', 'age':16, 'school': 'SIT'}, {'name': 'John', 'age':13, 'school': 'NUS'}]
keyfunc = lambda x: (x["name"], x["school"])
# needs to be sorted to use groupby
data.sort(key=keyfunc)
output = []
for k,v in itertools.groupby(data, key=keyfunc):
this_group = {
"name": k[0],
"school": k[1],
"age": [i["age"] for i in v],
}
output.append(this_group)
pprint(output)
The output is:
[{'age': [12, 14, 16], 'name': 'Jane', 'school': 'SIT'},
{'age': [13], 'name': 'John', 'school': 'NUS'},
{'age': [13], 'name': 'John', 'school': 'SMU'}]
If you wish to go with the solution based on a buffer dictionary, please check out the dict.setdefault() method.
Example:
buffer = {}
for i in data:
buffer.setdefault((i["name"], i["school"]), []).append(i["age"])
For reference:
https://docs.python.org/3/library/itertools.html#itertools.groupby
https://docs.python.org/3/library/stdtypes.html#dict.setdefault
x = [{'name': 'Jane', 'age':12, 'school': 'SIT'}, {'name': 'John', 'age':13, 'school': 'SMU'},{'name': 'Jane', 'age':14, 'school': 'SIT'}, {'name': 'Jane', 'age':16, 'school': 'SIT'}, {'name': 'John', 'age':13, 'school': 'NUS'}]
new_x = {}
for r in x:
if r['name'] in new_x.keys():
if not isinstance(new_x[r['name']]['age'], list):
new_x[r['name']]['age'] = [new_x[r['name']]['age']]
if r['age'] not in new_x[r['name']]['age']:
new_x[r['name']]['age'].append(r['age'])
else:
new_x[r['name']] = {'age': r['age'], 'school': r['school']}
z = [v.update(name=k) for k, v in new_x.items()]
z = [v for k, v in new_x.items()]
Here is a universal solution to your problem. Only name and school are considered "special". All other keys, like age are converted to list when a new value has to be added.
l = [
{"name": "Jane", "age": 12, "school": "SIT"},
{"name": "John", "age": 13, "school": "SMU"},
{"name": "Jane", "age": 14, "school": "SIT"},
{"name": "Jane", "age": 16, "school": "SIT"},
{"name": "John", "age": 13, "school": "NUS"},
]
r = {}
for x in l:
id = f"{x['name']}-{x['school']}"
if id in r:
for k,v in x.items():
if k not in ["name", "school"]:
if k in r[id]:
if isinstance(r[id][k], list):
r[id][k].append(v)
else:
r[id][k] = [r[id][k], v]
else:
r[id][k] = v
else:
r[id] = x
result = [x for x in r.values()]
I checked a few other posts and either they didn't contain the information I need or I didn't understand them. I want to make this program print the sentence for every entry in the nested dictionary, and maybe also make a function to do this as well (not familiar with these yet).
I know it will use a for loop but what I can't figure out is how to configure the keys(?).
people = {
1: {
'name': 'David Wallace',
'age': 50,
'occupation': 'CFO',
'ethnicity': 'American',
'location': 'New York'
},
2: {
'name': 'Michael',
'age': 42,
'occupation': 'Regional Manager',
'ethnicity': 'American',
'location': 'Scranton, Pennsylvania'
},
3: {
'name': 'Jim',
'age': 27,
'occupation': 'Sales Rep',
'ethnicity': 'American',
'location': 'Scranton, Pennsylvania'
}
}
print('{name} is a {age} year-old {ethnicity} {occupation} from {location}.'.format(**people))
You're really treating the top-level dict more like a list, so you can write a for loop traversing the top-level like so:
people = {
1: {
'name': 'David Wallace',
'age': 50,
'occupation': 'CFO',
'ethnicity': 'American',
'location': 'New York'
},
2: {
'name': 'Michael',
'age': 42,
'occupation': 'Regional Manager',
'ethnicity': 'American',
'location': 'Scranton, Pennsylvania'
},
3: {
'name': 'Jim',
'age': 27,
'occupation': 'Sales Rep',
'ethnicity': 'American',
'location': 'Scranton, Pennsylvania'
}
}
for person in people.values():
print('{name} is a {age} year-old {ethnicity} {occupation} from {location}.'.format(**person))
The full reference for Python dictionaries is here: https://docs.python.org/3/library/stdtypes.html#dict.items
Edit: Thanks to user Chris Charley for the suggestion to use people.values() instead of people.items()
I am trying to figure out how to filter for the dictionaries that have a status of "awaiting_delivery". I am not sure how to do this (or if it is impossible). I am new to python and programming. I am using Python 3.8.5 on VS Code on Ubuntu 20.04. The data below is sample data that I created that resembles json data from an API. Any help on how to filter for "status" would be great. Thank you.
nested_dict = {
'list_data': [
{
'id': 189530,
'total': 40.05,
'user_data': {
'id': 1001,
'first_name': 'jane',
'last_name': 'doe'
},
'status': 'future_delivery'
},
{
'id': 286524,
'total': 264.89,
'user_data': {
'id': 1002,
'first_name': 'john',
'last_name': 'doe'
},
'status': 'awaiting_delivery'
},
{
'id': 368725,
'total': 1054.98,
'user_data': {
'id': 1003,
'first_name': 'chris',
'last_name': 'nobody'
},
'status': 'awaiting_delivery'
},
{
'id': 422955,
'total': 4892.78,
'user_data': {
'id': 1004,
'first_name': 'mary',
'last_name': 'madeup'
},
'status': 'future_delivery'
}
],
'current_page': 1,
'total': 2,
'first': 1,
'last': 5,
'per_page': 20
}
#confirm that nested_dict is a dictionary
print(type(nested_dict))
#create a list(int_list) from the nested_dict dictionary
int_list = nested_dict['list_data']
#confirm that int_list is a list
print(type(int_list))
#create the int_dict dictionary from the int_list list
for int_dict in int_list:
print(int_dict)
#this is my attempt at filtering the int_dict dictionar for all orders with a status of awaiting_delivery
for order in int_dict:
int_dict.get('status')
print(order)
Output from Terminal Follows:
<class 'dict'>
<class 'list'>
{'id': 189530, 'total': 40.05, 'user_data': {'id': 1001, 'first_name': 'jane', 'last_name': 'doe'}, 'status': 'future_delivery'}
{'id': 286524, 'total': 264.89, 'user_data': {'id': 1002, 'first_name': 'john', 'last_name': 'doe'}, 'status': 'awaiting_delivery'}
{'id': 368725, 'total': 1054.98, 'user_data': {'id': 1003, 'first_name': 'chris', 'last_name': 'nobody'}, 'status': 'awaiting_delivery'}
{'id': 422955, 'total': 4892.78, 'user_data': {'id': 1004, 'first_name': 'mary', 'last_name': 'madeup'}, 'status': 'future_delivery'}
id
total
user_data
status
You can obtain a filtered list of dicts by doing conditional list comprehension on your list of dicts:
# filter the data
list_data_filtered = [entry for entry in nested_dict['list_data']
if entry['status'] == 'awaiting_delivery']
# print out the results
for entry in list_data_filtered:
print(entry)
# results
# {'id': 286524, 'total': 264.89, 'user_data': {'id': 1002, 'first_name': 'john', 'last_name': 'doe'}, 'status': 'awaiting_delivery'}
# {'id': 368725, 'total': 1054.98, 'user_data': {'id': 1003, 'first_name': 'chris', 'last_name': 'nobody'}, 'status': 'awaiting_delivery'}
I am trying to ge around with APIs in general. To test this I coded this little snippet of code to get a list of all the channels on the Swedish national public service radio, and I want to print the ID and NAME of the channels:
import requests as rq
import json
from pprint import pprint
resp = rq.get('http://api.sr.se/api/v2/channels?
format=json&indent=TRUE')
respjson = json.loads(resp.text)
pprint (respjson['id'])
And I get the error
File "sr-api.py", line 9, in <module>
pprint (respjson['id']['name'])
KeyError: 'id'
The (abbreviated) 'respjson' looks like this
{'channels': [{'channeltype': 'Rikskanal',
'color': '31a1bd',
'id': 132,
'image': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg',
'liveaudio': {'id': 132,
'statkey': '/app/direkt/p1[k(132)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/132.mp3'},
'name': 'P1',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=132',
'siteurl': 'http://sverigesradio.se/p1',
'xmltvid': 'p1.sr.se'},
{'channeltype': 'Lokal kanal',
'color': 'c31eaa',
'id': 200,
'image': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg',
'liveaudio': {'id': 200,
'statkey': '/app/direkt/p4 jämtland[k(200)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/200.mp3'},
'name': 'P4 Jämtland',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=200',
'siteurl': 'http://sverigesradio.se/jamtland/',
'xmltvid': 'p4jmtl.sr.se'}],
'copyright': 'Copyright Sveriges Radio 2017. All rights reserved.',
'pagination': {'nextpage': 'http://api.sr.se/v2/channelsformat=json&indent=true&page=2',
'page': 1,
'size': 10,
'totalhits': 55,
'totalpages': 6}}
Channels is a list. You have to iterate on it to get all channels and print their ids.
# starting from respjson
respjson = {
'channels': [
{
'channeltype': 'Rikskanal',
'color': '31a1bd',
'id': 132,
'image': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/132/2186745_512_512.jpg',
'liveaudio': {'id': 132,
'statkey': '/app/direkt/p1[k(132)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/132.mp3'},
'name': 'P1',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=132',
'siteurl': 'http://sverigesradio.se/p1',
'xmltvid': 'p1.sr.se'},
{
'channeltype': 'Lokal kanal',
'color': 'c31eaa',
'id': 200,
'image': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg?preset=api-default-square',
'imagetemplate': 'http://static-cdn.sr.se/sida/images/200/2186775_512_512.jpg',
'liveaudio': {'id': 200,
'statkey': '/app/direkt/p4 jämtland[k(200)]',
'url': 'http://sverigesradio.se/topsy/direkt/srapi/200.mp3'},
'name': 'P4 Jämtland',
'scheduleurl': 'http://api.sr.se/v2/scheduledepisodes?channelid=200',
'siteurl': 'http://sverigesradio.se/jamtland/',
'xmltvid': 'p4jmtl.sr.se'
}
],
'copyright': 'Copyright Sveriges Radio 2017. All rights reserved.',
'pagination': {
'nextpage': 'http://api.sr.se/v2/channelsformat=json&indent=true&page=2',
'page': 1,
'size': 10,
'totalhits': 55,
'totalpages': 6
}
}
for channel in respjson['channels']:
print(channel['id'])
What you want to do is look thru the dictionaries presented inside the channels, you can do that with the following...
for dic in respjson['channels']:
pprint(dic['id'])
I've been trying to figure this out all day and Im at my wits end. Maybe I'm just getting to old for this.
I'm trying to build a tree for the load_bulk feature on django-treebeard as specified here
To save you looking, it should look like this:
data = [{'data':{'desc':'1'}},
{'data':{'desc':'2'}, 'children':[
{'data':{'desc':'21'}},
{'data':{'desc':'22'}},
{'data':{'desc':'23'}, 'children':[
{'data':{'desc':'231'}},
]},
{'data':{'desc':'24'}},
]},
{'data':{'desc':'3'}},
{'data':{'desc':'4'}, 'children':[
{'data':{'desc':'41'}},
]},
]
'data' holds the record, and if it has children, 'children' is a list of more 'data' dicts (that can also contain a list of children and so on recursively)
I get the data as an ordered list (ordered as in depth first, not by id):
e.g:
[
{'id': 232, 'name': 'jon', 'parent': 'None'}
{'id': 3522, 'name': 'dave', 'parent': '232'}
{'id': 2277, 'name': 'alice', 'parent': '3522'}
{'id': 119, 'name': 'gary', 'parent': '232'}
{'id': 888, 'name': 'gunthe', 'parent': '119'}
{'id': 750, 'name': 'beavis', 'parent': 'None'}
{'id': 555, 'name': 'urte', 'parent': '750'}
]
How can I transform it into a treebeard compliant dictionary that would look like this (typo's excepted):
[
{'data': {'id': 232, 'name': 'jon', 'parent': 'None'},
'children': [
{'data': {'id': 3522, 'name': 'dave', 'parent': '232'},
'children': [
{'data': {'id': 2277, 'name': 'alice', 'parent': '3522'}}
]
}
{'data': {'id': 119, 'name': 'gary', 'parent': '232'},
'children': [
{'id': 888, 'name': 'gunthe', 'parent': '119'}
]
}
]
{'data': {'id': 750, 'name': 'beavis', 'parent': 'None'},
'children': [
{'id': 555, 'name': 'urte', 'parent': '750'}
]
}
]
I guess I need some kind of recursion function seeing as its a recursive structure but all my attempts have failed. My brain doesnt do recursion so good.
I did a lot of searching and found mostly solutions pertaining to lists or other structures that i cant mould to fit. I'm a relative noob. ps i had more fun manually typing out the example than i did the rest of day (apart from dinner time).
Maybe there are better ways, but here is one solution:
users = [
{
'id': 232,
'name': 'jon',
'parent': None
},
{
'id': 3522,
'name': 'dave',
'parent': 232
},
{
'id': 2277,
'name': 'alice',
'parent': 3522
},
{
'id': 119,
'name': 'gary',
'parent': 232
},
{
'id': 888,
'name': 'gunthe',
'parent': 119
},
{
'id': 750,
'name': 'beavis',
'parent': None
},
{
'id': 555,
'name': 'urte',
'parent': 750
}
]
users_map = {}
for user in users:
users_map[user['id']] = user
users_tree = []
for user in users:
if user['parent'] is None:
users_tree.append(user)
else:
parent = users_map[user['parent']]
if 'childs' not in parent:
parent['childs'] = []
parent['childs'].append(user)
print(users_tree)
#user as {data: user, childs: []}
users_map = {}
for user in users:
users_map[user['id']] = {'data': user, 'childs': []}
users_tree = []
for user in users:
if user['parent'] is None:
users_tree.append(users_map[user['id']])
else:
parent = users_map[user['parent']]
parent['childs'].append(users_map[user['id']])
print(users_tree)