Develop Reference

Wrong data on Youtube API

I am trying to retrieve videos details from Youtube. However, my code is not returning the data I am expecting. In fact, I don't even know what I am retrieving. I am not sure if my problem is the channel ID or just the way I am calling the API.
Here is what I am doing:
# These variables above are in the youtubevar.py files where I keep my variables
apikey ='xxxxxxxxxxxxxx'
channelid='UCWnfDPdZw6A23UtuBpYBbAg' # this supposed to be james charles channel
pageToken=''
url='https://www.googleapis.com/youtube/v3/search?key='+apikey+'&channelId='+channelid+'&part=snippet,id&order=date&maxResults=200'+pageToken
======================= == = == = = == = = = = =
# Main file
import requests
import pandas as pd
import seaborn as sns
%pylab inline
from youtubevar import*
#Here im calling the API
response=requests.get(url).json()
# having a look of what I get
print (response) # Not the data I expect
The according to the code, I am supposed to get the details of the video in descending order. SO I should get the latest one first but I am getting some turkish there.
Sample output
'kind': 'youtube#searchListResponse',
'etag': 'Baz6gP84Rb-IaJRTFB6BTXHSOOc',
'nextPageToken': 'CDIQAA',
'regionCode': 'GB',
'pageInfo': {'totalResults': 1000000, 'resultsPerPage': 50},
'items': [{'kind': 'youtube#searchResult',
'etag': 'Wvdfq24D5OzYpzICCND0Z9TypT8',
'id': {'kind': 'youtube#video', 'videoId': '5YiV7OyzjH0'},
'snippet': {'publishedAt': '2021-08-25T08:00:01Z',
'channelId': 'UCcNqSmZeXNxccDzVYEf7zpg',
'title': 'TUGAY GÖK A WAY OUT (AHA YAKALAN DIK !!!!)',
'description': 'Tugay Gök Nonolive https://www.nonolive.com/tugaygok Tugay Gök Youtube https://www.youtube.com/user/TheMaraztr Tugay Gök İnstagram ...',
'thumbnails': {'default': {'url': 'https://i.ytimg.com/vi/5YiV7OyzjH0/default.jpg',
'width': 120,
'height': 90},
'medium': {'url': 'https://i.ytimg.com/vi/5YiV7OyzjH0/mqdefault.jpg',
'width': 320,
'height': 180},
'high': {'url': 'https://i.ytimg.com/vi/5YiV7OyzjH0/hqdefault.jpg',
'width': 480,
'height': 360}},
'channelTitle': 'Tugay Gök Fan',
'liveBroadcastContent': 'none',
'publishTime': '2021-08-25T08:00:01Z'}},
{'kind': 'youtube#searchResult',

How to Match two APIs to update one API dataset using Python

I want to be able to GET information from API 1 and match it with API 2 and be able to update API 2's information with API 1. I am trying to figure out the most efficient/automated way to accomplish this as it also needs to be updated at a interval of every 10 minutes
I can query and get the results from API 1 this is my code and what my code looks like.
import json
import requests
myToken = '52c32f6588004cb3ab33b0ff320b8e4f'
myUrl = 'https://api1.com/api/v1/devices.json'
head = {'Authorization': 'Token {}'.format(myToken)}
response = requests.get(myUrl, headers=head)
r = json.loads(response.content)
r
The payload looks like this from API 1
{ "device" : {
"id": 153,
"battery_status" : 61,
"serial_no": "5QBYGKUI05",
"location_lat": "-45.948917",
"location_lng": "29.832179",
"location_address": "800 Laurel Rd, Lansdale, PA 192522,USA"}
}
I want to be able to take this information and match by "serial_no" and update all the other pieces of information for the corresponding device in API 2
I query the data for API 2 and this is what my code looks like
params = {
"location":'cf6707e3-f0ae-4040-a184-737b21a4bbd1',
"dateAdded":'ge:11/23/2020'}
url = requests.get('https://api2.com/api/assets',auth=('api2', '123456'), params=params)
r = json.loads(url.content)
r['items']
The JSON payload looks like this
[{'id': '064ca857-3783-460e-a7a2-245e054dcbe3',
'name': 'Apple Laptop 1',
'model': {'id': '50f5993e-2abf-49c8-86e0-8743dd58db6f',
'name': 'MacBook Pro'},
'manufacturer': {'id': 'f56244e2-76e3-46da-97dd-f72f92ca0779',
'name': 'APPLE'},
'room': {'id': '700ff2dc-0118-46c6-936a-01f0fa88c620',
'name': 'Storage Room 1',
'thirdPartyId': ''},
'location': {'id': 'cf6707e3-f0ae-4040-a184-737b21a4bbd1',
'name': 'Iron Mountain',
'thirdPartyId': ''},
'position': 'NonMounted',
'containerAsset': {'id': '00000000-0000-0000-0000-000000000000',
'name': None},
'baseAsset': {'id': '064ca857-3783-460e-a7a2-245e054dcbe3',
'name': 'Apple Laptop 1'},
'description': None,
'status': {'id': 'df9906d8-2856-45e3-9cba-bd7a1ac4971f',
'name': 'Production'},
'serialNumber': '5QBYGKUI06',
'tagNumber': None,
'alternateTagNumber': None,
'verificationStatus': {'id': 'cb3560a9-eef5-47b9-b033-394d3a09db18',
'name': 'Verified'},
'requiresRFID': False,
'requiresHangTag': False,
'bottomPosition': 0.0,
'leftPosition': 0.0,
'rackPosition': 'Front',
'labelX': None,
'labelY': None,
'verifyNameInRear': False,
'verifySerialNumberInRear': False,
'verifyBarcodeInRear': False,
'isNonDataCenter': False,
'rotate': False,
'customer': {'id': '00000000-0000-0000-0000-000000000000', 'name': None},
'thirdPartyId': '',
'temperature': None,
'dateLastScanned': None,
'placement': 'Floor',
'lastScannedLabelX': None,
'lastScannedLabelY': None,
'userDefinedValues': [{'userDefinedKeyId': '79e77a1e-4030-4308-a8ff-9caf40c04fbd',
'userDefinedKeyName': 'Longitude ',
'value': '-75.208917'},
{'userDefinedKeyId': '72c8056e-9b7d-40ac-9270-9f5929097e82',
'userDefinedKeyName': 'Address',
'value': '800 Laurel Rd, New York ,NY 19050, USA'},
{'userDefinedKeyId': '31aeeb91-daef-4364-8dd6-b0e3436d6a51',
'userDefinedKeyName': 'Battery Level',
'value': '67'},
{'userDefinedKeyId': '22b7ce4f-7d3d-4282-9ecb-e8ec2238acf2',
'userDefinedKeyName': 'Latitude',
'value': '35.932179'}]}
The documentation provided by API 2 tells me they only support PUT for updates as of right now but I would also want to know how I would do this using PATCH as it will be available in the future. So the data payload that I need to successful PUT is this
payload = {'id': '064ca857-3783-460e-a7a2-245e054dcbe3',
'name': 'Apple Laptop 1',
'model': {'id': '50f5993e-2abf-49c8-86e0-8743dd58db6f',
'name': 'MacBook Pro'},
'manufacturer': {'id': 'f56244e2-76e3-46da-97dd-f72f92ca0779',
'name': 'APPLE'},
'room': {'id': '700ff2dc-0118-46c6-936a-01f0fa88c620',
'name': 'Storage Room 1',
'thirdPartyId': ''},
'status': {'id': 'df9906d8-2856-45e3-9cba-bd7a1ac4971f',
'name': 'Production'},
'serialNumber': '5QBYGKUI06',
'verificationStatus': {'id': 'cb3560a9-eef5-47b9-b033-394d3a09db18',
'name': 'Verified'},
'requiresRFID': 'False',
'requiresHangTag': 'False',
'userDefinedValues': [{'userDefinedKeyId': '79e77a1e-4030-4308-a8ff-9caf40c04fbd',
'userDefinedKeyName': 'Longitude ',
'value': '-75.248920'},
{'userDefinedKeyId': '72c8056e-9b7d-40ac-9270-9f5929097e82',
'userDefinedKeyName': 'Address',
'value': '801 Laurel Rd, New York, Ny 192250, USA'},
{'userDefinedKeyId': '31aeeb91-daef-4364-8dd6-b0e3436d6a51',
'userDefinedKeyName': 'Battery Level',
'value': '67'},
{'userDefinedKeyId': '22b7ce4f-7d3d-4282-9ecb-e8ec2238acf2',
'userDefinedKeyName': 'Latitude',
'value': '29.782177'}]}
So apart of this is figuring out how I can query the json data portions that I need for the update
I am able to update the information using this line
requests.put('https://api2.com/api/assets/064ca857-3783-460e-a7a2-245e054dcbe3',auth=('API2', '123456'), data=json.dumps(payload))
but I need for it to dynamically update so I don't think the hard coded id parameter in the line will be efficient in a automation/efficiency standpoint. If anybody has any ideas, resources to point me in the right direction to know more about this process (I don't really know what it is even called) would be greatly appreciated.

Not entirely sure what you are trying to do here, but if you want to pull information nested in the responses you can do this.
Serial number from API 1
r['device']['serial_no']
Serial number for API 2
either r[0]['serialNumber'] or r['items'][0]['serialNumber'] depending on what you are showing
To modify the payload serial number, for example
payload['serialNumber'] = '123456abcdef'

How can I print specific integer variables in a nested dictionary by using Python?

This is my first question :)
I loop over a nested dictionary to print specific values. I am using the following code.
for i in lizzo_top_tracks['tracks']:
print('Track Name: ' + i['name'])
It works for string variables, but does not work for other variables. For example, when I use the following code for the date variable:
for i in lizzo_top_tracks['tracks']:
print('Album Release Date: ' + i['release_date'])
I receive a message like this KeyError: 'release_date'
What should I do?
Here is a sample of my nested dictionary:
{'tracks': [{'album': {'album_type': 'album',
'artists': [{'external_urls': {'spotify': 'https://open.spotify.com/artist/56oDRnqbIiwx4mymNEv7dS'},
'href': 'https://api.spotify.com/v1/artists/56oDRnqbIiwx4mymNEv7dS',
'id': '56oDRnqbIiwx4mymNEv7dS',
'name': 'Lizzo',
'type': 'artist',
'uri': 'spotify:artist:56oDRnqbIiwx4mymNEv7dS'}],
'external_urls': {'spotify': 'https://open.spotify.com/album/74gSdSHe71q7urGWMMn3qB'},
'href': 'https://api.spotify.com/v1/albums/74gSdSHe71q7urGWMMn3qB',
'id': '74gSdSHe71q7urGWMMn3qB',
'images': [{'height': 640,
'width': 640}],
'name': 'Cuz I Love You (Deluxe)',
'release_date': '2019-05-03',
'release_date_precision': 'day',
'total_tracks': 14,
'type': 'album',
'uri': 'spotify:album:74gSdSHe71q7urGWMMn3qB'}]}

The code you posted isn't syntactically correct; running it through a Python interpreter gives a syntax error on the last line. It looks like you lost a curly brace somewhere toward the end. :)
I went through it and fixed up the white space to make the structure easier to see; the way you had it formatted made it hard to see which keys were at which level of nesting, but with consistent indentation it becomes much clearer:
lizzo_top_tracks = {
'tracks': [{
'album': {
'album_type': 'album',
'artists': [{
'external_urls': {
'spotify': 'https://open.spotify.com/artist/56oDRnqbIiwx4mymNEv7dS'
},
'href': 'https://api.spotify.com/v1/artists/56oDRnqbIiwx4mymNEv7dS',
'id': '56oDRnqbIiwx4mymNEv7dS',
'name': 'Lizzo',
'type': 'artist',
'uri': 'spotify:artist:56oDRnqbIiwx4mymNEv7dS'
}],
'external_urls': {
'spotify': 'https://open.spotify.com/album/74gSdSHe71q7urGWMMn3qB'
},
'href': 'https://api.spotify.com/v1/albums/74gSdSHe71q7urGWMMn3qB',
'id': '74gSdSHe71q7urGWMMn3qB',
'images': [{'height': 640, 'width': 640}],
'name': 'Cuz I Love You (Deluxe)',
'release_date': '2019-05-03',
'release_date_precision': 'day',
'total_tracks': 14,
'type': 'album',
'uri': 'spotify:album:74gSdSHe71q7urGWMMn3qB'
}
}]
}
So the first (and only) value you get for i in lizzo_top_tracks['tracks'] is going to be this dictionary:
i = {
'album': {
'album_type': 'album',
'artists': [{
'external_urls': {
'spotify': 'https://open.spotify.com/artist/56oDRnqbIiwx4mymNEv7dS'
},
'href': 'https://api.spotify.com/v1/artists/56oDRnqbIiwx4mymNEv7dS',
'id': '56oDRnqbIiwx4mymNEv7dS',
'name': 'Lizzo',
'type': 'artist',
'uri': 'spotify:artist:56oDRnqbIiwx4mymNEv7dS'
}],
'external_urls': {
'spotify': 'https://open.spotify.com/album/74gSdSHe71q7urGWMMn3qB'
},
'href': 'https://api.spotify.com/v1/albums/74gSdSHe71q7urGWMMn3qB',
'id': '74gSdSHe71q7urGWMMn3qB',
'images': [{'height': 640, 'width': 640}],
'name': 'Cuz I Love You (Deluxe)',
'release_date': '2019-05-03',
'release_date_precision': 'day',
'total_tracks': 14,
'type': 'album',
'uri': 'spotify:album:74gSdSHe71q7urGWMMn3qB'
}
}
The only key in this dictionary is 'album', the value of which is another dictionary that contains all the other information. If you want to print, say, the album release date and a list of the artists' names, you'd do:
for track in lizzo_top_tracks['tracks']:
print('Album Release Date: ' + track['album']['release_date'])
print('Artists: ' + str([artist['name'] for artist in track['album']['artists']]))
If these are dictionaries that you're building yourself, you might want to remove some of the nesting layers where there's only a single key, since they just make it harder to navigate the structure without giving you any additional information. For example:
lizzo_top_albums = [{
'album_type': 'album',
'artists': [{
'external_urls': {
'spotify': 'https://open.spotify.com/artist/56oDRnqbIiwx4mymNEv7dS'
},
'href': 'https://api.spotify.com/v1/artists/56oDRnqbIiwx4mymNEv7dS',
'id': '56oDRnqbIiwx4mymNEv7dS',
'name': 'Lizzo',
'type': 'artist',
'uri': 'spotify:artist:56oDRnqbIiwx4mymNEv7dS'
}],
'external_urls': {
'spotify': 'https://open.spotify.com/album/74gSdSHe71q7urGWMMn3qB'
},
'href': 'https://api.spotify.com/v1/albums/74gSdSHe71q7urGWMMn3qB',
'id': '74gSdSHe71q7urGWMMn3qB',
'images': [{'height': 640, 'width': 640}],
'name': 'Cuz I Love You (Deluxe)',
'release_date': '2019-05-03',
'release_date_precision': 'day',
'total_tracks': 14,
'type': 'album',
'uri': 'spotify:album:74gSdSHe71q7urGWMMn3qB'
}]
This structure allows you to write the query the way you were originally trying to do it:
for album in lizzo_top_albums:
print('Album Release Date: ' + album['release_date'])
print('Artists: ' + str([artist['name'] for artist in album['artists']]))
Much simpler, right? :)

How to build a recursive dictionary tree from an ordered adjacency list

I've been trying to figure this out all day and Im at my wits end. Maybe I'm just getting to old for this.
I'm trying to build a tree for the load_bulk feature on django-treebeard as specified here
To save you looking, it should look like this:
data = [{'data':{'desc':'1'}},
{'data':{'desc':'2'}, 'children':[
{'data':{'desc':'21'}},
{'data':{'desc':'22'}},
{'data':{'desc':'23'}, 'children':[
{'data':{'desc':'231'}},
]},
{'data':{'desc':'24'}},
]},
{'data':{'desc':'3'}},
{'data':{'desc':'4'}, 'children':[
{'data':{'desc':'41'}},
]},
]
'data' holds the record, and if it has children, 'children' is a list of more 'data' dicts (that can also contain a list of children and so on recursively)
I get the data as an ordered list (ordered as in depth first, not by id):
e.g:
[
{'id': 232, 'name': 'jon', 'parent': 'None'}
{'id': 3522, 'name': 'dave', 'parent': '232'}
{'id': 2277, 'name': 'alice', 'parent': '3522'}
{'id': 119, 'name': 'gary', 'parent': '232'}
{'id': 888, 'name': 'gunthe', 'parent': '119'}
{'id': 750, 'name': 'beavis', 'parent': 'None'}
{'id': 555, 'name': 'urte', 'parent': '750'}
]
How can I transform it into a treebeard compliant dictionary that would look like this (typo's excepted):
[
{'data': {'id': 232, 'name': 'jon', 'parent': 'None'},
'children': [
{'data': {'id': 3522, 'name': 'dave', 'parent': '232'},
'children': [
{'data': {'id': 2277, 'name': 'alice', 'parent': '3522'}}
]
}
{'data': {'id': 119, 'name': 'gary', 'parent': '232'},
'children': [
{'id': 888, 'name': 'gunthe', 'parent': '119'}
]
}
]
{'data': {'id': 750, 'name': 'beavis', 'parent': 'None'},
'children': [
{'id': 555, 'name': 'urte', 'parent': '750'}
]
}
]
I guess I need some kind of recursion function seeing as its a recursive structure but all my attempts have failed. My brain doesnt do recursion so good.
I did a lot of searching and found mostly solutions pertaining to lists or other structures that i cant mould to fit. I'm a relative noob. ps i had more fun manually typing out the example than i did the rest of day (apart from dinner time).

Maybe there are better ways, but here is one solution:
users = [
{
'id': 232,
'name': 'jon',
'parent': None
},
{
'id': 3522,
'name': 'dave',
'parent': 232
},
{
'id': 2277,
'name': 'alice',
'parent': 3522
},
{
'id': 119,
'name': 'gary',
'parent': 232
},
{
'id': 888,
'name': 'gunthe',
'parent': 119
},
{
'id': 750,
'name': 'beavis',
'parent': None
},
{
'id': 555,
'name': 'urte',
'parent': 750
}
]
users_map = {}
for user in users:
users_map[user['id']] = user
users_tree = []
for user in users:
if user['parent'] is None:
users_tree.append(user)
else:
parent = users_map[user['parent']]
if 'childs' not in parent:
parent['childs'] = []
parent['childs'].append(user)
print(users_tree)
#user as {data: user, childs: []}
users_map = {}
for user in users:
users_map[user['id']] = {'data': user, 'childs': []}
users_tree = []
for user in users:
if user['parent'] is None:
users_tree.append(users_map[user['id']])
else:
parent = users_map[user['parent']]
parent['childs'].append(users_map[user['id']])
print(users_tree)

What does update() in mongodb returns

While writing python script for the api in mongodb..
We have..
new_posts = [{ 'name': 'A', 'age': 17, 'marks': 97, 'school': 'School1' },
{ 'name': 'B', 'age': 18, 'marks': 95, 'school': 'School2' },
{ 'name': 'C', 'age': 19, 'marks': 97, 'school': 'School2' }]
db.posts.insert( new_posts )
We create indexes as follows..
db.posts.create_index([('name',1),('school',1)],unique=True)
Now we perform two operations..
db.posts.update({ 'name':'A', 'age': 17, 'school': 'School3' },
{ 'name':'D', 'age': 17, 'marks': 70, 'school': 'School1' },
upsert=True )
db.posts.update({ 'name':'A', 'age': 17, 'school': 'School1' },
{ 'name':'A', 'age': 17, 'marks': 60, 'school': 'School1' },
upsert=True )
What does the update() returns here? How can we find out weather the document is inserted into the db or existing document is updated?
Can we do something like..
post1 = db.posts.update({ 'name':'A', 'age': 17, 'school': 'School3' },
{ 'name':'D', 'age': 17, 'marks': 70, 'school': 'School1' },
upsert=True )
post2 = db.posts.update({ 'name':'A', 'age': 17, 'school': 'School1' },
{ 'name':'A', 'age': 17, 'marks': 60, 'school': 'School1' },
upsert=True )
print post1
print post2

As the docs for update say, the method returns:
A document (dict) describing the effect of the update or None if write acknowledgement is disabled.
Just try it and print the return value to see what's available. You'll see something like:
{u'syncMillis': 0, u'ok': 1.0, u'err': None, u'writtenTo': None,
u'connectionId': 190, u'n': 1, u'updatedExisting': True}
The updatedExisting field is what you're looking for.