Develop Reference

Extract and download chapters from youtube videos [duplicate]

Recently, Youtube added the ability to break up their videos in the progress bar into sections called "chapters".
https://support.google.com/youtube/answer/9884579?hl=en
Currently I am able to get info about a video from the Youtube API. However, it doesn't seem like there's any info about a video's chapters, and I haven't found anything in the API documentation about chapters. Am I missing something, or is there simply no way to get chapter data yet?

As far as I know, such data is in plain text in the description of the video.
So, you can use the following example:
Video used in this demonstration: Top 10 Monsters with 2500 Attack in YuGiOh
URL Request:
https://www.googleapis.com/youtube/v3/videos?part=snippet&id=NNgYId7b4j0&key=[YOUR_API_KEY]
Response:
{
"kind": "youtube#videoListResponse",
"etag": "YpVLmrSx1iP8hAJOnumaTBoKqqQ",
"items": [
{
"kind": "youtube#video",
"etag": "oIoJq5F3RHvBbtVohafaJ_1SThU",
"id": "NNgYId7b4j0",
"snippet": {
"publishedAt": "2020-09-14T18:37:46Z",
"channelId": "UC0roOaAn95Rtgoe078RkVXQ",
"title": "Top 10 Monsters with 2500 Attack in YuGiOh",
"description": "In this video we'll go over the best monsters that have 2500 attack, and attack threshold for a lot of boss monsters actually.\n\nCheck out my DnD channel #TheD&DLogs \n\n--The List--\nIntro: (0:00)\n10- Blue-Eyes Spirit Dragon: (0:00)\n9- Invoked Mechaba: (2:14)\n8- Number S39: Utopia the Lightning: (3:23)\n7- Earthbound Immortal Aslla Piscu: (4:35)\n6- Eldlich the golden Lord: (6:04)\n5- True King Lithosagym the Disaster: (7:34)\n4- Block Dragon: (8:54)\n3- Astrograph sorcerer: (10:25)\n2- Beatrice lady of the eternal: (12:36)\n1- Firewall Dragon: (14:37)\n- \n-----------------------------------------\n#yugioh #top10 \n\nDuels are all done on EDOpro, its completely free and updated all the time. If you want it, just look for the EDOpro discord and you'll find all you need to download it from there\n\nSome of the Video backgrounds in this video were made by \"Amitai Angor AA VFX\" https://www.youtube.com/dvdangor2011\n\n\nhttps://twitter.com/hirumared\nhttps://twitter.com/TheDuelLogs",
"thumbnails": {
"default": {
"url": "https://i.ytimg.com/vi/NNgYId7b4j0/default.jpg",
"width": 120,
"height": 90
},
"medium": {
"url": "https://i.ytimg.com/vi/NNgYId7b4j0/mqdefault.jpg",
"width": 320,
"height": 180
},
"high": {
"url": "https://i.ytimg.com/vi/NNgYId7b4j0/hqdefault.jpg",
"width": 480,
"height": 360
},
"standard": {
"url": "https://i.ytimg.com/vi/NNgYId7b4j0/sddefault.jpg",
"width": 640,
"height": 480
},
"maxres": {
"url": "https://i.ytimg.com/vi/NNgYId7b4j0/maxresdefault.jpg",
"width": 1280,
"height": 720
}
},
"channelTitle": "TheDuelLogs",
"tags": [
"yugioh",
"ygo",
"dev",
"pro",
"link",
"duels",
"auto-matic duels",
"online",
"current",
"ban",
"list",
"dueling",
"network",
"theduellogs",
"the",
"duel",
"logs",
"loggs",
"Yu",
"Gi",
"Oh!",
"YGOpro",
"gimmick",
"links",
"top ten",
"2020",
"edopro"
],
"categoryId": "20",
"liveBroadcastContent": "none",
"localized": {
"title": "Top 10 Monsters with 2500 Attack in YuGiOh",
"description": "In this video we'll go over the best monsters that have 2500 attack, and attack threshold for a lot of boss monsters actually.\n\nCheck out my DnD channel #TheD&DLogs \n\n--The List--\nIntro: (0:00)\n10- Blue-Eyes Spirit Dragon: (0:00)\n9- Invoked Mechaba: (2:14)\n8- Number S39: Utopia the Lightning: (3:23)\n7- Earthbound Immortal Aslla Piscu: (4:35)\n6- Eldlich the golden Lord: (6:04)\n5- True King Lithosagym the Disaster: (7:34)\n4- Block Dragon: (8:54)\n3- Astrograph sorcerer: (10:25)\n2- Beatrice lady of the eternal: (12:36)\n1- Firewall Dragon: (14:37)\n- \n-----------------------------------------\n#yugioh #top10 \n\nDuels are all done on EDOpro, its completely free and updated all the time. If you want it, just look for the EDOpro discord and you'll find all you need to download it from there\n\nSome of the Video backgrounds in this video were made by \"Amitai Angor AA VFX\" https://www.youtube.com/dvdangor2011\n\n\nhttps://twitter.com/hirumared\nhttps://twitter.com/TheDuelLogs"
},
"defaultAudioLanguage": "en"
}
}
],
"pageInfo": {
"totalResults": 1,
"resultsPerPage": 1
}
}
Get the response:
response.items[0].snippet.description
Results:
"In this video we'll go over the best monsters that have 2500 attack, and attack threshold for a lot of boss monsters actually.
Check out my DnD channel #TheD&DLogs
--The List--
Intro: (0:00)
10- Blue-Eyes Spirit Dragon: (0:00)
9- Invoked Mechaba: (2:14)
8- Number S39: Utopia the Lightning: (3:23)
7- Earthbound Immortal Aslla Piscu: (4:35)
6- Eldlich the golden Lord: (6:04)
5- True King Lithosagym the Disaster: (7:34)
4- Block Dragon: (8:54)
3- Astrograph sorcerer: (10:25)
2- Beatrice lady of the eternal: (12:36)
1- Firewall Dragon: (14:37)
-
-----------------------------------------
#yugioh #top10
Duels are all done on EDOpro, its completely free and updated all the time. If you want it, just look for the EDOpro discord and you'll find all you need to download it from there
Some of the Video backgrounds in this video were made by "Amitai Angor AA VFX" https://www.youtube.com/dvdangor2011
https://twitter.com/hirumared
https://twitter.com/TheDuelLogs"

One more time YouTube Data API v3 doesn't provide a basic feature.
I would suggest you to use my open-source YouTube operational API, indeed by requesting https://yt.lemnoslife.com/videos?part=chapters&id=VIDEO_ID you would get a JSON with the video chapters (titles and timestamps) you are looking for in item['chapters']['chapters'].
Example of result with YouTube video id NNgYId7b4j0:
{
"kind": "youtube#videoListResponse",
"etag": "NotImplemented",
"items": [
{
"kind": "youtube#video",
"etag": "NotImplemented",
"id": "NNgYId7b4j0",
"chapters": {
"areAutoGenerated": false,
"chapters": [
{
"title": "10- Blue-Eyes Spirit Dragon",
"time": 0,
"thumbnails": [
{
"url": "https:\/\/i.ytimg.com\/vi\/NNgYId7b4j0\/hqdefault_4000.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLCoTrvu0Yu-iNxb7o4II-pxi5WVbQ",
"width": 168,
"height": 94
},
{
"url": "https:\/\/i.ytimg.com\/vi\/NNgYId7b4j0\/hqdefault_4000.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLCuupNwIgFIf9hXbjMsvpSGThFyhg",
"width": 336,
"height": 188
}
]
},
{
"title": "9- Invoked Mechaba",
"time": 134,
"thumbnails": [
{
"url": "https:\/\/i.ytimg.com\/vi\/NNgYId7b4j0\/hqdefault_135933.jpg?sqp=-oaymwEiCKgBEF5IWvKriqkDFQgBFQAAAAAYASUAAMhCPQCAokN4AQ==&rs=AOn4CLBe94BKNpQXvM2dUl75LtcgX0N03w",
"width": 168,
"height": 94
},
{
"url": "https:\/\/i.ytimg.com\/vi\/NNgYId7b4j0\/hqdefault_135933.jpg?sqp=-oaymwEjCNACELwBSFryq4qpAxUIARUAAAAAGAElAADIQj0AgKJDeAE=&rs=AOn4CLBULUhlI1OOjJiW6mpFDUhPzh4Adw",
"width": 336,
"height": 188
}
]
},
...
]
}
}
]
}

I am replying with this answer to help people such as myself who ended up on this video wanting to find a youtube chapter parser / extractor for text rather than where to find the chapter data. Just to add some further information, currently, there is no way to get the chapters from the official YouTube API, so the only way to get the chapters from a text-description response (like the YouTube API provides) is to parse it in some way:
My answer is in Javascript but it can easily be converted: The idea is to extract the MIN:SEC and HR:MIN:SEC timestamps then to generate the title we remove the word that includes them (So this would typically remove however people aesthetically wrap them too [00:00] or (00:00)
It's far from perfect, but in my experience it's better than the other solutions I've seen on github/npm at the time of writing this. You might want to also trim away starting and ending spaces and punctuational separators such as (-, :, ~, |) too
const parseChapters = (description) => {
// Extract timestamps (either 00:00:00, 0:00:00, 00:00 or 0:00)
const lines = description.split("\n")
const regex = /(\d{0,2}:?\d{1,2}:\d{2})/g
const chapters = []
for (const line of lines) {
// Match the regex and check if the line contains a matched regex
const matches = line.match(regex)
if (matches) {
const ts = matches[0]
const title = line
.split(" ")
.filter((l) => !l.includes(ts))
.join(" ")
chapters.push({
timestamp: ts,
title: title,
})
}
}
return chapters
}

Very late answer but it solved my problem.
You could use the code below. It's written in C# but it can easily be transcribed into another language. Since you can already get youtube video data, I assume you also have the description of the video.
private static IEnumerable<string> GetChaptersFromDescription(string text)
{
var lines = text.Split("\n");
var regex = new Regex(#"[0-9]:[0-9][0-9]");
foreach (var line in lines)
{
if (regex.IsMatch(line))
{
yield return line;
}
}
}

Issues looping through dict python

I'm running into an issue with actually successfully looping through a boto3 result for AWS Config. From my testing, I am able to perform the loop; however, it only outputs same violation instead of each violation for the different ARNs.
I realize where the issue is at in the code as I was doing the below:
for arn in configRule.items():
acmArn = configRule['EvaluationResults'][0]['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceId']
But, when I try and take out the [0] so it is not a specific item I get "list indices must be integers or slices, not str".
My question is if there is an actual way to get this value to be incremented, so it will be able to get the different ARNs of the violations? Sorry if this is an easy answer, still new to Python, so trying to figure all this out.
Output Example:
{
"EvaluationResults": [
{
"Annotation": "Certificate will expire on 2023-01-21T23:59:59.000Z",
"ComplianceType": "NON_COMPLIANT",
"ConfigRuleInvokedTime": "2022-01-03 18:28:54.939000+00:00",
"EvaluationResultIdentifier": {
"EvaluationResultQualifier": {
"ConfigRuleName": "acm-certificate-expiration-check",
"ResourceId": "arn:aws:acm:us-west-2:xxxx:certificate/39a95537-e5aa-46dd-bc9b-04d7b2606bd0",
"ResourceType": "AWS::ACM::Certificate"
},
"OrderingTimestamp": "2021-12-22 00:29:01+00:00"
},
"ResultRecordedTime": "2022-01-03 18:28:55.672000+00:00"
},
{
"Annotation": "Certificate will expire on 2023-01-10T23:59:59.000Z",
"ComplianceType": "NON_COMPLIANT",
"ConfigRuleInvokedTime": "2022-01-03 18:28:54.939000+00:00",
"EvaluationResultIdentifier": {
"EvaluationResultQualifier": {
"ConfigRuleName": "acm-certificate-expiration-check",
"ResourceId": "arn:aws:acm:us-west-2:xxxx:certificate/493de1e8-2bcb-42c7-96df-ce88bdeac64c",
"ResourceType": "AWS::ACM::Certificate"
},
"OrderingTimestamp": "2021-12-12 18:25:14+00:00"
},
"ResultRecordedTime": "2022-01-03 18:28:55.683000+00:00"
}
],
"ResponseMetadata": {
"HTTPHeaders": {
"content-length": "955",
"content-type": "application/x-amz-json-1.1",
"date": "Mon, 03 Jan 2022 20:13:06 GMT",
"strict-transport-security": "max-age=86400",
"x-amzn-requestid": "a6e51323-9e4c-44c7-a15a-ea0314392ab6"
},
"HTTPStatusCode": 200,
"RequestId": "a6e51323-9e4c-44c7-a15a-ea0314392ab6",
"RetryAttempts": 0
}
}

configRules.items() for your example would be a sequence containing a single tuple, ('EvaluationResults', [...]). You don't really care about that key: you hard-code it in the body. What you really want is to iterate over the dicts in the list configRule['EvaluationResults']:
for arn in configRule['EvaluationResults']:
acmArn = arn['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceId']

So, I'm not sure if this is the correct way to do this, but for now this is how I got this to work. This allows me to add a variable there and increase it each time it performs the loop.
I'm sure there is a better way to do this, but just what I found doing some testing.
i = 0
for key, value in configRule.items():
acmArn = configRule['EvaluationResults'][i]['EvaluationResultIdentifier']['EvaluationResultQualifier']['ResourceId']
i+=1

Is it possible to retreive object_story_spec for an ad creative which was not created with that object_story_spec? [Python Facebook API]

I have a set of ad creatives that I retreive through the Facebook Business Python SDK. I need these specifically to retreive the outbound URL when someone clicks on the ad: AdCreative['object_story_spec']['video_data']['call_to_action']['value']['link'].
I use the following call:
adcreatives = set.get_ad_creatives(fields=[
AdCreative.Field.id,
AdCreative.Field.name,
AdCreative.Field.object_story_spec,
AdCreative.Field.effective_object_story_id ,
])
Where set is an ad set.
For some cases, the result looks like this (with actual data removed), which is expected:
<AdCreative> {
"body": "[<BODY>]",
"effective_object_story_id": "[<EFFECTIVE_OBJECT_STORY_ID>]",
"id": "[<ID>]",
"name": "[<NAME>]",
"object_story_spec": {
"instagram_actor_id": "[<INSTAGRAM_ACTOR_ID>]",
"page_id": "[<PAGE_ID>]",
"video_data": {
"call_to_action": {
"type": "[<TYPE>]",
"value": {
"link": "[<LINK>]", <== This is what I need
"link_format": "[<LINK_FORMAT>]"
}
},
"image_hash": "[<IMAGE_HASH>]",
"image_url": "[<IMAGE_URL>]",
"message": "[<MESSAGE>]",
"video_id": "[<VIDEO_ID>]"
}
}
}
While sometimes results look like this:
<AdCreative> {
"effective_object_story_id": "[<EFFECTIVE_OBJECT_STORY_ID>]",
"id": "[<ID>]",
"name": "[<NAME>]",
"object_story_spec": {
"instagram_actor_id": "[<INSTAGRAM_ACTOR_ID>]",
"page_id": "[<PAGE_ID>]"
}
}
According to this earlier question: Can't get AdCreative ObjectStorySpec this is due to the fact that the object_story_spec is not populated if it is linked to a creative, instead of created along with the creative.
However, the video_data (and as such, the link), should be saved somewhere. Is there a way to retreive this? Maybe through effective_object_story_id?
The documentation page for object_story_spec (https://developers.facebook.com/docs/marketing-api/reference/ad-creative-object-story-spec/v12.0) does not have the information I am looking for.

How to retrieve values from a json file in a particular order?

I am new to python. I am writing a code, where I need to read the json file, and dump some of it's data to a new json file.
Following is my code:
if vmName=='IGW':
with open(APIJSONPath+NwIntfJSONFileName) as f:
data=json.load(f)
for item in data['Interfaces']:
jdata=item
with open(NwIntfJSONPath+vmName+'.json','w') as c:
json.dump(jdata,c,indent=2)
Following is a small part of my json file data from which this code is supposed to retrieve the interface details(Interface name,IPAddress, PrefixLength, DefaultGateway) of eth0 and eth1:
{
"Interfaces": [{
"Name": "eth0",
"IPConfigurations": [{
"IPAddress": "10.0.3.7",
"PrefixLength": 24,
"DefaultGateway": "10.0.3.1",
"IsPrimary": true
}],
"Description0": "connected to cloudsimple network",
"IsPrimary": true
} ,
{
"Name": "eth1",
"IPConfigurations": [{
"IPAddress": "10.0.3.8",
"PrefixLength": 24,
"DefaultGateway": "10.0.3.1",
"IsPrimary": true
}],
"Description1": "connected to internet"
}
]
}
But the data that is getting dumped the new json file is:
{
"Name": "eth1",
"IPConfigurations": [
{
"PrefixLength": 24,
"IsPrimary": true,
"IPAddress": "10.0.3.8",
"DefaultGateway": "10.0.3.1"
}
],
"Description1": "connected to internet"
}
Only eth1 details is getting dumped, not the eth0. The dumped data is also in an unordered manner.
Can someone please help me figure out, where I am going wrong, and how to fix this two issues in my code? Thanks in advance.

If you need all content of data['Interfaces'] in your output json use the below snippet.
if vmName=='IGW':
with open(APIJSONPath+NwIntfJSONFileName) as f:
data=json.load(f)
with open(NwIntfJSONPath+vmName+'.json','w') as c:
json.dump(data['Interfaces'],c,indent=2)
In your example you are looping through data['Interfaces'] and jdata holds the last value of the list. That is why you are only getting the last element in the output json.

google wave OnBlipSubmitted

I'm trying to create a wave robot, and I have the basic stuff working. I'm trying to create a new blip with help text when someone types #help but for some reason it doesnt create it. I'm getting no errors in the log console, and I'm seeing the info log 'in #log'
def OnBlipSubmitted(properties, context):
# Get the blip that was just submitted.
blip = context.GetBlipById(properties['blipId'])
text = blip.GetDocument().GetText()
if text.startswith('#help') == True:
logging.info('in #help')
blip.CreateChild().GetDocument().SetText('help text')

if it just started working, I have two suggestions...
-->Have you been updating the Robot Version in the constructor? You should change the values as you update changes so that the caches can be updated.
if __name__ == '__main__':
myRobot = robot.Robot('waverobotdev',
image_url = baseurl + 'assets/wave_robot_icon.png',
version = '61', # <-------------HERE
profile_url = baseurl)
-->The server connection between Wave and AppSpot has recently been extremely variable. Sometimes it takes 10+ minutes for the AppSpot server to receive my event, othertimes a few seconds. Verify you're receiving the events you expect.
Edit:
The code you provided looks good, so I wouldn't expect you're doing anything wrong in that respect.

Have you tried using Append() instead of SetText()? That's what I'd do in my C# API - I haven't used the Python API, but I'd imagine it's similar. Here's a sample from my demo robot:
protected override void OnBlipSubmitted(IEvent e)
{
if (e.Blip.Document.Text.Contains("robot"))
{
IBlip blip = e.Blip.CreateChild();
ITextView textView = blip.Document;
textView.Append("Are you talking to me?");
}
}
That works fine.
EDIT: Here's the resulting JSON from the above code:
{
"javaClass": "com.google.wave.api.impl.OperationMessageBundle",
"version": "173784133",
"operations": {
"javaClass": "java.util.ArrayList",
"list": [
{
"javaClass": "com.google.wave.api.impl.OperationImpl",
"type": "BLIP_CREATE_CHILD",
"waveId": "googlewave.com!w+PHAstGbKC",
"waveletId": "googlewave.com!conv+root",
"blipId": "b+Iw_Xw7FCC",
"index": -1,
"property": {
"javaClass": "com.google.wave.api.impl.BlipData",
"annotations": {
"javaClass": "java.util.ArrayList",
"list": []
},
"lastModifiedTime": -1,
"contributors": {
"javaClass": "java.util.ArrayList",
"list": []
},
"waveId": "googlewave.com!w+PHAstGbKC",
"waveletId": "googlewave.com!conv+root",
"version": -1,
"parentBlipId": null,
"creator": null,
"content": "\nAre you talking to me?",
"blipId": "410621dc-d7a1-4be5-876c-0a9d313858bb",
"elements": {
"map": {},
"javaClass": "java.util.HashMap"
},
"childBlipIds": {
"javaClass": "java.util.ArrayList",
"list": []
}
}
},
{
"javaClass": "com.google.wave.api.impl.OperationImpl",
"type": "DOCUMENT_APPEND",
"waveId": "googlewave.com!w+PHAstGbKC",
"waveletId": "googlewave.com!conv+root",
"blipId": "410621dc-d7a1-4be5-876c-0a9d313858bb",
"index": 0,
"property": "Are you talking to me?"
}
]
}
}
How does that compare with the JSON which comes out of your robot?

For some reason it just started working. I think the google wave is spotty.