Develop Reference

Python Boto3 SnapshotNotFound exception. Unable to delete snapshot

I'm new to AWS and I have written a Boto3 script which gets the snapshots by OwnerId and delete older snapshots one by one. I'm having a strange issue that boto client finds all Snapshots and when it reaches client.delete_snapshot(SnapshotId=snapshot.snapshot_id) It throws SnapshotNotFoundException - Unable to delete snapshot with id abcd.
It's weird that when i go to AWS Console and search for the ID, Snapshot is there and I can delete it from the AWS Console and I'm using the same account credentials with Boto3 config file.
[default]
aws_access_key_id=foo
aws_secret_access_key=bar
Here is what i have tried.
Boto3 Script
from datetime import datetime, timedelta, timezone
import boto3
ec2 = boto3.resource('ec2')
cl = boto3.client('ec2')
count=0
snapshots = ec2.snapshots.filter(OwnerIds=['xxxxxxxxxx'])
def if_associated_to_ami(client, snapshot_id):
img = client.describe_images(Filters=[{'Name': 'block-device-mapping.snapshot-id', 'Values': [snapshot_id]}])
try:
ami_id = img['Images'][0]['ImageId']
#print("Snapshot(" + snapshot_id + ") is associated to image(" + ami_id + "). Return True")
return True
except IndexError:
#print("Snapshot(" + snapshot_id + ") is not associated to any image. Return False")
return False
for snapshot in snapshots:
if if_associated_to_ami(cl, snapshot.snapshot_id):
print('Unabble to delete Snapshot with Id = {}. Snapshot is in used! '. format(snapshot.snapshot_id))
else:
start_time = snapshot.start_time
delete_time = datetime.now(tz=timezone.utc) - timedelta(days=90)
if delete_time > start_time:
#snapshot.delete()
cl.delete_snapshot(SnapshotId=snapshot.snapshot_id)
print('Snapshot with Id = {} is deleted '. format(snapshot.snapshot_id))
count+=1
if count == 1000:
break
if you face indentation issues, please check the file here.
https://github.com/DeveloperMujtaba/usual-resources/blob/master/boto3.py
can someone please indicate the issue please? It would be much appreciated. Thanks.

Honestly, just by looking at your code, I cannot tell why it bulks at that but also, b/c you already have the snapshot resource, why not just do:
snapshot.delete()

table.put_item w/ Partition and Sort Key in Python (Lambda to DynamoDB)

I am altering a Lambda function (written in Python) that puts an item into a DynamoDB table. I have no issues when using the function to write to a DDB table with only a primary key, but get the following error after aligning attribute names and writing to a table with a partition and sort key:
An error occurred (ValidationException) when calling the PutItem operation:
One or more parameter values were invalid: Missing the key ID
Here is my function:
# import the json utility package since we will be working with a JSON object
import json
# import the AWS SDK (for Python the package name is boto3)
import boto3
# import two packages to help us with dates and date formatting
from time import gmtime, strftime
# create a DynamoDB object using the AWS SDK
dynamodb = boto3.resource('dynamodb')
# use the DynamoDB object to select our table
table = dynamodb.Table('PSCartTracking')
# store the current time in a human readable format in a variable
now = strftime("%a, %d %b %Y %H:%M:%S +0000", gmtime())
# define the handler function that the Lambda service will use as an entry point
def lambda_handler(event, context):
# extract values from the event object we got from the Lambda service and store in a variable
login = event['login']
computer = event['computer']
timestamp = event['timestamp']
# write name and time to the DynamoDB table using the object we instantiated and save response in a variable
response = table.put_item(
Item={
'ID': computer,
'Timestamp': timestamp,
'Login': login
})
# return a properly formatted JSON object
return {
'statusCode': 200,
'body': json.dumps(login + ' checked out ' + computer + ' on ' + now)
}
Keys for Table:
Partition key
ID (String)
Sort key
Timestamp (String)
I've been trying to read documentation to figure out what about my formatting might be preventing it from recognizing the sort key, but I'm pretty positive that is the part of the composite it is not seeing.

Looking for a Python script to delete AWS snapshot by giving snapshot ID or Instance name as parameter with retention date

Hi I am looking for python script to delete snapshot by giving snapshot id or Instance name and retention period as parameter. I have the below script that deletes all of the snapshots older that the retention period set, however want to perform deletion for a specific instance name only.
# Delete snapshots older than retention period
import boto3
from botocore.exceptions import ClientError
from datetime import datetime,timedelta
def delete_snapshot(snapshot_id, reg):
try:
ec2resource = boto3.resource('ec2', region_name=reg)
snapshot = ec2resource.Snapshot(snapshot_id)
snapshot.delete()
except ClientError as e:
print "Caught exception: %s" % e
return
def lambda_handler(event, context):
account_id = 'xxxxxxxxxxxxxxx'
retention_days = 10
# Get current timestamp in UTC
now = datetime.now()
# AWS Account ID
# Define retention period in days
# Create EC2 client
ec2 = boto3.client('ec2')
# Get list of regions
regions = ec2.describe_regions().get('Regions',[] )
# Iterate over regions
for region in regions:
print "Checking region %s " % region['RegionName']
reg=region['RegionName']
# Connect to region
ec2 = boto3.client('ec2', region_name=reg)
# Filtering by snapshot timestamp comparison is not supported
# So we grab all snapshot id's
result = ec2.describe_snapshots( OwnerIds=[account_id] )
for snapshot in result['Snapshots']:
print "Checking snapshot %s which was created on %s" % (snapshot['SnapshotId'],snapshot['StartTime'])
# Remove timezone info from snapshot in order for comparison to work below
snapshot_time = snapshot['StartTime'].replace(tzinfo=None)
# Subtract snapshot time from now returns a timedelta
# Check if the timedelta is greater than retention days
if (now - snapshot_time) > timedelta(retention_days):
print "Snapshot is older than configured retention of %d days" % (retention_days)
delete_snapshot(snapshot['SnapshotId'], reg)
else:
print "Snapshot is newer than configured retention of %d days so we keep it" % (retention_days)

It appears you are asking how to identify which Amazon EC2 instance is associated with a given Snapshot.
A simple method is to add a tag when creating the Snapshot, with the tag containing the Instance ID and possibly even the tags from the instance. This means all relevant information would be attached to the snapshot itself.
Otherwise, you can:
Obtain the VolumeId from the results of describe_snapshots()
Call describe_volumes() passing in the VolumeId
If the volume is still attached to the EC2 instance, then Attachments.InstanceId will provide the instance identifier
If the volume is not attached to an instance, this information will not be available. However, when launching an instance, tags can be propagated from the instance to the volume. This could be a good way of identifying the purpose of the volume.
You could alternatively take a different approach:
Start with the list of Instance IDs
For each instance:
Call describe_instance() to obtain the VolumeId of attached volumes
Call describe_snapshots(), passing the VolumeId
This will just give the snapshots for the given instance, rather than starting with the snapshots and having to discover the instance.
You could do the same thing with tags by calling describe_instances() for a given tag, then loop through each resulting instance.

how to display all snapshots in my aws account using python boto3

the aim of this program is to delete snapshots that are older than 60 days. when run it displays the following error " a=snapshot[s].start_time
AttributeError: 'dict' object has no attribute 'start_time' " This is my code
#!/usr/bin/env python
import boto3
import datetime
client = boto3.client('ec2')
snapshot= client.describe_snapshots()
for s in snapshot:
a=snapshot[s].start_time
b=a.date()
c=datetime.datetime.now().date()
d=c-b
if d.days>60 :
snapshot[s].delete(dry_run=True)

Your error is in the line a=snapshot[s].start_time, use a=s.start_time
Note I would change "snapshot" to "snapshots". then in your for loop:
for snapshot in snapshots:
This makes the code easier to read and clear on what your variables represent.
Another item is that start_time is a string. You will need to parse this to get a number. Here is an example to help you:
delete_time = datetime.utcnow() - timedelta(days=days)
for snapshot in snapshots:
start_time = datetime.strptime(
snapshot.start_time,
'%Y-%m-%dT%H:%M:%S.000Z'
)
if start_time < delete_time:
***delete your snapshot here***

This should do it-
import boto3
import json
import sys
from pprint import pprint
region = 'us-east-1'
ec2 = boto3.client('ec2', region)
resp = ec2.describe_instances()
resp_describe_snapshots = ec2.describe_snapshots(OwnerIds=['*******'])
snapshot = resp_describe_snapshots['Snapshots']
snapshots = [''];
for snapshotIdList in resp_describe_snapshots['Snapshots']:
snapshots.append(snapshotIdList.get('SnapshotId'))
for id in snapshots:
print(id)

Adding expire functionality to python-rom objects

I am working with flask and redis. I've using the rom redis orm (http://pythonhosted.org/rom/) to manage some mildly complex data structures. I want to add the ability to set the objects to set an expiry time.
Based on https://github.com/josiahcarlson/rom/issues/40 and https://github.com/josiahcarlson/rom/pull/47
I have a rom model:
class A(rom.Model):
url = rom.String(required=True, unique=True)()
t = rom.String()
delete_at = rom.Float(index=True)
created_at = rom.Float(default=time.time, index=True)
which I can instantiate and save:
a_object = A(url=u, delete_at =time.time+7200) # 2 hour expiry
try:
ad_object.save()
except Exception,e:
print str(e)
I have a cronjob which executes every hour and so I want to do something like:
delete_list = A.get_by((time.time()-delete_at>0)) # pseudocode.
Obviously this is incorrect, but if I can get this into a list I could delete these. How can I express the above pseudocode using the rom ORM?

I emailed Josiah, the package developer. His answer:
"This can be done in one of 2 ways:
delete_list = A.get_by(delete_at=(0, time.time()))
delete_list = A.query.filter(delete_at=(0, time.time())).all()
The query attribute/object offers a collection of different filers, depending on your defined indexes."