Using boto in Python, how can I find the boto.ec2 instance object given an IP address?
boto3
ec2 = boto3.client('ec2')
filters = [{
'Name': 'ip-address',
'Values': ['1.1.1.1'],
}]
result_list = ec2.describe_instances(Filters=filters)
Digging through the boto documentation, I found the get_only_instances method, which you use to get all instances. You can pass a filter dictionary to it, to filter by IP Address (I found this in the EC2 API Reference under the Filter.N title).
So for example, to get the instance with IP 1.1.1.1, you would do:
filters = {"ip-address": "1.1.1.1"}
result_list = conn.get_only_instances(filters=filters)
Then result_list[0] should be the Instance object for the instance with that IP address.
Related
Need to find a way to identify AWS VPC Subnets that route through virtual private gateway by using Python Boto3. In another word, how can I use python boto3 to identify Private Subnets within a VPC?
The goal is to create a Lambda function that will identify private subnets within a given VPC, and then to launch another Lambda function within those Private Subnets.
Below is the code I got so far. It lists all subnets within a VPC that has a Virtual Private Gateway attached.
import boto3
def get_vpn_gateways():
ec2_client = boto3.client('ec2')
response = ec2_client.describe_vpn_gateways()
return response
def get_vpc_subnets(VpcId):
ec2 = boto3.resource('ec2')
vpc = ec2.Vpc(VpcId)
subnets = vpc.subnets.all()
return subnets
# Get VPC Ids associated with the virtual private gateway
vpc_list = []
virtual_gateways = get_vpn_gateways()
for virtual_gateway in virtual_gateways["VpnGateways"]:
vgwId = virtual_gateway["VpnGatewayId"]
vpcAttach = virtual_gateway["VpcAttachments"]
vpc_list.append(vpcAttach[0]["VpcId"])
for vpc in vpc_list:
print(vpc)
subnets = get_vpc_subnets(vpc)
for subnet in subnets:
print(subnet)
The code so far lists all the subnets within the VPC. I am thinking to use routetable as the key identifier for Private subnet. If there are routes going through VGW, then I will deem the subnet as Private. Does that make sense?
I think the routing for 0.0.0.0/0 is not an internet gateway, then that is the private subnet. The private subnet could be routed to NAT gateway or Virtual gateway but not the internet gateway directly. So, I wrote code as below.
import boto3
ec2 = boto3.resource('ec2')
route_tables = ec2.route_tables.all()
for route_table in route_tables:
for ra in route_table.routes_attribute:
if ra.get('DestinationCidrBlock') == '0.0.0.0/0' and ra.get('GatewayId') is None:
for rs in route_table.associations_attribute:
if rs.get('SubnetId') is not None:
print(rs.get('SubnetId'))
Here's the final working code that looks up private subnets within each VPC that has Virtual Private Gateway attached. It checks if the private subnets is in the VPC's subnet list, then proceeds to save it later for another Lambda function. This probably not the most effective/efficient way to achive my goal. Keen to see any other better solutions.
import boto3
def get_vpn_gateways():
ec2_client = boto3.client('ec2')
response = ec2_client.describe_vpn_gateways()
return response
def get_vpc_subnets(VpcId):
ec2 = boto3.resource('ec2')
vpc = ec2.Vpc(VpcId)
subnets = vpc.subnets.all()
return subnets
def get_private_subnets():
priv_subnet_list = []
ec2 = boto3.resource('ec2')
route_tables = ec2.route_tables.all()
for route_table in route_tables:
for ra in route_table.routes_attribute:
if ra.get('DestinationCidrBlock') == '0.0.0.0/0' and ra.get('GatewayId') is None:
for rs in route_table.associations_attribute:
if rs.get('SubnetId') is not None:
priv_subnet_list.append(rs.get('SubnetId'))
return priv_subnet_list
def lambda_handler(event, context):
vpc_list = []
vpc_subnet_list = []
virtual_gateways = get_vpn_gateways()
lambda_subnets = []
# Get VPC Ids associated with the virtual private gateway
for virtual_gateway in virtual_gateways["VpnGateways"]:
vgwId = virtual_gateway["VpnGatewayId"]
vpcAttach = virtual_gateway["VpcAttachments"]
vpc_list.append(vpcAttach[0]["VpcId"])
# Get subnets within the VPC
for vpc in vpc_list:
subnets = get_vpc_subnets(vpc)
for subnet in subnets:
vpc_subnet_list.append(subnet.id)
# Get Private subnets from the subnet list
for privsubnet in get_private_subnets():
if privsubnet in vpc_subnet_list:
lambda_subnets.append(privsubnet)
Below is my code to launch an instance on custom VPC but i am seeing below error. Need your assistance in finding the problem:
import json
import boto3
def lambda_handler(event, context):
session_client=boto3.client(service_name='ec2')
response = session_client.run_instances(
#SecurityGroupIds=['sg-0ffb80bb93205a004'],
#SubnetId='subnet-0bb30254459fd4714',
MaxCount=1,
MinCount=1,
NetworkInterfaces=[
{
'NetworkInterfaceId':'eni-0846a5c0c47e98b3c',
'Description': 'Network Interface Description',
'DeviceIndex': 123,
'SubnetId': 'subnet-0bb30254459fd4714'
},
],
LaunchTemplate={
'LaunchTemplateId': 'lt-0d3c7dd295796ad6b',
'Version': '1'
})
Error: Network interfaces and an instance-level subnet ID may not be specified on the same request",
"errorType": "ClientError",
Thanks,
Bapu Reddy Battu
There are two subnetid parameters that can be passed while creating an EC2 instance through boto3
1) SubnetId at root level in the request :
SubnetId (string) -- [EC2-VPC] The ID of the subnet to launch the
instance into. If you specify a network interface, you must specify
any subnets as part of the network interface.
Since you are passing the NetworkInterfaces parameter, this SubnetId will not be passed.
2) SubnetId as part of NetworkInterfaces parameter
SubnetId (string) -- The ID of the subnet associated with the network
interface. Applies only if creating a network interface when launching
an instance.
Since you are using an already existing network interface. You should not pass the SubnetId parameter inside NetworkInterfaces either.
Reference documentation : EC2 run instances - Boto3 documentation
I want to pass the volume id as a parameter which then returns the instance id in python
You will need to call describe_instances().
You can either filter the results yourself in Python, or pass Filters for block-device-mapping.volume-id.
import boto3
ec2_client = boto3.client('ec2', region_name='ap-southeast-2')
response = ec2_client.describe_instances(Filters=[{'Name':'block-device-mapping.volume-id','Values':['vol-deadbeef']}])
instance_id = response['Reservations'][0]['Instances'][0]['InstanceId']
print(instance_id)
A volume can only be attached to one instance at a time, so this code assumes only one instance is returned.
As #Rajesh pointed out, an easier way is to use DescribeVolumes, which returns Attachment information:
import boto3
ec2_client = boto3.client('ec2', region_name='ap-southeast-2')
response = ec2_client.describe_volumes(VolumeIds=['vol-deadbeef'])
print(response['Volumes'][0]['Attachments'][0]['InstanceId'])
This code assumes that the instance is the first attachment on the volume (since EBS volumes can only be attached to one instance).
I'm trying to set and get keys from ElastiCache (memcached) from a python lambda function using Boto3. I can figure out how to get the endpoints but that's pretty much it. Is there some documentation out there that shows the entire process?
It sounds like you are trying to interact with Memcached via Boto3. This is not possible. Boto3 is for interacting with the AWS API. You can manage your ElastiCache servers via the AWS API, but you can't interact with the Memcached software running on those servers. You need to use a Memcached client library like python-memcached in your Python code to actually get and set keys in your Memcached cluster.
Also, your Lambda function will need to reside in the same VPC as the ElastiCache node(s).
I had the exact timeout problem listed in the commment of the older post. My bug is in the security group for memcached. Here is the working version in terraform:
resource "aws_security_group" "memcached" {
vpc_id = "${aws_vpc.dev.id}"
name = "memcached SG"
ingress {
from_port = "${var.memcached_port}"
to_port = "${var.memcached_port}"
protocol = "tcp"
cidr_blocks = ["${var.public_subnet_cidr}"]
}
egress {
from_port = "${var.memcached_port}"
to_port = "${var.memcached_port}"
protocol = "tcp"
cidr_blocks = ["${var.public_subnet_cidr}"]
}
tags = {
Name = "memcached SG"
}
}
I tested the connection by creating a EC2 instance in public subnet and do "telnet (input your cache node URL) 11211".
Is it possible to create an ec2 instance using boto3 in python?
Boto3 document is not helping here, and I couldn't find any helping documents online. please provide some sample codes/links.
The API has changed but it's right there in the documentation
# Boto 3
ec2.create_instances(ImageId='<ami-image-id>', MinCount=1, MaxCount=5)
Link to the documentation:
http://boto3.readthedocs.org/en/latest/guide/migrationec2.html#launching-new-instances
You can run the code I used from the boto3 docs. You can add or remove parameters as per your requirements, but this is what you would normally require:
import boto3
client = boto3.client('ec2', region_name='us-west-2')
response = client.run_instances(
BlockDeviceMappings=[
{
'DeviceName': '/dev/xvda',
'Ebs': {
'DeleteOnTermination': True,
'VolumeSize': 8,
'VolumeType': 'gp2'
},
},
],
ImageId='ami-6cd6f714',
InstanceType='t3.micro',
MaxCount=1,
MinCount=1,
Monitoring={
'Enabled': False
},
SecurityGroupIds=[
'sg-1f39854x',
],
)
The link you're really looking for in the documentation is the create_instances() method of the ServiceResource object. This is the type of object you are calling if you create an EC2 resource like this:
s = boto3.Session(region_name="us-west-1")
ec2 = s.resource('ec2')
...
instance = ec2.create_instances(**y_kwargs)
This contains a more detailed example and a longer list of available parameters.
You can also get parameter values for AWS instances that are already running using the AWS command line interface:
$ aws ec2 describe-instances
This prints out a JSON file from which relevant parameters can be extracted and passed to the create_instances() method. (Or, you can use a boto client and call the describe_instances() method.)
(Note: If you're wondering what the difference is between the Client and the Resource, they serve different purposes for the same end - the client is a lower-level interface while the Resource is a higher-level interface.)
Refer to API docs has all available options to create instance
http://boto3.readthedocs.org/en/latest/reference/services/ec2.html#EC2.Subnet.create_instances
If your running from your windows computer you need configure AWS Cli with proper EC2 permisssion to launch instance.
#
import boto3
ec2 = boto3.resource('ec2')
instance = ec2.create_instances(
ImageId='ami-5eb63a32',
MinCount=1,
MaxCount=1,
InstanceType='t2.micro',
)
print(instance[0].id)