Develop Reference

Unable to register ECS task from boto3

I am trying to create a basic cloud infrastructure which can run through any AWS account. I am using python boto3 module to create a cluster,task definition and a service, which are created successfully from my code. However the task in not running from my service, when I checked the Events I can see error 'service my_service was unable to place a task because no container instance met all of its requirements. Reason: No Container Instances were found in your cluster'.
import boto3
client = boto3.client('ecs')
cluster_response = client.create_cluster(
clusterName='my_cluster',
)
print(cluster_response)
taskdef_response = client.register_task_definition(
family = 'my_taskdef',
containerDefinitions=[
{
'name': 'my_taskdef',
'image': '****/sample:latest'
}
],
memory='256',
cpu='1024',
taskRoleArn='ecsTaskExecutionRole'
)
print(taskdef_response)
service_response = client.create_service(
cluster='my_cluster',
serviceName='my_service',
taskDefinition='my_taskdef',
desiredCount=1,
launchType='EC2'
)
print(service_response)
I am expecting this to run my docker image in the task spawned by the service. The AWS account does not have any existing EC2 instance and I want to run a task that creates EC2 instance from this code(without the need of changing anything from inside EC2 instance). How can I run the task?

Connecting a python AWS Lambda function to ElastiCache

I'm trying to set and get keys from ElastiCache (memcached) from a python lambda function using Boto3. I can figure out how to get the endpoints but that's pretty much it. Is there some documentation out there that shows the entire process?

It sounds like you are trying to interact with Memcached via Boto3. This is not possible. Boto3 is for interacting with the AWS API. You can manage your ElastiCache servers via the AWS API, but you can't interact with the Memcached software running on those servers. You need to use a Memcached client library like python-memcached in your Python code to actually get and set keys in your Memcached cluster.
Also, your Lambda function will need to reside in the same VPC as the ElastiCache node(s).

I had the exact timeout problem listed in the commment of the older post. My bug is in the security group for memcached. Here is the working version in terraform:
resource "aws_security_group" "memcached" {
vpc_id = "${aws_vpc.dev.id}"
name = "memcached SG"
ingress {
from_port = "${var.memcached_port}"
to_port = "${var.memcached_port}"
protocol = "tcp"
cidr_blocks = ["${var.public_subnet_cidr}"]
}
egress {
from_port = "${var.memcached_port}"
to_port = "${var.memcached_port}"
protocol = "tcp"
cidr_blocks = ["${var.public_subnet_cidr}"]
}
tags = {
Name = "memcached SG"
}
}
I tested the connection by creating a EC2 instance in public subnet and do "telnet (input your cache node URL) 11211".

Is there a way for a Python script to "recognize" which EC2 instance it is running on?

Let's say that my script is running on an EC2 instance named ec2-test1 and it communicates with an S3 bucket named: s3-bucket-test1, but when the script is ran on ec2-test2 it's able to identify the EC2 instance it is currently running on and change the directory to reference s3-bucket-test2. Is there a way to do that? I know that for internal paths you can use os.path.dirname(os.path.realpath(__file__)) but was wondering if there is a way to do something like that for EC2 instance name in Python?

Use the following Boto3 to get the current instance name. Warning: No exception handling is included.
import boto3
import os
def get_inst_name():
# Get Instance ID from EC2 Instance Metadata
inst_id = os.popen("curl -s http://169.254.169.254/latest/meta-data/instance-id").read()
# Get EC2 instance object
ec2 = boto3.resource('ec2')
inst = ec2.Instance(inst_id)
# Obtain tags associated with the EC2 instance object
for tags in inst.tags:
if tags["Key"] == 'Name':
#print tags["Value"]
return tags["Value"]
Get the instance id from metadata server
Use the instance id to query Boto3 resource

How to create an ec2 instance using boto3

Is it possible to create an ec2 instance using boto3 in python?
Boto3 document is not helping here, and I couldn't find any helping documents online. please provide some sample codes/links.

The API has changed but it's right there in the documentation
# Boto 3
ec2.create_instances(ImageId='<ami-image-id>', MinCount=1, MaxCount=5)
Link to the documentation:
http://boto3.readthedocs.org/en/latest/guide/migrationec2.html#launching-new-instances

You can run the code I used from the boto3 docs. You can add or remove parameters as per your requirements, but this is what you would normally require:
import boto3
client = boto3.client('ec2', region_name='us-west-2')
response = client.run_instances(
BlockDeviceMappings=[
{
'DeviceName': '/dev/xvda',
'Ebs': {
'DeleteOnTermination': True,
'VolumeSize': 8,
'VolumeType': 'gp2'
},
},
],
ImageId='ami-6cd6f714',
InstanceType='t3.micro',
MaxCount=1,
MinCount=1,
Monitoring={
'Enabled': False
},
SecurityGroupIds=[
'sg-1f39854x',
],
)

The link you're really looking for in the documentation is the create_instances() method of the ServiceResource object. This is the type of object you are calling if you create an EC2 resource like this:
s = boto3.Session(region_name="us-west-1")
ec2 = s.resource('ec2')
...
instance = ec2.create_instances(**y_kwargs)
This contains a more detailed example and a longer list of available parameters.
You can also get parameter values for AWS instances that are already running using the AWS command line interface:
$ aws ec2 describe-instances
This prints out a JSON file from which relevant parameters can be extracted and passed to the create_instances() method. (Or, you can use a boto client and call the describe_instances() method.)
(Note: If you're wondering what the difference is between the Client and the Resource, they serve different purposes for the same end - the client is a lower-level interface while the Resource is a higher-level interface.)

Refer to API docs has all available options to create instance
http://boto3.readthedocs.org/en/latest/reference/services/ec2.html#EC2.Subnet.create_instances

If your running from your windows computer you need configure AWS Cli with proper EC2 permisssion to launch instance.
#
import boto3
ec2 = boto3.resource('ec2')
instance = ec2.create_instances(
ImageId='ami-5eb63a32',
MinCount=1,
MaxCount=1,
InstanceType='t2.micro',
)
print(instance[0].id)

Python Boto EC2 find instance given its IP address

Using boto in Python, how can I find the boto.ec2 instance object given an IP address?

boto3
ec2 = boto3.client('ec2')
filters = [{
'Name': 'ip-address',
'Values': ['1.1.1.1'],
}]
result_list = ec2.describe_instances(Filters=filters)

Digging through the boto documentation, I found the get_only_instances method, which you use to get all instances. You can pass a filter dictionary to it, to filter by IP Address (I found this in the EC2 API Reference under the Filter.N title).
So for example, to get the instance with IP 1.1.1.1, you would do:
filters = {"ip-address": "1.1.1.1"}
result_list = conn.get_only_instances(filters=filters)
Then result_list[0] should be the Instance object for the instance with that IP address.