Develop Reference

BOTO3 using Python to fetch information of a list of EC2

I am trying to compile information from a list of EC2s that I have on a .csv using Python + Boto3.
This .csv contains the Private IPs of those instances. The following command returns everything that I need:
aws ec2 describe-network-interfaces --filters Name=addresses.private-ip-address,Values="<PRIVATE IP>" --region <MY REGION>
So I've decided to use Boto3 to do something similar.
But my code isn't returning the information inside the dictionary because I cannot specify the Region inside the code.
The documentation allows me to specify the Availability Zone, but it won't just work.
ec2 = boto3.client('ec2')
describe_network_interfaces = ec2.describe_network_interfaces(
Filters=[
{
'Name': 'addresses.private-ip-address',
'Values': [
'<PRIVATE IP>'
],
'Name': 'availability-zone',
'Values': [
'<REGION>'
]
}
],
MaxResults=123
)
print(describe_network_interfaces)
☝️ This returns me this 👇
{'NetworkInterfaces': [], 'ResponseMetadata': { <LOTS OF METADATA> }}
I believe it is not working because I can't specify the Region with describe_network_interfaces of Boto3. But I can do it with the AWS CLI command.
Any suggestions?
OBS: popen is not a good idea for this current project.
Thanks in advance.

You can set the region at the client level with something like:
my_region = "us-east-1"
ec2 = boto3.client('ec2', region_name=my_region)
This worked in my environment successfully to get information about systems running in another region.

Custom wait condition in boto3 for efs

boto3 provides default waiters for some services like EC2, S3, etc. This is not provided by default for all services. Now, I've a case where an EFS volume is created and the lifecycle policy is added to the file system. The EFS creation takes some time and the lifecycle policy isn't in the required efs state. i.e., efs created.
How to wait for EFS to be created in a python boto3 code, so that policies can be added?

Looks like a waiter doesn't exist for this, but you can create your own customer waiters like this:
import boto3
from botocore.waiter import WaiterModel
from botocore.waiter import create_waiter_with_client
client = boto3.client('efs')
waiter_name = "LifeCycleCompleted"
waiter_config = {
"version": 2,
"waiters": {
"LifeCycleCompleted": {
"operation": "DescribeLifecycleConfiguration",
"delay": 60, # Number of seconds to delay
"maxAttempts": 5, # Max attempts before failure
"acceptors": [
{
"matcher": "path",
"expected": lifecycle_policy,
"argument": "length(LifecyclePolicies[]) > `0`",
"state": "success"
}
]
}
}
}
waiter_model = WaiterModel(waiter_config)
efs_lifecycle_waiter = create_waiter_with_client(waiter_name, waiter_model, client)
efs_lifecycle_waiter.wait(FileSystemId='MyFileSystemId')

Or Use something simpler as mentioned by #jordanm
import time
import boto3
efs_client = boto3.client('efs')
filesystem_status=""
while filesystem_status != "available":
response = efs_client.describe_file_systems(
FileSystemId=file_system_id,
)
filesystem_status=response["FileSystems"][0]["LifeCycleState"]
print(f"{file_system_id} {filesystem_status} .... waiting")
time.sleep(20)

How can I create and manage a Lambda Function from Python Script?

I would like to manage my AWS Lambda Functions from Python script, without having to use the AWS Website Console. The idea is to be able to quickly recreate/migrate/setup my applications objects (tables, functions etc.) into a new AWS Cloud Account or Region.
It is easy to do this with DynamoDB tables, for instance:
import boto3
resource = boto3.resource(service_name='dynamodb', region_name='region', ...)
resource.create_table(
TableName='table_name',
KeySchema=[...],
AttributeDefinitions={...},
ProvisionedThroughput={...}
)
Done! I just created a new DynamoDB table from Python Script. How can I do the same for Lambda Functions? Say... create a new function, configure ‘Function Name’ and ‘Runtime’, maybe set up a ‘Role’ and upload a script from file. That’d be really helpful.
Thanks in advance.

To create lambda function using boto3, you can use create_function.
The AWS docs also provide an example of how to use the create_function:
response = client.create_function(
Code={
'S3Bucket': 'my-bucket-1xpuxmplzrlbh',
'S3Key': 'function.zip',
},
Description='Process image objects from Amazon S3.',
Environment={
'Variables': {
'BUCKET': 'my-bucket-1xpuxmplzrlbh',
'PREFIX': 'inbound',
},
},
FunctionName='my-function',
Handler='index.handler',
KMSKeyArn='arn:aws:kms:us-west-2:123456789012:key/b0844d6c-xmpl-4463-97a4-d49f50839966',
MemorySize=256,
Publish=True,
Role='arn:aws:iam::123456789012:role/lambda-role',
Runtime='nodejs12.x',
Tags={
'DEPARTMENT': 'Assets',
},
Timeout=15,
TracingConfig={
'Mode': 'Active',
},
)
print(response)

Can I get a list of AWS EC2 instances running for each of my accounts?

I have a list of AWS accounts with account ids. I want to fetch the list of all the EC2 instances running in each of those accounts. Can I fetch that list using boto3?

First, you can access your AWS account programmatically by creating boto3 client.
import boto3
client = boto3.client(
's3',
aws_access_key_id=ACCESS_KEY,
aws_secret_access_key=SECRET_KEY,
aws_session_token=SESSION_TOKEN,
)
See: https://boto3.amazonaws.com/v1/documentation/api/latest/guide/configuration.html#method-parameters
After creating boto3 client with your credentials, You can use boto3's EC2.Client.describe_instances()
response = client.describe_instances(
Filters=[
{
'Name': 'string',
'Values': [
'string',
]
},
],
InstanceIds=[
'string',
],
DryRun=True|False,
MaxResults=123,
NextToken='string'
)
Use 'instance-state-name' : 'running' in Filters, like:
{
'Name': 'instance-state-name',
'Values': [
'running',
]
}
then you can get informations of your running ec2 instances.

To access AWS Account from CLI/Programs, you need to have 'aws_access_key_id_value' and 'aws_secret_access_key_value'. Once you have these details, you can use the following code to list EC2 instances from the AWS Account you specified keys of.
Create config.properties with the following code.
aws_access_key_id_value='YOUR-ACCESS-KEY-OF-THE-AWS-ACCOUNT'
aws_secret_access_key_value='YOUR-SECRETE-KEY-OF-THE-AWS-ACCOUNT'
Create list-ec2-instances.py with the following code.
import boto3
def getVarFromFile(filename):
import imp
f = open(filename)
global data
data = imp.load_source('data', '', f)
f.close()
getVarFromFile('config.properties')
ec2client = boto3.resource(
'ec2',
aws_access_key_id=data.aws_access_key_id_value,
aws_secret_access_key=data.aws_secret_access_key_value
)
for instance in ec2client.instances.all():
print(
"Id: {0}\nPlatform: {1}\nType: {2}\nPublic IPv4: {3}\nAMI: {4}\nState: {5}\n".format(
instance.id, instance.platform, instance.instance_type, instance.public_ip_address, instance.image.id, instance.state
)
)
Execute the following command.
python list-ec2-instances.py

aws: boto3 get all instances of a load balancers

I can able to get the load balancers using below
import boto3
elb = boto3.client('elbv2')
lbs = elb.describe_load_balancers()
How to get the instances of the lbs.
Also How Can I fetch the load balancers which state is not active as describe_load_balanacers only give state active load balanceres.

Classic Load Balancer
Use: client = boto3.client('elb')
Then describe_load_balancers() results include a list of instances:
'Instances': [
{
'InstanceId': 'string'
},
],
Application Load Balancer
Use: client = boto3.client('elbv2')
Call describe_target_groups() passing in the Load Balancer ARN to obtain a list of Target Groups associated with the Load Balancer
Then call describe_target_health() to obtain a list of targets (instances).
Here is a sample response:
{
'TargetHealthDescriptions': [
{
'Target': {
'Id': 'i-0f76fade',
'Port': 80,
},
...

for anyone looking for a quick snippet to see if your instance is in the LB:
from ec2_metadata import ec2_metadata
instance_id: str = ec2_metadata.instance_id
import boto3
client = boto3.client("elbv2" , region_name="us-west-2")
response = client.describe_target_groups(
LoadBalancerArn="your arn goes here"
)
target_group_arn = response["TargetGroups"][0]["TargetGroupArn"]
response = client.describe_target_health(TargetGroupArn=target_group_arn)
instances = map(lambda x: x["Target"]["Id"], response["TargetHealthDescriptions"])
print(f"target group instances {list(instances)}")
print(f"this instance {instance_id}")