In AWS, I have a centralized networking account that defines all the VPCs and subnets. And each VPC is shared with target accounts using Resource Access Manager (RAM). Given an IP, need to find out the target account ID with which the VPC/subnet has been shared with. Here is what I have done so far:
In the code below, vpc parameter contains the vpc lookup response and and ip_addr is the IP address we are looking for
def lookup_ipaddr (session, ec2_client, vpc, ip_addr):
found = False
if (ipaddress.ip_address(ip_addr) in ipaddress.ip_network(vpc['CidrBlock'])):
filters = [{'Name':'vpc-id', 'Values':[ vpc['VpcId'] ]}]
subnets = ec2_client.describe_subnets( Filters = filters )['Subnets']
for subnet in subnets:
if (ipaddress.ip_address(ip_addr) in ipaddress.ip_network(subnet['CidrBlock'])):
found = True
tags = subnet['Tags']
# tags returned by previous api is in different form than that required by RAM
for tag in tags:
tag['tagKey'] = tag['Key']
tag['tagValues'] = [tag['Value']]
del tag['Key']
del tag['Value']
print("\n\n")
print (tags)
print("\n\n")
resourceArn = subnet['SubnetArn']
ram_client = session.client('ram')
resp = ram_client.get_resource_shares (resourceOwner = 'SELF', tagFilters=tags)
However the API call get_resource_shares doesn't return any response (except Response Metadata). Any suggestion on how to find out the destination account ID/Principal with which the subnet was shared?
After a bit of digging, I was able to obtain the destination account id by using list_principals api of AWS Resource Access Manager (RAM): https://boto3.amazonaws.com/v1/documentation/api/latest/reference/services/ram.html#RAM.Client.list_principals
Here is the full python code:
def lookup_ipaddr (session, ec2_client, vpc, ip_addr):
found = False
filters = [{'Name':'vpc-id', 'Values':[ vpc['VpcId'] ]}]
subnets = ec2_client.describe_subnets( Filters = filters )['Subnets']
for subnet in subnets:
if (ipaddress.ip_address(ip_addr) in ipaddress.ip_network(subnet['CidrBlock'])):
resourceArn = subnet['SubnetArn']
ram_client = session.client('ram')
resp = ram_client.list_principals(
resourceOwner = 'SELF',
resourceArn = resourceArn
)
print(f"Subnet {subnet['SubnetId']} is shared with account [{resp['principals'][0]['id']}]")
found = True
break
return found
Related
I am intending to set up an Azure Blob storage data source for great expectations. The setup is done with the following string and seems to work, given it lists some files in my blob storage.
example_yaml = f"""
name: {datasource_name}
class_name: Datasource
execution_engine:
class_name: PandasExecutionEngine
azure_options:
account_url: MYURL
credential: MYKEY
data_connectors:
default_runtime_data_connector_name:
class_name: RuntimeDataConnector
batch_identifiers:
- default_identifier_name
default_inferred_data_connector_name:
class_name: InferredAssetAzureDataConnector
azure_options:
account_url: MYURL
credential: MYKEY
container: machinelearning
name_starts_with: waermeprognose/output_neuralnet/FVS/FERNWAERME.1
default_regex:
pattern: (.*)\_FVS_FERNWAERME.1_waermeprognose.parquet
group_names:
- date
"""
Context.test_yaml_config(yaml_config=example_yaml) then returns
Attempting to instantiate class from config...
Instantiating as a Datasource, since class_name is Datasource
Successfully instantiated Datasource
ExecutionEngine class name: PandasExecutionEngine
Data Connectors:
default_inferred_data_connector_name : InferredAssetAzureDataConnector
Available data_asset_names (1 of 1):
DEFAULT_ASSET_NAME (3 of 354): ['waermeprognose/output_neuralnet/FVS/FERNWAERME.1/2021-12-01_FVS_FERNWAERME*1_waermeprognose*parquet', 'waermeprognose/output_neuralnet/FVS/FERNWAERME.1/2021-12-02_FVS_FERNWAERME*1_waermeprognose*parquet', 'waermeprognose/output_neuralnet/FVS/FERNWAERME.1/2021-12-03_FVS_FERNWAERME*1_waermeprognose*parquet']
Unmatched data_references (0 of 0):[]
default_runtime_data_connector_name:RuntimeDataConnector
Available data_asset_names (0 of 0):
Note : RuntimeDataConnector will not have data_asset_names until they are passed in through RuntimeBatchRequest
Unmatched data_references (0 of 0): []
I then save the source to my context using sanitize_yaml_and_save_datasource(context, example_yaml, overwrite_existing=True), but when I try to set up some expectations, the Batch Request is empty:
context = DataContext(
context_root_dir=r'C:\Users\Philip\Documents\Projekte\X\Python\tests\great_expectations'
)
batch_request_parameters = {
"datasource_name": "dlsbeoptdev_machinelearning_waermeprognose_output_neuralnet",
"data_connector_name": "default_inferred_data_connector_name",
"data_asset_name": "DEFAULT_ASSET_NAME",
}
batch_request = BatchRequest(**batch_request_parameters)
print(context.get_batch_list(batch_request=batch_request))
Only return as empty list which I can't use to set up a validator. Did I miss any parameters to request a batch?
I am creating a EC2 instance using CfnInstance in CDK and I would like to use the ARN later in an IAM role, so I can give permission to that specific resource and to avoid to use *. How can I access the ARN of the EC2 instance just created. The code is as follows:
instance_profile = self.create_instance_profile()
self.instance = ec2.CfnInstance(self, 'Client',
image_id = image_id,
instance_type = instance_type,
subnet_id = subnet_id,
iam_instance_profile = instance_profile.ref,
security_group_ids = [cluster_security_group_id],
user_data = core.Fn.base64('\n'.join(self.user_data_commands)),
tags = [{ 'key': 'Name', 'value': 'MskEc2Client' }],
)
def create_instance_profile(self):
role = iam.Role(self, 'Role', assumed_by = iam.ServicePrincipal('ec2.amazonaws.com'))
ssm_policy_statement = iam.PolicyStatement(
resources = ['*'], #TODO GIVE PERMISSION TO THE SPECIFIC RESOURCE (EC2)
actions = [
'ssm:UpdateInstanceInformation', 'ssmmessages:CreateControlChannel',
'ssmmessages:CreateDataChannel', 'ssmmessages:OpenControlChannel', 'ssmmessages:OpenDataChannel'])
ssm_policy = iam.Policy(self, 'SessionManagerPolicy', statements = [ssm_policy_statement])
self.add_w12_suppression(ssm_policy, 'Session Manager actions do not support resource level permissions')
ssm_policy.attach_to_role(role)
msk_policy = iam.Policy(self, 'MskPolicy', #TODO GIVE PERMISSION TO SPECIFIC RESOURCES (EC2)
statements = [iam.PolicyStatement(resources = ['*'], actions = ['kafka:DescribeCluster', 'kafka:GetBootstrapBrokers'])]
)
self.add_w12_suppression(msk_policy, 'MSK actions do not support resource level permissions')
msk_policy.attach_to_role(role)
cfn_role = role.node.default_child
return iam.CfnInstanceProfile(self, 'InstanceProfile', roles = [cfn_role.ref])
You can use the default return value of the instance to construct the arn to populate the resources
ssm_policy_statement = iam.PolicyStatement(
resources = [f'arn:{self.partition}:ec2:{self.region}:{self.account}:instance/{self.instance.ref}'],
actions = [
'ssm:UpdateInstanceInformation', 'ssmmessages:CreateControlChannel',
'ssmmessages:CreateDataChannel', 'ssmmessages:OpenControlChannel', 'ssmmessages:OpenDataChannel'
]
)
I am trying to get all group members of Active Directory.
I have this code:
from ldap3 import Server, Connection, ALL, core
server = Server(address, get_info=ALL)
ad_conn = Connection(server, dn, password, auto_bind=True)
members = []
AD_GROUP_FILTER = '(&(objectClass=GROUP)(cn={group_name}))'
ad_filter = AD_GROUP_FILTER.replace('{group_name}', group_name)
result = ad_conn.search_s('OU details', ldap3.SCOPE_SUBTREE, ad_filter)
if result:
if len(result[0]) >= 2 and 'member' in result[0][1]:
members_tmp = result[0][1]['member']
for m in members_tmp:
email = get_email_by_dn(m, ad_conn)
if email:
members.append(email)
return members
But I am getting an error
'Connection' object has no attribute 'search_s'
Use search(), specify the attributes you need (it seems you build 'email' from user dn but if it were present in the directory), and fix the arguments in function call (arg. order filter then scope, plus use the proper constant SUBTREE) :
from ldap3 import Server, Connection, ALL, core
server = Server(address, get_info=ALL)
ad_conn = Connection(server, dn, password, auto_bind=True)
members = []
AD_GROUP_FILTER = '(&(objectClass=GROUP)(cn={group_name}))'
ad_filter = AD_GROUP_FILTER.replace('{group_name}', group_name)
ad_conn.search('OU details', ad_filter, ldap3.SUBTREE, attributes=['member', 'mail'])
if len(ad_conn.response):
# To grab data, you might prefer the following - or use ad_conn.entries :
for entry in ad_conn.response:
print(entry['dn'], entry['attributes'])
Looking develop a python script that will scan EC2 instances for Public IP's and if found, stop those instances.
I'm sorta new to Python so have challenges trying to put two pieces of code together.
ec2Resource = boto3.resource('ec2')
def lambda_handler(event, context):
instances = ec2Resource.instances.all()
for instance in instances:
#print("inst.public_ip_address",inst.public_ip_address)
if instance.public_ip_address:
print("Instance ID: ",instance.id," , Instance Platform: ",instance.platform," , Public IP: ",instance.public_ip_address,", Instance Type:",instance.instance_type,",Instance Image Id: ",instance.image.id)
response = client.stop_instances(
InstanceIds=[
'string',
],
Hibernate=True,
DryRun=False,
Force=False
)
Basically, looking for a automated script that discovers public IPs on EC2's and then stops them. My apologies if the script above looks hamburglar
UPDATE:
Think I cleaned it up to create lists and then from that list, issue a stop.
#!/usr/bin/python
'''
Finds instance id, Instance Platform, Public IP, instance type based on tags.
Returns a list of instances found
'''
import boto3
def instances_find(name, value):
'''
Finds instance id's based on tags.
Returns a list of instances found.
'''
list_instances = []
# filter based on tags
filters =[
{
'Name': name,
'Values': [
value,
]
},
]
instances = ec2_resource.instances.filter(Filters=filters)
for instance in instances:
# for each instance, append to list
list_instances.append("Instance ID: ",instance.id," , Instance Platform: ",instance.platform," , Public IP: ",instance.public_ip_address,", Instance Type:",instance.instance_type,",Instance Image Id: ",instance.image.id)
return list_instances
def instances_stop(list):
'''
Stops instances defined in the list.
'''
ec2_client.stop_instances(InstanceIds=list)
# enter tag name and value
tag_name = 'tag:environment'
tag_value = 'dev'
ec2_resource = boto3.resource('ec2')
ec2_client = boto3.client('ec2')
# find instances
ec2_list = instances_find(tag_name, tag_value)
# stop instances
ec2_stop = instances_stop(ec2_list)
print('stopped instances: ' + str(ec2_list))
In python xmpp module, I'm able to retrieve the nickname of any contacts as follows:
self.connection.auth(userJid.getNode(), self.password)
self.roster = self.connection.getRoster()
name = self.roster.getName(buddyJid)
..where buddyJid is of the form user#gmail.com.
Now, I need to retrieve the nickname of the user who authenticates the connection (userJid). I cannot find the name using the above method.
Which method can I use retrieve the name of the current user?
This information is not in the roster. You will need to query the clients individually and get their vCard by sending this IQ :
<iq from='stpeter#jabber.org/roundabout'
id='v1'
type='get'>
<vCard xmlns='vcard-temp'/>
</iq>
Thank you nicholas_o, this is a sample function I put together based your suggestion. (The XML logic isn't ideal, but it was sufficient for the simple task I needed this for)
def vcard(disp, jid):
msg = xmpp.protocol.Iq()
msg.setType('get')
msg.setTo(jid)
qc = msg.addChild('vCard')
qc.setAttr('xmlns', 'vcard-temp')
rep = disp.SendAndWaitForResponse(msg)
# to see what other fields are available in the XML output:
# print rep
userid=fname=lname=title=department=region=None
for i in rep.getChildren():
for j in i.getChildren():
if j.getName() == "TITLE":
title = j.getData().encode('utf-8')
for k in j.getChildren():
if k.getName() == "GIVEN":
fname = k.getData().encode('utf-8')
if k.getName() == "FAMILY":
lname = k.getData().encode('utf-8')
if k.getName() == "ORGUNIT":
department = k.getData().encode('utf-8')
if k.getName() == "REGION":
region = k.getData().encode('utf-8')
return fname, lname, title, department, region