Develop Reference

jsii.errors.JSIIError: ARNs must start with "arn:" and have at least 6 components

Similar issues but not in python cdk : How best to retrieve AWS SSM parameters from the AWS CDK?
Hi I am saving a arn in ssm and my variable name is like this
test-value-for-my-job-executor-new-function-lambda-arn
and when I try to get this using ssm like this
from aws_cdk import aws_ssm as ssm
my_arn = ssm.StringParameter.value_from_lookup(self, "test-value-for-my-job-executor-new-function-lambda-arn")
self.scan_pre_process = _lambda.Function.from_function_arn(self, "my-job-executor", my_arn)
I get this error
jsii.errors.JSIIError: ARNs must start with "arn:" and have at least 6 components: dummy-value-for-test-value-for-my-job-executor-new-function-lambda-arn
even I tried to increase name of this still same issue .

value_from_lookup is a Context Method. It looks up the SSM Parameter's cloud-side value once at synth-time and caches its value in cdk.context.json. Context methods return dummy variables until they resolve. That's what's happening in your case. CDK is trying to use my_arn before the lookup has occurred.
There are several fixes:
Quick-and-dirty: Comment out the _lambda.Function.from_function_arn line. Synth the app. Uncomment out the line. This simply forces the caching to happen before the value is used.
Hardcode the Lambda ARN into _lambda.Function.from_function_arn. If the Function name and version/alias are stable, no real downside to hardcoding. Tip: use the Stack.format_arn method.
There is an elegant third alternative using cdk.Lazy. However, I believe Lazy is still broken in Python. Anyway, here is how it would look in Typescript:
const my_arn = cdk.Lazy.string({
produce: () =>
ssm.StringParameter.valueFromLookup(
this,
'test-value-for-my-job-executor-new-function-lambda-arn'
),
});

Sorting Terraform variables.tf by variable name using Python

I am working on creating a new VPC where I need to provide some variables as input.
All the variables are listed in variables.tf. The file is very long (I only copied couple of them here) and variables are defined in no particular order.
I need to find a Pythonic way to sort my variables.tf by variable name.
variable "region" {
description = "The region to use when creating resources"
type = string
default = "us-east-1"
}
variable "create_vpc" {
description = "Controls if VPC should be created"
type = bool
default = true
}
variable "name" {
description = "Name to be used, no default, required"
type = string
}
The sorted variables.tf should look like this:
variable "create_vpc" {
description = "Controls if VPC should be created"
type = bool
default = true
}
variable "name" {
description = "Name to be used, no default, required"
type = string
}
variable "region" {
description = "The region to use when creating resources"
type = string
default = "us-east-1"
}

"Pythonic" might be the wrong approach here - you are still comfortably sitting behind a python interpreter, but for better or worse (worse) you are playing by Terraform's rules. Check out the links below. Hashicorp "enables" python via their CDK, and there are several other projects out there on github.
Once you are up and running with something like that, and you have Terraform fully ported over to your Python setup, then you can start thinking pythonic. </IMO>
https://github.com/hashicorp/terraform-cdk
https://github.com/beelit94/python-terraform/blob/develop/python_terraform/terraform.py

Here's what I came across last year https://github.com/hashicorp/terraform/issues/12959 and is why I created https://gist.github.com/yermulnik/7e0cf991962680d406692e1db1b551e6 out of curiosity. Not Python but awk :shrugging:
Simple awk script to sort TF files. Not just variables, but any of 1st level resource definition blocks.

Naming CDK resources dynamically

I'm using the CDK to create some infrastructure from a yaml template file. Some resources require multiple instances. I thought writing a function would be the easiest way to create multiple instance of the resource
Function
def create_vpn_connection_route(cidr_count, destination_cidr):
vpn_connection_route = aws_ec2.CfnVPNConnectionRoute(
self,
f'vpn_connection_route{cidr_count}',
vpn_connection_id=vpn_connection.ref,
destination_cidr_block=destination_cidr
)
return vpn_connection_route
I then loop over it and generate the "Id" by enumarating over the destination_cidrs like so
for cidr_count, destination_cidr in enumerate(tenant_config['vpn_config'][0]['destination_cidrs']):
create_vpn_connection_route(cidr_count, destination_cidr)
This is what's in my yaml
vpn_config:
- private_ip:
- 10.1.195.201/32
- 10.1.80.20/32
- 10.1.101.8/32
Is there a better way to do this in the CDK? and can I dynamically generate Id'S for resources?
Cheers

I don't know that it makes your code much better, but you can use a Construct instead of a function.
class VpnConnectionRoute(core.Construct):
def __init__(self, scope, id_, vpn_connection, destination_cidr):
super().__init__(scope, id_)
self.vpn_connection_route = aws_ec2.CfnVPNConnectionRoute(
self,
'vpn_connection_route',
vpn_connection_id=vpn_connection.vpn_id,
destination_cidr_block=destination_cidr
)
# ...
for cidr_count, destination_cidr in enumerate(tenant_config['vpn_config'][0]['destination_cidrs']):
VpnConnectionRoute(self, f"route{cidr_count}", vpn_connection, destination_cidr)
VpnConnectionRoute(self, f"route{cidr_count}", vpn_connection, destination_cidr)
VpnConnectionRoute(self, f"route{cidr_count}", vpn_connection, destination_cidr)
CDK will automatically name your resources based on both the construct and your name. So the end result will look like:
"route1vpnconnectionrouteAE1C11A9": {
"Type": "AWS::EC2::VPNConnectionRoute",
"Properties": {
"DestinationCidrBlock": "10.1.195.201/32",
"VpnConnectionId": {
"Ref": "Vpn6F669752"
}
},
"Metadata": {
"aws:cdk:path": "app/route1/vpn_connection_route"
}
},
You can also just put destination_cidr inside your route name. CDK will remove all unsupported characters for you automatically.
for destination_cidr in tenant_config['vpn_config'][0]['destination_cidrs']:
aws_ec2.CfnVPNConnectionRoute(
self,
f'VPN Connection Route for {destination_cidr}',
vpn_connection_id=vpn_connection.vpn_id,
destination_cidr_block=destination_cidr
)
The best solution here probably depends on what you want to happen when these addresses change. For this particular resource type, any change in the name or the values will require a replacement anyway. So keeping the names consistent while the values change might not matter that much.

How to pass a Troposphere object attribute conditionally?

I'm using Troposphere to build CloudFormation stacks and would like to pass the Elastic Load Balancer ConnectionSettings attribute only if it's set in my configuration, otherwise I don't want to specify it.
If I set it to default None then I get an error about the value not being of the expected type of troposphere.elasticloadbalancing.ConnectionSettings.
I'd rather try to avoid setting an explicit default in the call because it might override other settings.
Idealy, I would like to be able to add attributes to an existing object, e.g.:
lb = template.add_resource(elb.LoadBalancer(
...
))
if condition:
lb.add_attribute(ConnectionSettings = elb.ConnectionSettings(
...
))
Is there a way to achieve that?
UPDATE: I achieved it using a hidden Troposphere method, which works but I'm not happy with:
if condition:
lb.__setattr__('ConnectionSettings', elb.ConnectionSettings(
....
))
I'm still interested in a solution which doesn't involve using a private method from outside the module.

The main README eludes to just using the attribute names like this:
from troposphere import Template
import troposphere.elasticloadbalancing as elb
template = Template()
webelb = elb.LoadBalancer(
'ElasticLoadBalancer',
Listeners=[
elb.Listener(
LoadBalancerPort="80",
InstancePort="80",
Protocol="HTTP",
),
],
)
if True:
webelb.ConnectionSettings = elb.ConnectionSettings(IdleTimeout=30)
elasticLB = template.add_resource(webelb)
print(template.to_json())

So the big question is - where does the configuration for ConnectionSettings come from? Inside Cloudformation itself (and troposphere) there are Conditions, Parameters, and the AWS::NoValue Ref. I use that fairly heavily in the stacker RDS templates:
Here's the Parameter: https://github.com/remind101/stacker/blob/master/stacker/blueprints/rds/base.py#L126
Here's the condition: https://github.com/remind101/stacker/blob/master/stacker/blueprints/rds/base.py#L243
And here's how it's used in a resource later, optionally - if the StorageType Parameter is blank, we use AWS::NoValue, which is a pseudo Ref for not actually setting something: (Sorry, can't post more than 2 links - go to line 304 in the same file to see what I'm talking about)
If you're not using Parameters however, and instead doing all your conditions in python, you could do something similar. Something like:
connection_setting = condition and <actual connection setting code> or Ref("AWS::NoValue")
The other option is to do it entirely in python, which is basically your example. Hopefully that helps, there are a lot of ways to deal with this, including creating two different ELB objects (one with connection settings, one without) and then picking either one with either python code (if condtion) or cloudformation conditions.

If the value is known in Python (i.e., it does not originate from a CloudFormation Parameter), you can use a dictionary to add optional attributes to resources in a Troposphere template:
from troposphere import Template
import troposphere.elasticloadbalancing as elb
template = Template()
my_idle_timeout = 30 # replace this with your method for determining the value
my_elb_params = {}
if my_idle_timeout is not None:
my_elb_params['ConnectionSettings'] = elb.ConnectionSettings(
IdleTimeout=my_idle_timeout,
)
my_elb = template.add_resource(elb.LoadBalancer(
'ElasticLoadBalancer',
Listeners=[
elb.Listener(
LoadBalancerPort="80",
InstancePort="80",
Protocol="HTTP",
),
],
**my_elb_params,
))
print(template.to_json())
If the value originates from a CloudFormation Parameter, you need to create a Condition to test the value of the parameter and use Ref("AWS::NoValue") if no value was provided for the parameter, e.g.:
from troposphere import Template, Parameter, Equals, Ref, If
import troposphere.elasticloadbalancing as elb
template = Template()
my_idle_timeout = template.add_parameter(Parameter(
"ElbIdleTimeout",
Description="Idle timeout for the Elastic Load Balancer",
Type="Number",
))
no_idle_timeout = "NoIdleTimeout"
template.add_condition(
no_idle_timeout,
Equals(Ref(my_idle_timeout), ""),
)
my_elb = template.add_resource(elb.LoadBalancer(
'ElasticLoadBalancer',
Listeners=[
elb.Listener(
LoadBalancerPort="80",
InstancePort="80",
Protocol="HTTP",
),
],
ConnectionSettings=If(
no_idle_timeout,
Ref("AWS::NoValue"),
elb.ConnectionSettings(
IdleTimeout=Ref(my_idle_timeout),
),
),
))
print(template.to_json())

Making google analytics ID a variable

My app serves multiple domains which I understand should be done by namespaces which I'm researching. Since multiple domains should have multiple analytics ID:s I get the analytics ID from the code but I want to make it even more configurable:
if os.environ.get('HTTP_HOST').endswith('.br') \
or os.environ['SERVER_NAME'].endswith('.br'):
data[u'analytics'] = 'UA-637933-12'
else:
data[u'analytics'] = 'UA-637933-18'
self.response.out.write(template.render(os.path.join(os.path.dirname(__file__),
'templates', name + '.html'), data))
The above sets analytics ID to ..-12 if it's my brazilian domain and to the other ID ...-18 if it is my dot com. But this is only for 2 domains and it's not easiliy generalizable. How can I achieve this function in a more scientific and scalable way so that it becomes easy to add my application to a domain without manually adding the domain to my application?
I suppose namespaces is the way to go here since the domains are google apps domains but I don't understand how to use namespaces:
def namespace_manager_default_namespace_for_request():
"""Determine which namespace is to be used for a request.
The value of _NAMESPACE_PICKER has the following effects:
If _USE_SERVER_NAME, we read server name
foo.guestbook-isv.appspot.com and set the namespace.
If _USE_GOOGLE_APPS_DOMAIN, we allow the namespace manager to infer
the namespace from the request.
If _USE_COOKIE, then the ISV might have a gateway page that sets a
cookie called 'namespace', and we set the namespace to the cookie's value
"""
name = None
if _NAMESPACE_PICKER == _USE_SERVER_NAME:
name = os.environ['SERVER_NAME']
elif _NAMESPACE_PICKER == _USE_GOOGLE_APPS_DOMAIN:
name = namespace_manager.google_apps_namespace()
elif _NAMESPACE_PICKER == _USE_COOKIE:
cookies = os.environ.get('HTTP_COOKIE', None)
if cookies:
name = Cookie.BaseCookie(cookies).get('namespace')
return name
I suppose I should use the namespace manager, get the namespace and set the analytics ID according to the namespace but how?
Thank you

The simplest way to do this is with a Python dict:
analytics_ids = {
'mydomain.br': 'UA-637933-12',
'mydomain.com': 'UA-637933-18',
}
data['analytics'] = analytics_ids[self.request.host]
If you have other per-domain stats, you may want to make each dictionary entry a tuple, a nested dict, or a configuration object of some sort, then fetch and store it against the current request for easy reference.
If you want to be able to reconfigure this at runtime, you could use a datastore model, but that will impose extra latency on requests that need to fetch it; it seems likely to me that redeploying each time you add a domain isn't likely to be a problem in your case.
Namespaces are tangential to what you're doing. They're a good way to divide up the rest of your data between different domains, but they're not useful for dividing up configuration data.

I presume you have two instances of the same application running.
Instead of fiddling with namespaces, I suggest you turn the Analytics ID into a configuration variable.
That is, either store it in a config file or a database your web is using. Then set one ID for each deployment (in each place your web is running from) and fetch it in the runtime.
For example:
Config file:
analyticsId="UA-637933-12"
Code:
data[u'analytics'] = getValueFromConfig("analyticsId")
where getValueFromConfig is a function you define to read the appropriate value. (To use configuration files effortlessly, you may use the ConfigParser module.)
Now you've gained a lot more flexibility - you don't have to do any checking and switching at runtime. You only have to define the value once per web site and be done with it.