I'm using ConfigObj 5.0.6 to hold many user-defined values, some of which are nested. I use a local.ini to supercede typical values. There is no front-end, so users edit the configs as needed. To make that easier and more intuitive, there are some values that belong at the 'root' level of the config object, but are more easily understood below a nested section of the local.ini file.
I'm using a local.ini to supercede defaults. The flow of the app suggests a config layout that would have non-nested values below nested values.
# un-nested
title = my_title
# nested
[section_1]
val_s1 = val
[section_2]
val_s2 = val
# nested, but I want to be un-nested
val_2 = val
This layout, as expected, puts val_2 under section_2:
{
'title': 'my_title',
{'section_1': {'val_s1': 'val'}},
{'section_2': {'val_s2': 'val'},
{'val_2': 'val'}}
}
Is it possible to define val_2 on a line below section_2, but access it under the 'main' section of the config object?
I would like to end up with a config object like this:
{
'title': 'my_title',
{'section_1': {'val_s1': 'val'}},
{'section_2': {'val_s2': 'val'}},
'val_2': 'val'
}
The order of the config dictionary isn't important, of course; what I'm interested in is being able to use nested sections, but from within the .ini, exit a section into its parent.
I haven't tested, but suspect nesting everything from the first line onward and then slicing the config object would work. I.e., write local.ini such that it creates:
{
'main_level':
{
'title': 'my_title',
{'section_1': {'val_s1': 'val'}},
{'section_2': {'val_s2': 'val'}},
'val_2': 'val'
}
}
Then I could use config = config['main_level'] when I first instantiate the config object, but I'm wondering if I'm just missing some easy, correct way that isn't a hack.
According to the documentation, that is not possible:
In the outer section, single values can only appear before any sub-section.
Related
I'm using the CDK to create some infrastructure from a yaml template file. Some resources require multiple instances. I thought writing a function would be the easiest way to create multiple instance of the resource
Function
def create_vpn_connection_route(cidr_count, destination_cidr):
vpn_connection_route = aws_ec2.CfnVPNConnectionRoute(
self,
f'vpn_connection_route{cidr_count}',
vpn_connection_id=vpn_connection.ref,
destination_cidr_block=destination_cidr
)
return vpn_connection_route
I then loop over it and generate the "Id" by enumarating over the destination_cidrs like so
for cidr_count, destination_cidr in enumerate(tenant_config['vpn_config'][0]['destination_cidrs']):
create_vpn_connection_route(cidr_count, destination_cidr)
This is what's in my yaml
vpn_config:
- private_ip:
- 10.1.195.201/32
- 10.1.80.20/32
- 10.1.101.8/32
Is there a better way to do this in the CDK? and can I dynamically generate Id'S for resources?
Cheers
I don't know that it makes your code much better, but you can use a Construct instead of a function.
class VpnConnectionRoute(core.Construct):
def __init__(self, scope, id_, vpn_connection, destination_cidr):
super().__init__(scope, id_)
self.vpn_connection_route = aws_ec2.CfnVPNConnectionRoute(
self,
'vpn_connection_route',
vpn_connection_id=vpn_connection.vpn_id,
destination_cidr_block=destination_cidr
)
# ...
for cidr_count, destination_cidr in enumerate(tenant_config['vpn_config'][0]['destination_cidrs']):
VpnConnectionRoute(self, f"route{cidr_count}", vpn_connection, destination_cidr)
VpnConnectionRoute(self, f"route{cidr_count}", vpn_connection, destination_cidr)
VpnConnectionRoute(self, f"route{cidr_count}", vpn_connection, destination_cidr)
CDK will automatically name your resources based on both the construct and your name. So the end result will look like:
"route1vpnconnectionrouteAE1C11A9": {
"Type": "AWS::EC2::VPNConnectionRoute",
"Properties": {
"DestinationCidrBlock": "10.1.195.201/32",
"VpnConnectionId": {
"Ref": "Vpn6F669752"
}
},
"Metadata": {
"aws:cdk:path": "app/route1/vpn_connection_route"
}
},
You can also just put destination_cidr inside your route name. CDK will remove all unsupported characters for you automatically.
for destination_cidr in tenant_config['vpn_config'][0]['destination_cidrs']:
aws_ec2.CfnVPNConnectionRoute(
self,
f'VPN Connection Route for {destination_cidr}',
vpn_connection_id=vpn_connection.vpn_id,
destination_cidr_block=destination_cidr
)
The best solution here probably depends on what you want to happen when these addresses change. For this particular resource type, any change in the name or the values will require a replacement anyway. So keeping the names consistent while the values change might not matter that much.
I have such .proto file
syntax = "proto3";
import "google/protobuf/any.proto";
message Request {
google.protobuf.Any request_parameters = 1;
}
How can I create Request object and populate its fields? I tried this:
import ma_pb2
from google.protobuf.any_pb2 import Any
parameters = {"a": 1, "b": 2}
Request = ma_pb2.Request()
some_any = Any()
some_any.CopyFrom(parameters)
Request.request_parameters = some_any
But I have an error:
TypeError: Parameter to CopyFrom() must be instance of same class: expected google.protobuf.Any got dict.
UPDATE
Following prompts of #Kevin I added new message to .proto file:
message Small {
string a = 1;
}
Now code looks like this:
Request = ma_pb2.Request()
small = ma_pb2.Small()
small.a = "1"
some_any = Any()
some_any.Pack(small)
Request.request_parameters = small
But at the last assignment I have an error:
Request.request_parameters = small
AttributeError: Assignment not allowed to field "request_parameters" in protocol message object.
What did I do wrong?
Any is not a magic box for storing arbitrary keys and values. The purpose of Any is to denote "any" message type, in cases where you might not know which message you want to use until runtime. But at runtime, you still need to have some specific message in mind. You can then use the .Pack() and .Unpack() methods to convert that message into an Any, and at that point you would do something like Request.request_parameters.CopyFrom(some_any).
So, if you want to store this specific dictionary:
{"a": 1, "b": 2}
...you'll need a .proto file which describes some message type that has integer fields named a and b. Personally, I'd see that as overkill; just throw your a and b fields directly into the Request message, unless you have a good reason for separating them out. If you "forget" one of these keys, you can always add it later, so don't worry too much about completeness.
If you really want a "magic box for storing arbitrary keys and values" rather than what I described above, you could use a Map instead of Any. This has the advantage of not requiring you to declare all of your keys upfront, in cases where the set of keys might include arbitrary strings (for example, HTTP headers). It has the disadvantage of being harder to lint or type-check (especially in statically-typed languages), because you can misspell a string more easily than an attribute. As shown in the linked resource, Maps are basically syntactic sugar for a repeated field like the following (that is, the on-wire representation is exactly the same as what you'd get from doing this, so it's backwards compatible to clients which don't support Maps):
message MapFieldEntry {
key_type key = 1;
value_type value = 2;
}
repeated MapFieldEntry map_field = N;
I am trying to put parent/child relationships into ElasticSearch. I was able to put my parents into ES but when I try to add the child objects I get the following error
elasticsearch.exceptions.RequestsError: TransportError(400:, 'illegal_argument_exception', "Can't specify parent if no parent field has been configured")
I understand, that I have to tell ES that my parents are actually parents somewhere in mappings. But where and how?
This is how I do it right now:
# the parent
es.index(index='www01', doc_type='domain', id=k, body=json.dumps(rest))
# the child
es.index(index='www01', doc_type='framework', id=key, body=json.dumps(value), parent=k)
Edit:
Alright. I think I've got it now. You really have to type everything that deviates from the standard settings explicitly into a mapping and create an empty index with that mapping first.
In this mapping you have to say that your child type has a parent with a specific type. (The ElasticSearch documentation is pretty clear) RTFM
After you've created your specific mapping on an empty index, you can put your stuff in.
Note: You don't have to specify everything in your mapping, just everything that is not "standard". So at first it is enough to just give the relationships. If you later want to say that some fields are dates or geo-coordinates you can later specify that. But keep in mind, that you have to empty your index first. So save your data somewhere!
Here is what I did:
def del_all_indices():
print "----- deleting the following indices (not >>.kibana<<) -----"
for idx in es.indices.get('*'):
print idx
if not idx == ".kibana":
es.indices.delete(index=idx, ignore=[400, 404])
print "----- remaining indices -----"
for idx in es.indices.get('*'):
print idx
create_index_body = {
"mappings": {
"domain": {},
"framework": {
"_parent": {
"type": "domain"
}
}
}
}
del_all_indices()
es.indices.create(index='www01', body=create_index_body)
# the parent
es.index(index='www01', doc_type='domain', id=k, body=json.dumps(rest))
# the child
es.index(index='www01', doc_type='framework', id=key, body=json.dumps(value), parent=k)
It goes without saying, that if I'm missing something, please tell me. (I hope this helps, since there is almost no documentation on that topic.)
I am looking for the "good pythonic" way to define a dictionary with base elements. Let me explain:
I have a serie of key:value that configure my system. A group of key:value represent a configuration. Each configuration must have several common base elements. On the other hand each configuration has the possibility to define extra elements.
I could use python dictionaries but I do not think that could enforce the user to define "Mandatory_Elt_1" & "Mandatory_Elt_2" (or is it?):
Config_1 = { 'Mandatory_Elt_1':1,
'Mandatory_Elt_2':2,
'Optional_Elt_3':3 }
Config_2 = { 'Mandatory_Elt_1':5,
'Mandatory_Elt_2':6,
'Optional_Elt_4':7 }
A second option would be to subclass dict and define a specific constructor method where I would require the mandatory elements:
Config = MyDict( { 'Mandatory_Elt_1':5,
'Mandatory_Elt_2':6,
'Optional_Elt_4':7 } )
A third possibility would be to review the "input interface" such that other coders would not directly touch the base data. Instead, they would need to pass through a sanitization API. Or should I move to a databse based system altogether?
Are there any other alternatives? What would be the "lowest friction" way to force some elements to be defined in a dictionary?
If you need some mandatory elements in final dictionary,
you may use a basic dictionary and then update it with the user's dictionary.
E.g.
base_config = {'m1':1, 'm2': 2}
user_config = {'m1':3, 'new_param':4}
base_config.update(user_config)
final_config = base_config
>>> print final_config
{'new_param': 4, 'm1': 3, 'm2': 2}
I found some solutions to iterate through dict in django templates (such as How to iterate over nested dictionaries in django templates) but it's always for a defined depth.
For instance, if I have this dict :
{'a':{'b':'c',
'd':'e',
'f':'g',
'h':{'i':'j',
'k':'l',
'm':'n'
}
}
'o':'p'
}
But I can also have :
{'a':{'b':{'c':{'d':'e',
'f':'g'
}
}
}
}
Is there another way than the recursive one to do this ? If not, is it possible to do recursion in django templates ?
The only way to do this I think is to create custom filters such as get_sons and has_son and work with it. Is it the good way to do this ?
Additional informations :
I have the structure stored in a model such as :
class XMLStructure(Model.models):
node_name = models.CharField()
parent = models.ForeignKey('XMLStructure')
#staticmethod
def get_root(self):
# return the root of the structure
def has_children(self):
# return true / false if it's a leaf
#staticmethod
def get_leaves(self):
# return all leaves of my xml
and a function build who generate the xml structure as a dict.
I tried to use code from Django Snippets but I cannot make them work (Always got an RuntimeError: maximum recursion depth exceeded while calling a Python object)
I though it would be easier to transform my dict into a list. My first dict would be :
dict_as_list = ['a','/','b','d','f','h','/','i','k','m','\','\','o','\']
and then store the values into a dict. As you can see, I'm pretty desperate.
Thanks