PyYAML replace dash in keys with underscore - python

I would like to map directly some configuration parameters from YAML into Python argument names. Just wondering if there is a way without writing extra-code (to modify keys afterwards) to let YAML parser replace dash '-' in a key with an underscore '_'.
some-parameter: xyz
some-other-parameter: 123
Should become when parsed with PyYAML (or may be other lib) a dictionary with values:
{'some_parameter': 'xyz', 'some_other_parameter': 123}
Than I can pass the dictionary to a function as named parameters:
foo(**parsed_data)
I know I can iterate through the keys afterwards and modify their values, but I don't want to do that :)

At least for your stated case, you don't need to transform the keys. Given:
import pprint
def foo(**kwargs):
print 'KWARGS:', pprint.pformat(kwargs)
If you set:
values = {
'some-parameter': 'xyz',
'some-other-parameter': 123,
}
And then call:
foo(**values)
You get:
KWARGS: {'some-other-parameter': 123, 'some-parameter': 'xyz'}
If you goal is is actually to call a function like this:
def foo(some_parameter=None, some_other_parameter=None):
pass
Then sure, you would need to map the key names. But you could just do this:
foo(**dict((k.replace('-','_'),v) for k,v in values.items()))

I think I found a solution: There is a package which is called yconf: https://pypi.python.org/pypi/yconf
I can map there values and work with it using the well-known argparse-interface:
config.yml
logging:
log-level: debug
Argparse like definition:
parser.add_argument("--log-level", dest="logging.log-level")

Well, If you parse the YAML file as a python dictionary, you can use the following code to convert all dash (inside all nested dictionaries and arrays) with dash.
def hyphen_to_underscore(dictionary):
"""
Takes an Array or dictionary and replace all the hyphen('-') in any of its keys with a underscore('_')
:param dictionary:
:return: the same object with all hyphens replaced by underscore
"""
# By default return the same object
final_dict = dictionary
# for Array perform this method on every object
if type(dictionary) is type([]):
final_dict = []
for item in dictionary:
final_dict.append(hyphen_to_underscore(item))
# for dictionary traverse all the keys and replace hyphen with underscore
elif type(dictionary) is type({}):
final_dict = {}
for k, v in dictionary.items():
# If there is a sub dictionary or an array perform this method of it recursively
if type(dictionary[k]) is type({}) or type(dictionary[k]) is type([]):
value = hyphen_to_underscore(v)
final_dict[k.replace('-', '_')] = value
else:
final_dict[k.replace('-', '_')] = v
return final_dict
Here is a sample usage
customer_information = {
"first-name":"Farhan",
"last-name":"Haider",
"address":[{
"address-line-1": "Blue Mall",
"address-line-2": None,
"address-type": "Work"
},{
"address-line-1": "DHA",
"address-line-2": "24-H",
"address-type": "Home"
}],
"driver_license":{
"number": "209384092834",
"state-region": "AB"
}
}
print(hyphen_to_underscore(customer_information))
# {'first_name': 'Farhan', 'last_name': 'Haider', 'address': [{'address_line_1': 'Blue Mall', 'address_line_2': None, 'address_type': 'Work'}, {'address_line_1': 'DHA', 'address_line_2': '24-H', 'address_type': 'Home'}], 'driver_license': {'number': '209384092834', 'state_region': 'AB'}}

Related

Nested Dictionary (JSON): Merge multiple keys stored in a list to access its value from the dict

I have a JSON with an unknown number of keys & values, I need to store the user's selection in a list & then access the selected key's value; (it'll be guaranteed that the keys in the list are always stored in the correct sequence).
Example
I need to access the value_key1-2.
mydict = {
'key1': {
'key1-1': {
'key1-2': 'value_key1-2'
},
},
'key2': 'value_key2'
}
I can see the keys & they're limited so I can manually use:
>>> print(mydict['key1']['key1-1']['key1-2'])
>>> 'value_key1-2'
Now after storing the user's selections in a list, we have the following list:
Uselection = ['key1', 'key1-1', 'key1-2']
How can I convert those list elements into the similar code we used earlier?
How can I automate it using Python?
You have to loop the list of keys and update the "current value" on each step.
val = mydict
try:
for key in Uselection:
val = val[key]
except KeyError:
handle non-existing keys here
Another, more 'posh' way to do the same (not generally recommended):
from functools import reduce
val = reduce(dict.get, Uselection, mydict)

Creating python objectify Element and then adding attributes

I am trying to serialize a python class named Origin, containing a dictionary as an attribute, into an xml with lxml objectify. This dictionary is initialized with the value "default" for each key.
class Origin:
def __init__(self):
dict = {"A":"default", "B":"default" ...} // my dictionnary has 6 keys actually
The dictionary is filled by first parsing a XML. Not every key is filled. For exemple: dict = {A:"hello", B:"default" ...}
I want to create my Origin XML Element with my dictionary as attribute but I don't want to have the "default" keys.
My solution is to have nested if:
ìf(self.dict["A"] != "default"):
if(self.dict["B"] != "default"):
...
objectify.Element("Origin", A=self.dict["A"], B=self.dict["B"]...)
But it's an ugly and non practical solution if you have more than one or two keys.
Is there a way to first create my Element origin = objectify.Element("Origin") and then add my dictionary keys' if there are different from "default"?
Something more dynamic, like
for key in self.dict:
if(self.dict[key] != "default"):
origin.addAttribute(key=self.dict[key])
Thank you
I would filter the dictionary to only the values that are not "default".
The dict comprehension feature is a big help here.
Example:
data = {
"A": "foo",
"B": "default",
"C": "bar",
}
data = {key: value for key, value in data.items() if value != "default"}
print(data)
Output:
{'A': 'foo', 'C': 'bar'}

Iteration through nested JSON objects

I'm a beginner in Python pulling JSON data consisting of nested objects (dictionaries?). I'm trying to iterate through everything to locate a key all of them share, and select only the objects that have a specific value in that key. I spent days researching and applying and now everything is kind of blurring together in some mix of JS/Python analysis paralysis. This is the general format for the JSON data:
{
"things":{
"firstThing":{
"one":"x",
"two":"y",
"three":"z"
},
"secondThing":{
"one":"a",
"two":"b",
"three":"c"
},
"thirdThing":{
"one":"x",
"two":"y",
"three":"z"
}
}
}
In this example I want to isolate the dictionaries where two == y. I'm unsure if I should be using
JSON selection (things.things[i].two)
for loop through things, then things[i] looking for two
k/v when I have 3 sets of keys
Can anyone point me in the right direction ?
Assuming this is only ever one level deep (things), and you want a 'duplicate' of this dictionary with only the matching child dicts included, then you can do this with a dictionary comprehension:
data = {
"things":{
"firstThing":{
"one":"x",
"two":"y",
"three":"z"
},
"secondThing":{
"one":"a",
"two":"b",
"three":"c"
},
"thirdThing":{
"one":"x",
"two":"y",
"three":"z"
}
}
}
print({"things": {k:v for k, v in data['things'].items() if 'two' in v and v['two'] == 'y'}})
Since you've tagged this with python I assume you'd prefer a python solution. If you know that your 'two' key (whatever it is) is only present at the level of objects that you want, this might be a nice place for a recursive solution: a generator that takes a dictionary and yields any sub-dictionaries that have the correct key and value. This way you don't have to think too much about the structure of your data. Something like this will work, if you're using at least Python 3.3:
def findSubdictsMatching(target, targetKey, targetValue):
if not isinstance(target, dict):
# base case
return
# check "in" rather than get() to allow None as target value
if targetKey in target and targetKey[target] == targetValue:
yield target
else:
for key, value in target.items():
yield from findSubdictsMatching(value, targetKey, targetValue)
This code allows You to add objects with "two":"y" to list:
import json
m = '{"things":{"firstThing":{"one":"x","two":"y","three":"z"},"secondThing":{"one":"a","two":"b","three":"c"},"thirdThing":{"one":"x","two":"y","three":"z"}}}'
l = json.loads(m)
y_objects = []
for key in l["things"]:
l_2 = l["things"][key]
for key_1 in l_2:
if key_1 == "two":
if l_2[key_1] == 'y':
y_objects.append(l_2)
print(y_objects)
Console:
[{'one': 'x', 'two': 'y', 'three': 'z'}, {'one': 'x', 'two': 'y', 'three': 'z'}]

Formatting dicts and nested dicts

Amazon's DynamoDB requires specially formatted JSON when inserting items into the database.
I have a function that takes a dictionary and transforms values into a nested dict formatted for insertion; the value is transformed into a nested dict where the nested key is the value's data type.
For example, input like {'id':1, 'firstName':'joe'} would be transformed to {'id': {'N':1}, 'firstName': {'S':'joe'}}
This is currently successful with this function:
type_map = {
str:'S', unicode:'S', dict:'M',
float:'N', int:'N', bool:'BOOL'
}
def format_row(self, row):
""" Accepts a dict, formats for DynamoDB insertion. """
formatted = {}
for k,v in row.iteritems():
type_dict = {}
type_dict[ self.type_map[type(v)] ] = v
formatted[k] = type_dict
return formatted
I need to modify this function to handle values that might be dicts.
So, for example:
{
'id':1,
'a':{'x':'hey', 'y':1},
'b':{'x':1}
}
Should be transformed to:
{
'id': {'N':1},
'a':{'M': {'x': {'S':'hey'}, 'y':{'N':1}}},
'b': {'M': {'x': {'N':1}}}
}
I'm thinking the correct way to do this must be to call the function from within the function right?
Note: I'm using Python 2.7
What ultimately ended up working for me was the following function:
def format_row(self, row):
""" Accepts a dict, formats for DynamoDB insertion. """
formatted = {}
for k,v in row.iteritems():
if type(v) == dict:
v = self.format_row(v)
type_dict = {}
type_dict['M'] = v
formatted[k] = type_dict
else:
type_dict = {}
type_dict[ self.type_map[type(v)] ] = v
formatted[k] = type_dict
return formatted
If anyone has a better way of doing this, please let me know!

Retrieve a key-value pair from a dict as another dict

I have a dictionary that looks like:
{u'message': u'Approved', u'reference': u'A71E7A739E24', u'success': True}
I would like to retrieve the key-value pair for reference, i.e. { 'reference' : 'A71E7A739E24' }.
I'm trying to do this using iteritems which does return k, v pairs, and then I'm adding them to a new dictionary. But then, the resulting value is unicode rather than str for some reason and I'm not sure if this is the most straightforward way to do it:
ref = {}
for k, v in charge.iteritems():
if k == 'reference':
ref['reference'] = v
print ref
{'reference': u'A71E7A739E24'}
Is there a built-in way to do this more easily? Or, at least, to avoid using iteritems and simply return:
{ 'reference' : 'A71E7A739E24' }
The trouble with using iteritems is that you increase lookup time to O(n) where n is dictionary size, because you are no longer using a hash table
If you only need to get one key-value pair, it's as simple as
ref = { key: d[key] }
If there may be multiple pairs that are selected by some condition,
either use dict from iterable constructor (the 2nd version is better if your condition depends on values, too):
ref = dict(k,d[k] for k in charge if <condition>)
ref = dict(k,v for k,v in charge.iteritems() if <condition>)
or (since 2.7) a dict comprehension (which is syntactic sugar for the above):
ref = {k,d[k] for k in charge if <condition>}
<same as above>
I dont understand the question:
is this what you are trying to do:
ref={'reference',charge["reference"]}

Categories

Resources