I'm trying to iterate through a list and get key and the corresponding values i need like category, current_severity, event, component_name and component_type.
The challenge i am facing is, the content of the list might change, like the key and values will remain the same but the stanzas might be less, zero or more. So i need something more dynamic.
I need to assign the dynamically created variables to prometheus metrics which will also need to be dynamic.
i tried to create dynamic variables like this, it partially works, like it creates duplicates.
the code i tried:-
url = htpp://url
data = json.loads(r.get(url, headers=headers,
verify=False).content.decode('UTF-8'))
var = string.ascii_lowercase
d = {}
k = 0
for value in data:
d[var[k]] = value['software']
k += 1
d[var[k]] = value['current_severity']
k += 1
print(d)
This prints out duplicates like
{'a': 'software', 'b': low}
{'a': 'software', 'b': low, 'c': 'software', 'd': medium}
I want to assign the values to variables and those to be assigned to prometheus metrics. Prometheus metrics are also to be created based on how many ever variables are there.
My desired output:
metric = Metric('prometheus_metric_name', 'prometheus metric name', 'gauge')
metric.add_sample('prometheus_metric_name', value=float(0), labels={'software':(variable of software will go here), 'severity':(variable of current_severity will go here)})
the metric.add_sample line should be created dynamically for how many ever stanzas have been returned. Below is the list returned by data
[{'category': 'software', 'code': 110, 'actual': '["5.1.4"]', 'opened':
'2018-10-16T09:18:12Z', 'component_type': 'update', 'event': 'new update
available', 'current_severity': 'info', 'details': '', 'expected': None,
'id': 10088862, 'component_name': 'Purity//FA'}, {'category': 'software',
'code': 67, 'actual': None, 'opened': '2018-10-18T01:14:45Z',
'component_type': 'host', 'event': 'misconfiguration', 'current_severity':
'critical', 'details': '', 'expected': None, 'id': 10088898,
'component_name': 'pudc-vm-001'}
By stanza i mean - in the above data there are two stanzas. So how many ever there are, i need to get key and value pairs out of it and assign and create metrics as to those.
Related
I am trying to extract values from Graph database. i am trying with the below gremlin console command but it is returning key value pairs, we can convert this to list.
%%gremlin
g.V().hasLabel('airport').limit(2).project('id','label','region','country').by(id()).by(label()).by('region').by('country').fold()
output
[{'id': '1', 'label': 'airport', 'region': 'US-GA', 'country': 'US'}, {'id': '2', 'label': 'airport', 'region': 'US-AK', 'country': 'US'}]
Expected output:
'1', 'airport', 'US-GA', 'US'
'2', 'airport', 'US-AK', 'US'
or
[['1','airport','US-GA','US'], ['2','airport', 'US-AK','US']]
Rather than use project you can use values. Steps like project and valueMap return a key:value map, whereas values does not include the keys in its result.
gremlin> g.V().
hasLabel('airport').
limit(2).
local(union(id(),label(),values('region','country')).fold())
==>[1,airport,US,US-GA]
==>[2,airport,US,US-AK]
As an alternative you can just add a select(values) to your current query which I think I prefer as it avoids needing the local and union steps.
gremlin> g.V().
hasLabel('airport').
limit(2).
project('id','label','region','country').
by(id()).
by(label()).
by('region').by('country').
select(values).
fold()
==>[[1,airport,US-GA,US],[2,airport,US-AK,US]]
I've got a json format list with some dictionaries within each list, it looks like the following:
[{"id":13, "name":"Albert", "venue":{"id":123, "town":"Birmingham"}, "month":"February"},
{"id":17, "name":"Alfred", "venue":{"id":456, "town":"London"}, "month":"February"},
{"id":20, "name":"David", "venue":{"id":14, "town":"Southampton"}, "month":"June"},
{"id":17, "name":"Mary", "venue":{"id":56, "town":"London"}, "month":"December"}]
The amount of entries within the list can be up to 100. I plan to present the 'name' for each entry, one result at a time, for those that have London as a town. The rest are of no use to me. I'm a beginner at python so I would appreciate a suggestion in how to go about this efficiently. I initially thought it would be best to remove all entries that don't have London and then I can go through them one by one.
I also wondered if it might be quicker to not filter but to cycle through the entire json and select the names of entries that have the town as London.
You can use filter:
data = [{"id":13, "name":"Albert", "venue":{"id":123, "town":"Birmingham"}, "month":"February"},
{"id":17, "name":"Alfred", "venue":{"id":456, "town":"London"}, "month":"February"},
{"id":20, "name":"David", "venue":{"id":14, "town":"Southampton"}, "month":"June"},
{"id":17, "name":"Mary", "venue":{"id":56, "town":"London"}, "month":"December"}]
london_dicts = filter(lambda d: d['venue']['town'] == 'London', data)
for d in london_dicts:
print(d)
This is as efficient as it can get because:
The loop is written in C (in case of CPython)
filter returns an iterator (in Python 3), which means that the results are loaded to memory one by one as required
One way is to use list comprehension:
>>> data = [{"id":13, "name":"Albert", "venue":{"id":123, "town":"Birmingham"}, "month":"February"},
{"id":17, "name":"Alfred", "venue":{"id":456, "town":"London"}, "month":"February"},
{"id":20, "name":"David", "venue":{"id":14, "town":"Southampton"}, "month":"June"},
{"id":17, "name":"Mary", "venue":{"id":56, "town":"London"}, "month":"December"}]
>>> [d for d in data if d['venue']['town'] == 'London']
[{'id': 17,
'name': 'Alfred',
'venue': {'id': 456, 'town': 'London'},
'month': 'February'},
{'id': 17,
'name': 'Mary',
'venue': {'id': 56, 'town': 'London'},
'month': 'December'}]
I would like to filter items out of one dictionary where that dictionary contains items of another dictionary. So, say that I have two dictionary's dict1 and dict2 where
dict1 = {
1:{'account_id':1234, 'case':1234, 'date': 12/31/15, 'content': 'some content'},
2:{'account_id':1235, 'case':1235, 'date': 12/15/15, 'content': 'some content'}
}
dict2 = {
1:{'account_id':1234, 'case':1234, 'date': 12/31/15, 'content': 'some different content'},
2:{'account_id':4321, 'case':4321, 'date': 6/12/15, 'content': 'some different content'},
3:{'account_id':1235, 'case':1235, 'date': 12/15/15, 'content': 'some different content'}
}
I would like to match on account_id, case and date and have the output be a third dictionary with matched entries from dict2 being 1 and 3.
out = {
1:{'account_id':1234, 'case':1234, 'date': 12/31/15, 'content': 'some different content'},
2:{'account_id':1235, 'case':1235, 'date': 12/15/15, 'content': 'some different content'}
}
How would I accomplish this? I am using Python 3.5
Well then, I believe this is what you are looking for:
from itertools import count
from operator import itemgetter
# Set the criteria for unique entry (prevents us from needing to write this twice)
get_identifier = itemgetter("account_id","case","date")
# Create a set of all unique items.
unique_entries = set(map(get_identifier, dict1.values()))
# Get all entries that match one of the unique entries
matched_entires = (d for d in dict2.values() if get_identifier(d) in unique_entries)
# Recreate a new dict together with a counter for items.
out = dict(zip(count(1), matched_entires))
For more info about count() and itemgetter(), see their respective docs.
Using a set and generator comprehensions ensures efficiency at the highest level.
I am trying to extract a specific column or layer, not sure what you want to call it.. that is inside a json object of which I have converted to what I think is a layered list but I have two problems, my check to see if "return" is in the list is not finding anything even though when printing jsonb I can see it is in the list, my second problem is how do I extract a certain column from a layer.. in this case I need the number "43343243" out the second layer and put into a variable I tried referencing it with jsonb["return"][0] and I got a key error..
My code:
def worker(pairtxt):
while (1 < 2):
balanceobject = requests.post(urlauth, headers=headers, data=paybytes)
json_stringb = str(balanceobject.content, 'utf8')
jsonb = json.loads(json_stringb)
print(jsonb)
if "return" in jsonb: #fails
print(jsonb["return"]["0"]) # key error
print(jsonb["return"]) # prints everything even layers under the number
My jsonb print output
{'success': 1, 'return': {'43343243': {'status': 0, 'pair': 'rgeg',
'type': 'sell', 'amount': 0.01000002, 'rate': 1.0,
'timestamp_created': 1502642258}}}
Because 43343243 is a key not a value, you need to get the keys of return :
jsonb = {'success': 1, 'return': {'43343243': {'status': 0, 'pair': 'rgeg', 'type': 'sell', 'amount': 0.01000002, 'rate': 1.0, 'timestamp_created': 1502642258}}}
numberWanted = list(jsonb['return'].keys())[0]
print(numberWanted) # => 43343243
I think you are looking at the output jsonb as a list (which is not true). jsonb is a dictionary. To verify that you can do this:
print(type(jsonb))
A dictionary has key-value pairs.
success is a key and 1 is the value.
return is a key and the value is another dictionary.
{
'43343243': {
'status': 0,
'pair': 'rgeg',
'type': 'sell',
'amount': 0.01000002,
'rate': 1.0,
'timestamp_created': 1502642258
}
}
If you want to access 43343243, then you can do jsonb['return']['43343243']
I'm trying to perform operations on a nested dictionary (data retrieved from a yaml file):
data = {'services': {'web': {'name': 'x'}}, 'networks': {'prod': 'value'}}
I'm trying to modify the above using the inputs like:
{'services.web.name': 'new'}
I converted the above to a list of indices ['services', 'web', 'name']. But I'm not able to/not sure how to perform the below operation in a loop:
data['services']['web']['name'] = new
That way I can modify dict the data. There are other values I plan to change in the above dictionary (it is extensive one) so I need a solution that works in cases where I have to change, EG:
data['services2']['web2']['networks']['local'].
Is there a easy way to do this? Any help is appreciated.
You may iterate over the keys while moving a reference:
data = {'networks': {'prod': 'value'}, 'services': {'web': {'name': 'x'}}}
modification = {'services.web.name': 'new'}
for key, value in modification.items():
keyparts = key.split('.')
to_modify = data
for keypart in keyparts[:-1]:
to_modify = to_modify[keypart]
to_modify[keyparts[-1]] = value
print(data)
Giving:
{'networks': {'prod': 'value'}, 'services': {'web': {'name': 'new'}}}