I am trying to extract values from Graph database. i am trying with the below gremlin console command but it is returning key value pairs, we can convert this to list.
%%gremlin
g.V().hasLabel('airport').limit(2).project('id','label','region','country').by(id()).by(label()).by('region').by('country').fold()
output
[{'id': '1', 'label': 'airport', 'region': 'US-GA', 'country': 'US'}, {'id': '2', 'label': 'airport', 'region': 'US-AK', 'country': 'US'}]
Expected output:
'1', 'airport', 'US-GA', 'US'
'2', 'airport', 'US-AK', 'US'
or
[['1','airport','US-GA','US'], ['2','airport', 'US-AK','US']]
Rather than use project you can use values. Steps like project and valueMap return a key:value map, whereas values does not include the keys in its result.
gremlin> g.V().
hasLabel('airport').
limit(2).
local(union(id(),label(),values('region','country')).fold())
==>[1,airport,US,US-GA]
==>[2,airport,US,US-AK]
As an alternative you can just add a select(values) to your current query which I think I prefer as it avoids needing the local and union steps.
gremlin> g.V().
hasLabel('airport').
limit(2).
project('id','label','region','country').
by(id()).
by(label()).
by('region').by('country').
select(values).
fold()
==>[[1,airport,US-GA,US],[2,airport,US-AK,US]]
Related
I've got a json format list with some dictionaries within each list, it looks like the following:
[{"id":13, "name":"Albert", "venue":{"id":123, "town":"Birmingham"}, "month":"February"},
{"id":17, "name":"Alfred", "venue":{"id":456, "town":"London"}, "month":"February"},
{"id":20, "name":"David", "venue":{"id":14, "town":"Southampton"}, "month":"June"},
{"id":17, "name":"Mary", "venue":{"id":56, "town":"London"}, "month":"December"}]
The amount of entries within the list can be up to 100. I plan to present the 'name' for each entry, one result at a time, for those that have London as a town. The rest are of no use to me. I'm a beginner at python so I would appreciate a suggestion in how to go about this efficiently. I initially thought it would be best to remove all entries that don't have London and then I can go through them one by one.
I also wondered if it might be quicker to not filter but to cycle through the entire json and select the names of entries that have the town as London.
You can use filter:
data = [{"id":13, "name":"Albert", "venue":{"id":123, "town":"Birmingham"}, "month":"February"},
{"id":17, "name":"Alfred", "venue":{"id":456, "town":"London"}, "month":"February"},
{"id":20, "name":"David", "venue":{"id":14, "town":"Southampton"}, "month":"June"},
{"id":17, "name":"Mary", "venue":{"id":56, "town":"London"}, "month":"December"}]
london_dicts = filter(lambda d: d['venue']['town'] == 'London', data)
for d in london_dicts:
print(d)
This is as efficient as it can get because:
The loop is written in C (in case of CPython)
filter returns an iterator (in Python 3), which means that the results are loaded to memory one by one as required
One way is to use list comprehension:
>>> data = [{"id":13, "name":"Albert", "venue":{"id":123, "town":"Birmingham"}, "month":"February"},
{"id":17, "name":"Alfred", "venue":{"id":456, "town":"London"}, "month":"February"},
{"id":20, "name":"David", "venue":{"id":14, "town":"Southampton"}, "month":"June"},
{"id":17, "name":"Mary", "venue":{"id":56, "town":"London"}, "month":"December"}]
>>> [d for d in data if d['venue']['town'] == 'London']
[{'id': 17,
'name': 'Alfred',
'venue': {'id': 456, 'town': 'London'},
'month': 'February'},
{'id': 17,
'name': 'Mary',
'venue': {'id': 56, 'town': 'London'},
'month': 'December'}]
I'm trying to iterate through a list and get key and the corresponding values i need like category, current_severity, event, component_name and component_type.
The challenge i am facing is, the content of the list might change, like the key and values will remain the same but the stanzas might be less, zero or more. So i need something more dynamic.
I need to assign the dynamically created variables to prometheus metrics which will also need to be dynamic.
i tried to create dynamic variables like this, it partially works, like it creates duplicates.
the code i tried:-
url = htpp://url
data = json.loads(r.get(url, headers=headers,
verify=False).content.decode('UTF-8'))
var = string.ascii_lowercase
d = {}
k = 0
for value in data:
d[var[k]] = value['software']
k += 1
d[var[k]] = value['current_severity']
k += 1
print(d)
This prints out duplicates like
{'a': 'software', 'b': low}
{'a': 'software', 'b': low, 'c': 'software', 'd': medium}
I want to assign the values to variables and those to be assigned to prometheus metrics. Prometheus metrics are also to be created based on how many ever variables are there.
My desired output:
metric = Metric('prometheus_metric_name', 'prometheus metric name', 'gauge')
metric.add_sample('prometheus_metric_name', value=float(0), labels={'software':(variable of software will go here), 'severity':(variable of current_severity will go here)})
the metric.add_sample line should be created dynamically for how many ever stanzas have been returned. Below is the list returned by data
[{'category': 'software', 'code': 110, 'actual': '["5.1.4"]', 'opened':
'2018-10-16T09:18:12Z', 'component_type': 'update', 'event': 'new update
available', 'current_severity': 'info', 'details': '', 'expected': None,
'id': 10088862, 'component_name': 'Purity//FA'}, {'category': 'software',
'code': 67, 'actual': None, 'opened': '2018-10-18T01:14:45Z',
'component_type': 'host', 'event': 'misconfiguration', 'current_severity':
'critical', 'details': '', 'expected': None, 'id': 10088898,
'component_name': 'pudc-vm-001'}
By stanza i mean - in the above data there are two stanzas. So how many ever there are, i need to get key and value pairs out of it and assign and create metrics as to those.
I'm inserting documents into elasticsearch and trying to sort on a given field that's present in all documents. However, whenever I update a document, indexing seems to break and I do not get a sorted order. I have created an index by doing:
self.conn = ES(server=url)
self.conn.create_index("test.test")
For instance, I would like to sort on a "_ts" field. Given the following dictionaries and code:
def update_or_insert(doc):
doc_type = "string"
index = doc['ns']
doc['_id'] = str(doc['_id'])
doc_id = doc['_id']
self.conn.index(doc, index, doc_type, doc_id)
to_insert = [
{'_id': '4', 'name': 'John', '_ts': 3, 'ns':'test.test'},
{'_id': '5', 'name': 'Paul', '_ts': 2', ns':'test.test'},
{'_id': '6', 'name': 'George', '_ts': 1', ns':'test.test'},
{'_id': '6', 'name': 'Ringo', '_ts': 4, 'ns':'test.test'} ,
]
for x in to_insert:
update_or_insert(x)
result = self.conn.search(q, sort={'_ts:desc'})
for it in result:
print it
I would expect to get an ordering of "Ringo, John, Paul" but instead get an ordering of "John, Paul, Ringo". Any reason why this might be the case? I see there's a bug here:
https://github.com/elasticsearch/elasticsearch/issues/3078
But that seems to affect ES .90.0 and I'm using .90.1.
It should be:
sort={"_ts":"desc"}
I am tyring to match the value of two dicts of two sperate keys by looping over them-with hopefully if i in line_aum['id_class'] == line_investor['id_class'] becoming True, then the next sum dunction will work:
Tho it kicks out a different result
so far I have:
for line_aum in aum_obj:
for line_investor in investor_obj:
if i in line_aum['id_class'] == line_investor['id_class']:
total = (sum,line_investor['amount'], line_aum['value'])
amount = line['id_class']
print(amount,total)
Example data:
{'fund_name': '', 'fund_code': 'LFE', 'aumc': '406.37', 'value': '500', 'ddate': '2013-01-01', 'id_fund': '165', 'currency': 'EUR', 'nav': '24.02', 'shares': '16.918', 'estimate': '0', 'id_class': '4526', 'class_name': 'LTD - CLASS B (EUR)'}
Use itertools.product instead of nested loops if both aum_obj and investor_obj are lists:
from itertools import product
for line_aum, line_investor in product(aum_obj, investor_obj):
if line_aum['id_class'] == line_investor['id_class']:
# `line_aum` and `line_investor` have matching values for the `id_class` keys.
Background:
I have the following example data structure in JSON:
{'sensor' : [
{'assertions_enabled': 'ucr+',
'deassertions_enabled': 'ucr+',
'entity_id': '7.0',
'lower_critical': 'na',
'lower_non_critical': 'na',
'lower_non_recoverable': 'na',
'reading_type': 'analog',
'sensor_id': 'SR5680 TEMP (0x5d)',
'sensor_reading': {'confidence_interval': '0.500',
'units': 'degrees C',
'value': '42'},
'sensor_type': 'Temperature',
'status': 'ok',
'upper_critical': '59.000',
'upper_non_critical': 'na',
'upper_non_recoverable': 'na'}
]}
The sensor list will actually contain many of these dicts containing sensor info.
Problem:
I'm trying to query the list using jsonpath to return me a subset of sensor dicts that have sensor_type=='Temperature' but I'm getting 'False' returned (no match). Here's my jsonpath expression:
results = jsonpath.jsonpath(ipmi_node, "$.sensor[?(#.['sensor_type']=='Temperature')]")
When I remove the filter expression and just use "$.sensor.*" I get a list of all sensors, so I'm sure the problem is in the filter expression.
I've scanned multiple sites/posts for examples and I can't seem to find anything specific to Python (Javascript and PHP seem to be more prominent). Could anyone offer some guidance please?
The following expression does what you need (notice how the attribute is specified):
jsonpath.jsonpath(impi_node, "$.sensor[?(#.sensor_type=='Temperature')]")
I am using jsonpath-ng which seems to be active (as of 23.11.20) and I provide solution based on to Pedro's jsonpath expression:
data = {
'sensor' : [
{'sensor_type': 'Temperature', 'id': '1'},
{'sensor_type': 'Humidity' , 'id': '2'},
{'sensor_type': 'Temperature', 'id': '3'},
{'sensor_type': 'Density' , 'id': '4'}
]}
from jsonpath_ng.ext import parser
for match in parser.parse("$.sensor[?(#.sensor_type=='Temperature')]").find(data):
print(match.value)
Output:
{'sensor_type': 'Temperature', 'id': '1'}
{'sensor_type': 'Temperature', 'id': '3'}
NOTE: besides basic documentation provided on project's homepage I found additional information in tests.