Develop Reference

Get first item from nested list in glom

from glom import glom, T
target = {
"items": [
{
"label": "valuation",
"value": [
"900 USD"
]
},]
}
spec = ('items',[T['value'][0]])
r = glom(target,spec)
print(r)
The above code returns a list, ['900 USD'] but I'd like to just get the content of that list, i.e the first item in the 'value' list. In this case the result should just be 900 USD
Part 2
from glom import glom, T, Check, SKIP
target = {
"items": [
{
"label": "valuation",
"value": [
"900 USD"
]
},
{
"label": "other_info",
"value": [
"700 USD"
]
},]
}
spec = ({
'answer': ('items', [Check('label', equal_to='valuation', default=SKIP)],([T['value'][0]]))
})
r = glom(target,spec)
print(r)
The above code results in {'answer': ['900 USD'] but I need to just return 900 USD.
Tried adding [0] at the end of the brackets but that didn't work.
Playing around with the T type also didn't result in what I'm looking for

I solved it by iterating over my result list and picking out the first element.
The following spec worked
spec = ({
'answer': ('items', [Check('label', equal_to='valuation', default=SKIP)],( [T['value'][0]] ,Iter().first()) )
})
Notice the Iter().first() function call was added.

spec = {
'answer': ('items', Iter().filter(
lambda x: x['label'] is 'valuation'
).map('value.0').first())
}
Note, this is a streaming solution, thus performing better for larger datasets.
A string spec -- here 'value.0', the argument to the map method -- understands indexing into deep lists.
Depending on the specific logic that is desired this might work as well:
spec = {
'answer': ('items', Iter().first(
lambda x: x['label'] is 'valuation'
), 'value.0')
}
Here, we have combined filter logic with the restriction of a single result.

Is there a way to get the particular Values from JSON Array using robot or Python code?

JSON OUTPUT:
${response}= [
{
"Name":"7122Project",
"checkBy":[
{
"keyId":"NA",
"target":"1232"
}
],
"Enabled":false,
"aceess":"123"
},
{
"Name":"7122Project",
"checkBy":[
{
"keyId":"_GU6S3",
"target":"123"
}
],
"aceess":"11222",
"Enabled":false
},
{
"Name":"7122Project",
"checkBy":[
{
"keyId":"-1lLUy",
"target":"e123"
}
],
"aceess":"123"
}
]
Need to get the keyId values from json without using hardcoded index using robot?
I did
${ID}= set variable ${response[0]['checkBy'][0]['keyId']}
But I need to check the length get all keyID values and store the values that dose not contain NA
How can I do check length and use for loop using robot framework?

I suppose you can have more elements in checkBy arrays, like so:
response = [
{
"Name":"7122Project",
"checkBy": [
{
"keyId": "NA",
"target": "1232"
}
],
"Enabled": False,
"aceess": "123"
},
{
"Name": "7122Project",
"checkBy": [
{
"keyId": "_GUO6g6S3",
"target": "123"
}
],
"aceess": "11222",
"Enabled": False
},
{
"Name": "7122Project",
"checkBy": [
{
"keyId": "-1lLlZOUy",
"target": "e123"
},
{
"keyId": "test",
"target": "e123"
}
],
"aceess": "123"
}
]
then you can key all keyIds in Python with this code:
def get_key_ids(response):
checkbys = [x["checkBy"] for x in response]
key_ids = []
for check_by in checkbys:
for key_id in check_by:
key_ids.append(key_id["keyId"])
return key_ids
for the example above, it will return: ['NA', '_GUO6g6S3', '-1lLlZOUy', 'test_NA'].
You want to get both ids with NA and without NA, so perhaps you can change the function a bit:
def get_key_ids(response, predicate):
checkbys = [x["checkBy"] for x in response]
key_ids = []
for check_by in checkbys:
for key_id in check_by:
if predicate(key_id["keyId"]):
key_ids.append(key_id["keyId"])
return key_ids
and use it like so:
get_key_ids(response, lambda id: id == "NA") # ['NA']
get_key_ids(response, lambda id: id != "NA") # ['_GUO6g6S3', '-1lLlZOUy', 'test_NA']
get_key_ids(response, lambda id: "NA" in id) # ['NA', 'test_NA']
get_key_ids(response, lambda id: "NA" not in id) # ['_GUO6g6S3', '-1lLlZOUy']
Now it's just a matter of creating a library and importing it into RF. You can get inspiration in the official documentation.
But I need to check the length get all keyID values and store the values that dose not contain NA
I don't completely understand what you are up to. Do you mean length of keyId strings, like "NA" and its length of 2, or the number of keyIds in the response?
How can I do check length and use for loop using robot framework?
You can use keyword Should Be Equal * from BuiltIn library. Some examples of for loops could be found in the user guide here.
Now you should have all the parts you need to accomplish your task, you can try to put it all together.

modifying json - deleting certain elements within a json structure using python

My json structure is as follows :
"AGENT": {
"pending": [],
"active": null,
"completed": [
**{
"result": {
"job1.AGENT": "SUCCESS",
"job2.AGENT": "SUCCESS"
},
"return_value": {
"job1.AGENT": "",
"job2.AGENT": ""
},
"visible": true,
"global": true,
"locale": [
"en_US"
],
"complete_time": "2018-01-24T17:44:33.484Z",
"persist": true,
"type": "script",
"script": "<script_name>.py",
"preset_status": "CONFIGURING",
"parameters": {},
"submit_time": "2018-01-24T17:44:26.747Z"
}**,
{
"result": {
..
},
"return_value": {
..
},
"visible": true,
"global": true,
"locale": [
"en_US"
],
"complete_time": "2018-04-2T17:44:40.049Z",
"submit_time": "2018-04-2T17:44:26.817Z"
}
I need to delete the entire result block based on complete_time, like delete the result block before 2018-04-03
How can i acheive this in python ?
I have tried the following so far :
json_data = json.dumps(data)
item_dict = json.loads(data)
print item_dict["AGENT"]["completed"][0]["complete_time"]
This prints the complete time. However my problem is "AGENT" is not a constant string. The string can vary. Also I will need to figure out the logic to remove the entire json block based on complete_time

Ok, I assume that you were able to correctly load the json into a Python dictionnary, let call it item_dict, but the key may vary.
What you need now it to walk inside that Python object, and decode the complete_time field. Unfortunately, Python strptime does not know about the Z time zone, so we will have to skip that last character.
Additionaly, you should never modify a collection object while iterating it, so the bullet proof way is to store indices to remove and later remove them. Code could be:
datelimit = datetime.datetime(2018, 4, 1) # limit date for completed_time
to_remove = []
dateformat = '%Y-%m-%dT%H:%M:%S.%f'
for k, v in item_dict.items(): # enumerate top_level objects
for i, block in enumerate(v['completed']): # enumerate inner blocks
complete_time = datetime.datetime.strptime( # skip last char from complete_time
block["complete_time"][:-1], dateformat)
# print(k, i, complete_time) # uncomment for tests
if complete_time < datelimit: # too old
to_remove.append((k, i)) # store the index for later processing
for k, i in reversed(to_remove): # start from the end to keep consistent indices
del item_dict[k]["completed"][i] # actual deletion

Parsing json with python3

Newbie python programmer here, I have the following json response:
[
{
"type": "Incursion",
"state": "mobilizing",
"influence": 1,
"has_boss": true,
"faction_id": 500019,
"constellation_id": 20000739,
"staging_solar_system_id": 30005054,
"infested_solar_systems": [
30005050,
30005051,
30005052,
30005053,
30005054,
30005055
]
},
{
"type": "Incursion",
"state": "established",
"influence": 0,
"has_boss": false,
"faction_id": 500019,
"constellation_id": 20000035,
"staging_solar_system_id": 30000248,
"infested_solar_systems": [
30000244,
30000245,
30000246,
30000247,
30000248,
30000249,
30000250,
30000251,
30000252,
30000253
]
},
{
"type": "Incursion",
"state": "mobilizing",
"influence": 0,
"has_boss": false,
"faction_id": 500019,
"constellation_id": 20000161,
"staging_solar_system_id": 30001101,
"infested_solar_systems": [
30001097,
30001098,
30001099,
30001100,
30001101,
30001102
]
},
{
"type": "Incursion",
"state": "established",
"influence": 0,
"has_boss": false,
"faction_id": 500019,
"constellation_id": 20000647,
"staging_solar_system_id": 30004434,
"infested_solar_systems": [
30004425,
30004426,
30004427,
30004428,
30004429,
30004430,
30004431,
30004432,
30004433,
30004434,
30004435
]
},
{
"type": "Incursion",
"state": "established",
"influence": 0.061500001698732376,
"has_boss": false,
"faction_id": 500019,
"constellation_id": 20000570,
"staging_solar_system_id": 30003910,
"infested_solar_systems": [
30003904,
30003906,
30003908,
30003909,
30003910,
30003903
]
}
]
The original code was written to parse an XML reponse.
This is the code in question:
incursion_constellations = []
if (online):
inc = urllib2.urlopen('https://esi.tech.ccp.is/latest/incursions/')
else:
inc = file(r'incursions.json', 'r')
jinc = json.load(inc)
for j in jinc['items']:
incursion_constellations.append(str(j['constellation']['id_str']))
for s in all_stations:
cur.execute("SELECT constellationID FROM mapSolarSystems WHERE solarSystemID = " + str(s['ssid']))
res = cur.fetchone()
cid = str(res[0])
s['incursion'] = cid in incursion_constellations
The area I have having a hard time understanding is this: for j in jinc['items']:
I am getting this error:
Traceback (most recent call last):
File "./stations.py", line 201, in <module>
for j in jinc['items']:
TypeError: list indices must be integers, not str
Can anyone help me understand how to convert this into being able to parse the json response and retrieve the constellation_id and append it to a list?
Thanks in advance.

Change your original loop to:
for j in jinc:
incursion_constellations.append(str(j['constellation_id']))
But you need to be sure that constellation_id in json is the same id that was under ['constellation']['id_str'] previously

Seeing the [ and ] at the beginning and the end of the response, it seems like this json response is list, not a dict, just as your error is suggesting.
If it is a list, you should be using integer as index, instead of str, like you'd do in dict. Hence, your code should be something like
jinc[0]['constellation_id']
(I don't see where the ['constellation']['id_str'] part comes from)
whatever goes inside the [ and ] is in a list, and should be using an integer index. the ones in { and } are in dict, and should use str index.
to loop through it, just use range and len.
a similar question has been answered here.

Adding key to values in json using Python

This is the structure of my JSON:
"docs": [
{
"key": [
null,
null,
"some_name",
"12345567",
"test_name"
],
"value": {
"lat": "29.538208354844658",
"long": "71.98762580927113"
}
},
I want to add the keys to the key list. This is what I want the output to look like:
"docs": [
{
"key": [
"key1":null,
"key2":null,
"key3":"some_name",
"key4":"12345567",
"key5":"test_name"
],
"value": {
"lat": "29.538208354844658",
"long": "71.98762580927113"
}
},
What's a good way to do it. I tried this but doesn't work:
for item in data['docs']:
item['test'] = data['docs'][3]['key'][0]
UPDATE 1
Based on the answer below, I have tweaked the code to this:
for number, item in enumerate(data['docs']):
# pprint (item)
# print item['key'][4]
newdict["key1"] = item['key'][0]
newdict["yek1"] = item['key'][1]
newdict["key2"] = item['key'][2]
newdict["yek2"] = item['key'][3]
newdict["key3"] = item['key'][4]
newdict["latitude"] = item['value']['lat']
newdict["longitude"] = item['value']['long']
This creates the JSON I am looking for (and I can eliminate the list I had previously). How does one make this JSON persist outside the for loop? Outside the loop, only the last value from the dictionary is added otherwise.

In your first block, key is a list, but in your second block it's a dict. You need to completely replace the key item.
newdict = {}
for number,item in enumerate(data['docs']['key']):
newdict['key%d' % (number+1)] = item
data['docs']['key'] = newdict