Related
In TestRail, I have created several testruns. When I execute:
test_runs = client.send_get('get_runs/1')
pprint(test_runs)
The following results are returned:
{'_links': {'next': None, 'prev': None},
'limit': 250,
'offset': 0,
'runs': [{'assignedto_id': None,
'blocked_count': 0,
'completed_on': None,
'config': None,
'config_ids': [],
'created_by': 1,
'created_on': 1651790693,
'custom_status1_count': 0,
'custom_status2_count': 0,
'custom_status3_count': 0,
'custom_status4_count': 0,
'custom_status5_count': 0,
'custom_status6_count': 0,
'custom_status7_count': 0,
'description': None,
'failed_count': 1,
'id': 13,
'include_all': False,
'is_completed': False,
'milestone_id': None,
'name': '2022-05-05-testrun',
'passed_count': 2,
'plan_id': None,
'project_id': 1,
'refs': None,
'retest_count': 0,
'suite_id': 1,
'untested_count': 0,
'updated_on': 1651790693,
'url': 'https://xxxxxxxxxx.testrail.io/index.php?/runs/view/13'},
{'assignedto_id': None,
'blocked_count': 0,
'completed_on': 1650989972,
'config': None,
'config_ids': [],
'created_by': 5,
'created_on': 1650966329,
'custom_status1_count': 0,
'custom_status2_count': 0,
'custom_status3_count': 0,
'custom_status4_count': 0,
'custom_status5_count': 0,
'custom_status6_count': 0,
'custom_status7_count': 0,
'description': None,
'failed_count': 0,
'id': 9,
'include_all': False,
'is_completed': True,
'milestone_id': None,
'name': 'This is a new test run',
'passed_count': 0,
'plan_id': None,
'project_id': 1,
'refs': None,
'retest_count': 0,
'suite_id': 1,
'untested_count': 3,
'updated_on': 1650966329,
'url': 'https://xxxxxxxxxx.testrail.io/index.php?/runs/view/9'}],
'size': 2}
In my code, I am trying to scan through all of the resulting testruns, locate the testrun I'm interested in by matching the testrun name, and then have the ID for the testrun returned.
from pprint import pprint
from testrail import *
class connecting():
def connectPostRun(self):
client = APIClient('https://xxxxxxxxxx.testrail.io')
client.user = 'abc#abc.com'
client.password = 'abc123'
test_run_name = '2022-05-05-testrun'
test_runs = client.send_get('get_runs/1')
pprint(test_runs)
for test_run in test_runs:
if test_run['name'] == test_run_name:
run_id = test_run['id']
break
return run_id
pprint(run_id)
c=connecting()
c.connectPostRun()
Executing the code as is now results in the following error:
if test_run['name'] == test_run_name:
TypeError: string indices must be integers
You're looping over the wrong part of the datastructure that the function returned. The loop for test_run in test_runs: only iterates over the keys of the top-level dictionary ("_links", "limit", etc.).
You want to be looping over test_runs['runs'], which will give you dictionaries with the "name" keys you're matching against. Try making your loop look like this:
for test_run in test_runs['runs']:
if test_run['name'] == test_run_name:
run_id = test_run['id']
break
I'd note that there's a potential problem in this code, that if you never find a matching run, the run_id variable will never be assigned to, so the return statement at the end of the function will raise an exception. If you think that could ever happen, you should probably either set a default value, or perhaps raise your own exception (with a more clear message) if you get into that situation.
I'm trying to make a function that would take nested array (dict/list in any order) and a key name as arguments and return all values of that key in a list.
my_key = "Items"
my_dict = [{'z': 0, 'x': 0, 'y': 0, 'Items': [{'Slot': 1, 'id': 'minecraft:rail', 'Count': 1}, {'Slot': 2, 'id': 'minecraft:white_shulker_box', 'tag': {'BlockEntityTag': {'id': 'minecraft:shulker_box', 'Items': [{'Slot': 0, 'Count': 1, 'tag': {'Items': [{'id': 'minecraft:amethyst_shard', 'Count': 1}]}, 'id': 'minecraft:bundle'}]}}, 'Count': 1}]}]
def recursive_lookup(data, key):
if isinstance(data, list):
for i in data:
recursive_lookup(i, key)
elif isinstance(data, dict):
for i, v in data.items():
if i == key:
print(f'{v = }')
if isinstance(v, list) or isinstance(v, dict): recursive_lookup(v, key)
print(recursive_lookup(my_dict, my_key))
Currently it prints out found items at print(f'{v = }'). How can I store those in a list and pass as a function return?
You can use .extend() to concatenate the result of recursive calls to a list.
def recursive_lookup(data, key):
values = []
if isinstance(data, list):
for i in data:
values.extend(recursive_lookup(i, key))
elif isinstance(data, dict):
for i, v in data.items():
if i == key:
values.append(v)
if isinstance(v, list) or isinstance(v, dict):
values.extend(recursive_lookup(v, key))
return values
You can what you want without any explicit recursion at all by making use of the json module in the standard library (assuming your data can be serialized into that format). This is because the JSON decoder supports an object_hook argument which is a function it will call everytime it encounters a dictionary.
The basic idea is to specify a function via this argument that merely "watches" what is being decoded and checks it for the sought-after key.
Here's what I mean:
import json
my_key = "Items"
my_dict = [{'z': 0, 'x': 0, 'y': 0, 'Items': [{'Slot': 1, 'id': 'minecraft:rail', 'Count': 1}, {'Slot': 2, 'id': 'minecraft:white_shulker_box', 'tag': {'BlockEntityTag': {'id': 'minecraft:shulker_box', 'Items': [{'Slot': 0, 'Count': 1, 'tag': {'Items': [{'id': 'minecraft:amethyst_shard', 'Count': 1}]}, 'id': 'minecraft:bundle'}]}}, 'Count': 1}]}]
def lookup(data, key):
results = []
def decode_dict(a_dict):
try:
results.append(a_dict[key])
except KeyError:
pass
return a_dict
json_repr = json.dumps(data) # Convert to JSON format.
json.loads(json_repr, object_hook=decode_dict) # Return value ignored.
return results
from pprint import pprint
pprint(lookup(my_dict, my_key), sort_dicts=False)
Pretty-printed result list:
[[{'id': 'minecraft:amethyst_shard', 'Count': 1}],
[{'Slot': 0,
'Count': 1,
'tag': {'Items': [{'id': 'minecraft:amethyst_shard', 'Count': 1}]},
'id': 'minecraft:bundle'}],
[{'Slot': 1, 'id': 'minecraft:rail', 'Count': 1},
{'Slot': 2,
'id': 'minecraft:white_shulker_box',
'tag': {'BlockEntityTag': {'id': 'minecraft:shulker_box',
'Items': [{'Slot': 0,
'Count': 1,
'tag': {'Items': [{'id': 'minecraft:amethyst_shard',
'Count': 1}]},
'id': 'minecraft:bundle'}]}},
'Count': 1}]]
You can keep a running list:
def recursive_lookup(data, key):
lst = []
if isinstance(data, list):
for i in data:
lst.append(recursive_lookup(i, key))
elif isinstance(data, dict):
for i, v in data.items():
if i == key:
lst.append([v])
if isinstance(v, list) or isinstance(v, dict): lst.append(recursive_lookup(v, key))
return lst
print(recursive_lookup(data, 'Items'))
I have the following piece of code in python:
def pushHashtagPosts(hashtagPosts):
from bson.json_util import loads
myclient = pymongo.MongoClient(mongoUri)
mydb = myclient["myDB"]
mycol = mydb["hashtags"]
data = loads(hashtagPosts)
posts = mycol.insert_many(data)
Whereas, the content of hashtagPosts looks something like this:
hashtagPosts = [{'hashtag': '###!', 'PostHashHex': '13fc9904028fb62490a3b5dc2111689376e52a06dc636c3322cfa16e33a41398', 'post': {'_id': {'$oid': '608f8eb73718c7977f9c0a43'}, 'PostHashHex': '13fc9904028fb62490a3b5dc2111689376e52a06dc636c3322cfa16e33a41398', 'PosterPublicKeyBase58Check': 'BC1YLhKJZZcPB2WbZSSekFF19UshsmmPoEjtEqrYakzusLmL25xxAJv', 'ParentStakeID': '', 'Body': 'Need hashtags ####! Or else it’s a bit difficult to create personal brand and niche on this platform. \n\nDevs are u listening?', 'ImageURLs': [], 'RecloutedPostEntryResponse': None, 'CreatorBasisPoints': 0, 'StakeMultipleBasisPoints': 12500, 'TimestampNanos': 1.6177643730879583e+18, 'IsHidden': False, 'ConfirmationBlockHeight': 13248, 'InMempool': False, 'StakeEntry': {'TotalPostStake': 0, 'StakeList': []}, 'StakeEntryStats': {'TotalStakeNanos': 0, 'TotalStakeOwedNanos': 0, 'TotalCreatorEarningsNanos': 0, 'TotalFeesBurnedNanos': 0, 'TotalPostStakeNanos': 0}, 'ProfileEntryResponse': None, 'Comments': None, 'LikeCount': 5, 'PostEntryReaderState': None, 'InGlobalFeed': False, 'IsPinned': False, 'PostExtraData': {}, 'CommentCount': 2, 'RecloutCount': 0, 'ParentPosts': None, 'PublicKeyBase58Check': 'BC1YLhKJZZcPB2WbZSSekFF19UshsmmPoEjtEqrYakzusLmL25xxAJv', 'Username': ''}},
{'hashtag': 'investementstrategy', 'PostHashHex': '92f2d08ac8f2b47fe5868b748c7f472e13ad12c284bb0e327cf317b4c2514f83', 'post': {'_id': {'$oid': '608f8eb73718c7977f9c0a3f'}, 'PostHashHex': '92f2d08ac8f2b47fe5868b748c7f472e13ad12c284bb0e327cf317b4c2514f83', 'PosterPublicKeyBase58Check': 'BC1YLhKJZZcPB2WbZSSekFF19UshsmmPoEjtEqrYakzusLmL25xxAJv', 'ParentStakeID': '', 'Body': 'Don’t say that you are going to buy ur own coin to have a steady growth of ur coin \U0001fa99. That doesn’t show the strength of ur investment nor the coin.📉📈 Strength lies in others believing in ur talent, creativity and passion enough to invest in U. 🚀🚀🚀\n#InvestementStrategy', 'ImageURLs': [], 'RecloutedPostEntryResponse': None, 'CreatorBasisPoints': 0, 'StakeMultipleBasisPoints': 12500, 'TimestampNanos': 1.6178065064906166e+18, 'IsHidden': False, 'ConfirmationBlockHeight': 13397, 'InMempool': False, 'StakeEntry': {'TotalPostStake': 0, 'StakeList': []}, 'StakeEntryStats': {'TotalStakeNanos': 0, 'TotalStakeOwedNanos': 0, 'TotalCreatorEarningsNanos': 0, 'TotalFeesBurnedNanos': 0, 'TotalPostStakeNanos': 0}, 'ProfileEntryResponse': None, 'Comments': None, 'LikeCount': 2, 'PostEntryReaderState': None, 'InGlobalFeed': False, 'IsPinned': False, 'PostExtraData': {}, 'CommentCount': 1, 'RecloutCount': 0, 'ParentPosts': None, 'PublicKeyBase58Check': 'BC1YLhKJZZcPB2WbZSSekFF19UshsmmPoEjtEqrYakzusLmL25xxAJv', 'Username': ''}},
{'hashtag': 'productivity', 'PostHashHex': 'c8fabd96f5d624d06ec8d23e90de19cf07ad4b6696dac321fda815c3000fbf1b', 'post': {'_id': {'$oid': '608f8eb73718c7977f9c0a3d'}, 'PostHashHex': 'c8fabd96f5d624d06ec8d23e90de19cf07ad4b6696dac321fda815c3000fbf1b', 'PosterPublicKeyBase58Check': 'BC1YLhKJZZcPB2WbZSSekFF19UshsmmPoEjtEqrYakzusLmL25xxAJv', 'ParentStakeID': '', 'Body': 'What is the most productive thing u have done in last 24 hours apart from Bitclout???\n\n\U0001f9d0😏🙌🏼 #productivity', 'ImageURLs': [], 'RecloutedPostEntryResponse': None, 'CreatorBasisPoints': 0, 'StakeMultipleBasisPoints': 12500, 'TimestampNanos': 1.6178362054980055e+18, 'IsHidden': False, 'ConfirmationBlockHeight': 13487, 'InMempool': False, 'StakeEntry': {'TotalPostStake': 0, 'StakeList': []}, 'StakeEntryStats': {'TotalStakeNanos': 0, 'TotalStakeOwedNanos': 0, 'TotalCreatorEarningsNanos': 0, 'TotalFeesBurnedNanos': 0, 'TotalPostStakeNanos': 0}, 'ProfileEntryResponse': None, 'Comments': None, 'LikeCount': 30, 'PostEntryReaderState': None, 'InGlobalFeed': True, 'IsPinned': False, 'PostExtraData': {}, 'CommentCount': 59, 'RecloutCount': 0, 'ParentPosts': None, 'PublicKeyBase58Check': 'BC1YLhKJZZcPB2WbZSSekFF19UshsmmPoEjtEqrYakzusLmL25xxAJv', 'Username': ''}}]
When I try to insert this data as insert_many() into mongodb I get the following error:
File "test.py", line X, in pushHashtagPosts
data = loads(hashtagPosts) TypeError: the JSON object must be str, bytes or bytearray, not 'list'
However, I have inserted the line 'data = loads(hashtagPosts)' based on the solution at bson.errors.InvalidDocument: key '$oid' must not start with '$' trying to insert document with pymongo because without the 'data = loads(hashtagPosts)' I was getting the following error:
bson.errors.InvalidDocument: key '$oid' must not start with '$'
How to resolve this and successfully insert many documents in the collection?
Your issue is that hashtagPosts is a list but loads expects to work on a string.
So working backwards, the question becomes how did you construct hashtagPosts in the first place? As it contains $oid values, it looks like an output from dumps; but an output from dumps is a string. not a list. So how did it become a list?
If you are creating it manually, then just set it using ObjectId, e.g.
from bson import ObjectId
item = {'_id': ObjectId('608f8eb73718c7977f9c0a43')}
and then you won't need to use loads.
Currently I'm trying to get 'stringency' data from a json file which contains dates and countries. Here's an excerpt of what the json output looks like:
import pandas as pd
import json
from bs4 import BeautifulSoup
# load file
with open("Stringency April 8.txt") as file:
stringency_data = json.load(file)
stringency_data["data"]
#this gives the output:
{'2020-01-02': {'ABW': {'confirmed': None,`
'country_code': 'ABW',
'date_value': '2020-01-02',
'deaths': None,
'stringency': 0,
'stringency_actual': 0},
'AFG': {'confirmed': 0,
'country_code': 'AFG',
'date_value': '2020-01-02',
'deaths': 0,
'stringency': 0,
'stringency_actual': 0},
'AGO': {'confirmed': None,
'country_code': 'AGO',
'date_value': '2020-01-02',
'deaths': None,
'stringency': 0,
'stringency_actual': 0},
'AUS': {'confirmed': 0,
'country_code': 'AUS',
'date_value': '2020-01-02',
'deaths': 0,
'stringency': 7.14,
'stringency_actual': 7.14},
'AUT': {'confirmed': 0,
'country_code': 'AUT',
'date_value': '2020-01-02',
'deaths': 0,
'stringency': 0,
'stringency_actual': 0},.........
Here's my code so far (I've shortened it a bit for the sake of this post):
# create empty list for dates
date_index = []
[date_index.append(date) for date in stringency_data["data"]]
#creates empty lists for countries
Australia = []
Austria = []
...
US = []
# put these lists into a list
countries_lists = [Australia, Austria,...US]
# put country codes into a list
country_codes = ["AUS", "AUT",..."USA"]
# loop through countries
i = 0
for country, code in zip(countries_lists, country_codes):
while i<=len(date_index):
country.append(stringency_data["data"][date_index[i]][code]["stringency_actual"])
i+=1
When I print the list "Australia" I get all the values I want. But ever country from Austria onwards is still an empty list.
I get the output - KeyError: "AUS". This indicates that the code retrieved the whole time series, but only for the first country (Australia). How can I loop this for each country code?
Here's what I see about the data you've described/shown:
file data is a dictionary; single known/desired key is "data", value is a dictionary.
--> keys are all date_strings. Each value is a dictionary.
-----> keys are all country_codes. Each value is a dictionary.
--------> a key "stringency_actual" is present, and its value is desired.
So a straightforward plan for getting this data out could look like this:
1. grab file['data']
2. iterate all keys and values in this dictionary. (Actually, you may only care about the values.)
3. iterate all keys and values in this dictionary. Keys are country_codes, which tell you to which list you want to append the stringency_actual value you're about to get.
4. grab this dictionary['stringency_actual'] and append it to the list corresponding to the correct country.
4b. translate the country_code to the country_name, since that's apparently how you would like to store this data for now.
I changed the data retrieval because the data is all dictionaries so it's self-describing by its keys. Doing it this way can help prevent the KeyError I see mentioned in the original question and a comment. (Without the complete input file or the line number of the KeyError, I think none of us is 100% certain which value in the input is causing that KeyError.)
Potential answer:
import json
# Input sample data; would actually be retrieved from file.
stringency_data = json.loads("""
{"data": {"2020-01-02": {"ABW": {"confirmed": null,
"country_code": "ABW",
"date_value": "2020-01-02",
"deaths": null,
"stringency": 0,
"stringency_actual": 0},
"AFG": {"confirmed": 0,
"country_code": "AFG",
"date_value": "2020-01-02",
"deaths": 0,
"stringency": 0,
"stringency_actual": 0},
"AGO": {"confirmed": null,
"country_code": "AGO",
"date_value": "2020-01-02",
"deaths": null,
"stringency": 0,
"stringency_actual": 0},
"AUS": {"confirmed": 0,
"country_code": "AUS",
"date_value": "2020-01-02",
"deaths": 0,
"stringency": 7.14,
"stringency_actual": 7.14},
"AUT": {"confirmed": 0,
"country_code": "AUT",
"date_value": "2020-01-02",
"deaths": 0,
"stringency": 0,
"stringency_actual": 0}}}
}""")
country_name_by_code = {
'ABW': 'Aruba',
'AFG': 'Afghanistan',
'AUS': 'Australia',
'AUT': 'Austria',
# ...
'USA': 'United States'
}
# Output data we want to create
actual_stringencies_by_country_name = {}
# Helper method to store data we're interested in
def append_country_stringency(country_code, actual_stringency_value):
if country_code not in country_name_by_code:
print(f'Unknown country_code value "{country_code}"; ignoring.')
return
country_name = country_name_by_code[country_code]
if country_name not in actual_stringencies_by_country_name:
actual_stringencies_by_country_name[country_name] = []
actual_stringencies_by_country_name[country_name].append(actual_stringency_value)
# Walk our input data and store the parts we're looking for
for date_string, data_this_date in stringency_data['data'].items():
for country_code, country_data in data_this_date.items():
append_country_stringency(country_code, country_data['stringency_actual'])
print(actual_stringencies_by_country_name)
My output:
C:\some_dir>python test.py
Unknown country_code value "AGO"; ignoring.
{'Aruba': [0], 'Afghanistan': [0], 'Australia': [7.14], 'Austria': [0]}
I am trying to call all the values from a dictionary that is within a list if the value of the key is within a separate list.
For example, I have this listed dictionary:
status = [{'name': 'Carrousel', 'wait': 0, 'isOpen': True, 'single_rider': None},
{'name': 'Balloon Flite', 'wait': 0, 'isOpen': True, 'single_rider': None},
{'name': 'Skyrush', 'wait': 0, 'isOpen': False, 'single_rider': None},
{'name': 'SooperDooperLooper',
'wait': 5,
'isOpen': True,
'single_rider': None},
{'name': 'Fahrenheit', 'wait': 20, 'isOpen': True, 'single_rider': None},
{'name': 'Dummy', 'wait': 0, 'isOpen': False, 'single_rider': None}]
I also have this list:
route = ['Skyrush', 'SooperDooperLooper', 'Carrousel', 'Fahrenheit']
Basically, I wanted to print out the values of 'wait' in status for those names in route.
I know how to call the values of the row if I know the index but I'm having trouble trying to call only rows that contains the specific value of 'name'.
My expected result is something like:
0
5
0
20
Those are basically the 'wait' times of each respective rides in consecutive order within route.
Thank you! Any help would be greatly appreciated. I looked through other postings but couldn't find anything that is similar to my question.
Adding too #Chris's answer, to get the expected order:
[d['wait'] for d in sorted(status, key=lambda x: ''.join(route).find(x['name'])) if d['name'] in route]
Output:
[0, 5, 0, 20]
Use list comprehension:
[d['wait'] for d in status if d['name'] in route]
Output:
[0, 0, 5, 20]
As indicated in #Chris's answer, a list comprehension is the way to go.
However, if you want the order to be the order matching your route, this would be a solution:
[next(s for s in status if s['name'] == name)['wait'] for name in route]
That actually gets you [0, 5, 0, 20] instead of [0, 0, 5, 20].
This also directly answers your actual question: how to access a list item in a list of dictionaries, by referencing a dictionary key.
next(item for item in some_list_of_dicts if s['key'] == 'some value')
Gets you the first item matching the condition.
Another approach: Change your input so you can access things more easily
name_indexed_status = dict(map(lambda x: (x.pop("name"), x), status))
print (str(name_indexed_status))
for r in route:
print(name_indexed_status[r]["wait"])
{
'Carrousel': {'wait': 0, 'isOpen': True, 'single_rider': None},
'Balloon Flite': {'wait': 0, 'isOpen': True, 'single_rider': None},
'Skyrush': {'wait': 0, 'isOpen': False, 'single_rider': None},
'SooperDooperLooper': {'wait': 5, 'isOpen': True, 'single_rider': None},
'Fahrenheit': {'wait': 20, 'isOpen': True, 'single_rider': None},
'Dummy': {'wait': 0, 'isOpen': False, 'single_rider': None}
}
0
5
0
20