I am trying to build dictionary from string can someone help me to build the efficient way.
I have api which takes string with comma separate value and returns if value if API has matching values
example:
# my paramaters to my API car, bike, moter-cycle, airplane, boat
# www.api.com?words=car,bike,moter-cycle,airplane,boat
field_json = json.loads(response.text)
data = field_json['response']
list1=""
for i in data['docs']:
list[i['keyword_s']]= i['url_s']
list1 = str(list)
print(list)
return list1
from above I just get
{'bike':'http://bikes.com/bikes',
'boat':'http://boat.com/boat'}
if API find bike and boat as matching I would need my dictonary as
{'car':'none',
'bike':'http://bikes.com/bikes',
'moter-cycle': 'none',
'airplane':'none',
'boat':'http://boat.com/boat'
}
You don't need to involve strings, just fill in hole values in your returned api response
dict_a = {'bike':'http://bikes.com/bikes', 'boat':'http://boat.com/boat'}
keywords = ['car','bike','motorcycle', 'airplane', 'boat']
dict_b = {k: None for k in keywords}
dict_b.update(dict_a)
print(dict_b)
Output
{'car': None, 'bike': 'http://bikes.com/bikes', 'motorcycle': None, 'airplane': None, 'boat': 'http://boat.com/boat'}
Related
I have this list of dictionaries I want to convert to one dictionary
vpcs = [{'VPCRegion': 'us-east-1', 'VPCId': '12ededd4'},
{'VPCRegion': 'us-east-1', 'VPCId': '9847'},
{'VPCRegion': 'us-west-2', 'VPCId': '99485003'}]
I want to convert it to
{'us-east-1': '12ededd4', 'us-east-1': '9847', 'us-west-2': '99485003'}
I used this function
def convert_dict(tags):
return {tag['VPCRegion']:tag['VPCId'] for tag in tags}
but get this output it doesn't convert the first dictionary in the list
{'us-east-1': '9847', 'us-west-2': '99485003'}
Perhaps a list of dictionary may fit your need - see code below:
[{'us-east-1': '12ededd4'}, {'us-east-1': '9847'}, {'us-west-2': '99485003'}]
To elaborate on what other commented about dictionary key has to be unique, you can see that in the commented line which zip up the list_dict would result error if the 'vpcs' has 2 duplicate 'VPCRegion': 'us-east-1' and successfully create new dict if you take out one of the 'VPCRegion': 'us-east-1'.
vpcs = [{'VPCRegion': 'us-east-1', 'VPCId': '12ededd4'},
{'VPCRegion': 'us-east-1', 'VPCId': '9847'},
{'VPCRegion': 'us-west-2', 'VPCId': '99485003'}]
def changekey(listofdict):
new_dict = {}
new_list = []
for member in listofdict:
new_key = member['VPCRegion']
new_val = member['VPCId']
new_dict.update({new_key:new_val})
new_list.append({new_key:new_val})
return new_dict, new_list
dict1,list_dict=changekey(vpcs)
print(dict1)
print(list_dict)
#dict4=dict(zip(*[iter(list_dict)]*2))
#print(dict4)
Since your output must group several values under the same name, your output will be a dict of lists, not a dict of strings.
One way to quickly do it:
import collections
def group_by_region(vpcs):
result = collections.defaultdict(list)
for vpc in vpcs:
result[vpc['VPCRegion']].append(vpc['VPCId'])
return result
The result of group_by_region(vpcs) will be {'us-east-1': ['12ededd4', '9847'], 'us-west-2': ['99485003']}).
As an entertainment, here's a cryptic but efficient way to get this in one expression:
import itertools
{key: [rec['VPCId'] for rec in group]
for (key, group) in itertools.groupby(vpcs, lambda vpc: vpc['VPCRegion'])}
I have a huge list of dictionaries appended to the list from JSON in the manner below. I would like to access the "raw" from each dictionary and store it entire in a list of dictionary or a one huge dictionary. The final goal is to access the keys in the raw and convert them to dataframe columns using pandas.
results = [{
'FirstSentences': None,
'PrintableUri': '',
'hasHtmlVersion': False,
'hasMobileHtmlVersion': False,
'isRecommendation': False,
'isTopResult': False,
'parentResult': None,
'percentScore': 100.0,
'printableUriHighlights': [],
'rankingInfo': None,
'rating': 3.0,
'raw': {'distance': 1892760.0,
'distancekm': 1892.76,
'distancemi': 1176.11,
'objecttype': 'Account',
'objecttypename': 'Account',
'plocepp': 'False',
'plochpp': 'False',
'plocsdp': 'False'}}]
The code and the error I'm getting is as below:
response = [x['raw'] for x in results]
File "<ipython-input-90-64993f9dcd67>", line 1, in <module>
response = [x['raw'] for x in results]
TypeError: list indices must be integers, not str
I have searched a lot of answers here but couldn't find the solution to my problem. Thanks a lot for the help in advance.
The key is to iterate through the list and for each element in the list, index the dictionary key 'raw'.
One simple way is to iterate over the list and and access the key 'raw' in the dictionary.
Simple for loop method:
response = []
for d in results:
response.append(d['raw'])
List comprehension method
response = [d['raw'] for d in results]
I'm trying to query a MongoDB database and throw the two sets of results ('_id' and 'Team') into two separate lists.
import pymongo
client = pymongo.MongoClient('localhost:27017')
db = client['db_name']
query = {'Team': {'$exists': 1}}
projection = {'_id': 1, 'Team': 1}
data = db['collection_name'].find(query, projection) # line 9
id_list = [value for dict in data for key, value in dict.iteritems() if key == '_id']
teams_list = [value for dict in data for key, value in dict.iteritems() if key == 'Team']
print id_list
print teams_list
client.close()
For the code above, the 'id_list' is as expected but 'teams_list' is empty. When I put 'teams_list' before 'id_list' I get the expected 'teams_list' output and 'id_list' is empty. And when I repeat the data call (line 9) in between the two list comprehensions I get the expected output for both lists.
Any idea why this is happening?
You need to define your data as:
data = list(db['collection_name'].find(query, projection))
As find() returns the generator. Once you iterate the values, those are lost. You need to store them as list. Here list() does that i.e. stores the items returns by generator as list.
Instead of iterating the list twice, better way will be two do it single loop as:
id_list, teams_list = [], []
# v `dict` is in-built data type, you should not be using it as variable
for d in data:
for key, value in d.iteritems():
if key == '_id':
id_list.append(value)
elif key == 'Team':
teams_list.append(value)
Refer Generator wiki for more information related to generators
As already mentioned the culprit here is the find() method which returns a Cursor object which is consumed when you iterate it the first time.
But you are using the wrong method for the job. You need to use the .aggregate() method.
query = {'Team': {'$exists': 1}}
cursor = db['collection_name'].aggregate([
{'$match': query }
{ '$group': {
'_id': None,
'id_list': {'$push': '$_id'},
'teams_list': {'$push': '$Team'}
}}
])
The .aggregate() method like his partner in crime .find() returns a CommandCursor over the result set which is a generator like object.
Because we are grouping by None, iterating the cursor will yield a single document which means that you can safely do:
print list(cursor)[0] # return a dictionary
or
result = list(cursor)[0]
print result['id_list']
print result['teams_list']
Hoping someone can help me out. I've spent the past couple hours trying to solve this, and fair warning, I'm still fairly new to python.
This is a repost of a question I recently deleted. I've misinterpreted my code in the last example.The correct example is:
I have a dictionary, with a list that looks similar to:
dic = [
{
'name': 'john',
'items': ['pants_1', 'shirt_2','socks_3']
},
{
'name': 'bob',
items: ['jacket_1', 'hat_1']
}
]
I'm using .append for both 'name', and 'items', which adds the dic values into two new lists:
for x in dic:
dic_name.append(dic['name'])
dic_items.append(dic['items'])
I need to split the item value using '_' as the delimiter, so I've also split the values by doing:
name, items = [i if i is None else i.split('_')[0] for i in dic_name],
[if i is None else i.split('_')[0] for i in chain(*dic_items)])
None is used in case there is no value. This provides me with a new list for name, items, with the delimiter used. Disregard the fact that I used '_' split for names in this example.
When I use this, the index for name, and item no longer match. Do i need to create the listed items in an array to match the name index, and if so, how?
Ideally, I want name[0] (which is john), to also match items[0] (as an array of the items in the list, so pants, shirt, socks). This way when I refer to index 0 for name, it also grabs all the values for items as index 0. The same thing regarding the index used for bob [1], which should match his items with the same index.
#avinash-raj, thanks for your patience, as I've had to update my question to reflect more closely to the code I'm working with.
I'm reading a little bit between the lines but are you trying to just collapse the list and get rid of the field names, e.g.:
>>> dic = [{'name': 'john', 'items':['pants_1','shirt_2','socks_3']},
{'name': 'bob', 'items':['jacket_1','hat_1']}]
>>> data = {d['name']: dict(i.split('_') for i in d['items']) for d in dic}
>>> data
{'bob': {'hat': '1', 'jacket': '1'},
'john': {'pants': '1', 'shirt': '2', 'socks': '3'}}
Now the data is directly related vs. indirectly related via a common index into 2 lists. If you want the dictionary split out you can always
>>> dic_name, dic_items = zip(*data.items())
>>> dic_name
('bob', 'john')
>>> dic_items
({'hat': '1', 'jacket': '1'}, {'pants': '1', 'shirt': '2', 'socks': '3'})
You need a list of dictionaries because the duplicate keys name and items are overwritten:
items = [[i.split('_')[0] for i in d['items']] for d in your_list]
names = [d['name'] for d in your_list] # then grab names from list
Alternatively, you can do this in one line with the built-in zip method and generators, like so:
names, items = zip(*((i['name'], [j.split('_')[0] for j in i['items']]) for i in dic))
From Looping Techniques in the Tutorial.
for name, items in div.items():
names.append(name)
items.append(item)
That will work if your dict is structured
{'name':[item1]}
In the loop body of
for x in dic:
dic_name.append(dic['name'])
dic_items.append(dic['items'])
you'll probably want to access x (to which the items in dic will be assigned in turn) rather than dic.
I would like to map directly some configuration parameters from YAML into Python argument names. Just wondering if there is a way without writing extra-code (to modify keys afterwards) to let YAML parser replace dash '-' in a key with an underscore '_'.
some-parameter: xyz
some-other-parameter: 123
Should become when parsed with PyYAML (or may be other lib) a dictionary with values:
{'some_parameter': 'xyz', 'some_other_parameter': 123}
Than I can pass the dictionary to a function as named parameters:
foo(**parsed_data)
I know I can iterate through the keys afterwards and modify their values, but I don't want to do that :)
At least for your stated case, you don't need to transform the keys. Given:
import pprint
def foo(**kwargs):
print 'KWARGS:', pprint.pformat(kwargs)
If you set:
values = {
'some-parameter': 'xyz',
'some-other-parameter': 123,
}
And then call:
foo(**values)
You get:
KWARGS: {'some-other-parameter': 123, 'some-parameter': 'xyz'}
If you goal is is actually to call a function like this:
def foo(some_parameter=None, some_other_parameter=None):
pass
Then sure, you would need to map the key names. But you could just do this:
foo(**dict((k.replace('-','_'),v) for k,v in values.items()))
I think I found a solution: There is a package which is called yconf: https://pypi.python.org/pypi/yconf
I can map there values and work with it using the well-known argparse-interface:
config.yml
logging:
log-level: debug
Argparse like definition:
parser.add_argument("--log-level", dest="logging.log-level")
Well, If you parse the YAML file as a python dictionary, you can use the following code to convert all dash (inside all nested dictionaries and arrays) with dash.
def hyphen_to_underscore(dictionary):
"""
Takes an Array or dictionary and replace all the hyphen('-') in any of its keys with a underscore('_')
:param dictionary:
:return: the same object with all hyphens replaced by underscore
"""
# By default return the same object
final_dict = dictionary
# for Array perform this method on every object
if type(dictionary) is type([]):
final_dict = []
for item in dictionary:
final_dict.append(hyphen_to_underscore(item))
# for dictionary traverse all the keys and replace hyphen with underscore
elif type(dictionary) is type({}):
final_dict = {}
for k, v in dictionary.items():
# If there is a sub dictionary or an array perform this method of it recursively
if type(dictionary[k]) is type({}) or type(dictionary[k]) is type([]):
value = hyphen_to_underscore(v)
final_dict[k.replace('-', '_')] = value
else:
final_dict[k.replace('-', '_')] = v
return final_dict
Here is a sample usage
customer_information = {
"first-name":"Farhan",
"last-name":"Haider",
"address":[{
"address-line-1": "Blue Mall",
"address-line-2": None,
"address-type": "Work"
},{
"address-line-1": "DHA",
"address-line-2": "24-H",
"address-type": "Home"
}],
"driver_license":{
"number": "209384092834",
"state-region": "AB"
}
}
print(hyphen_to_underscore(customer_information))
# {'first_name': 'Farhan', 'last_name': 'Haider', 'address': [{'address_line_1': 'Blue Mall', 'address_line_2': None, 'address_type': 'Work'}, {'address_line_1': 'DHA', 'address_line_2': '24-H', 'address_type': 'Home'}], 'driver_license': {'number': '209384092834', 'state_region': 'AB'}}