Develop Reference

Converting a dictionary with lists into a list of dictionaries

I have this dictionary containing lists as they're values for keys.
d = {'n': ['a', 'b', 'x'], 'a': [1, 2, 3], 'p': ['123', '321', '456']}
And I want to convert into a list of dictionary as such
[{'n': 'a', 'a': 1, 'p': '123'}, {'n': 'b', 'a': 2, 'p':
'321'}, {'n': 'x', 'a': 3, 'p': '456'}]
My current solution is,
the_kv = d.items()
f = {}
c = {}
e = {}
f_c_e = []
for i, j in the_kv:
f[i] = j[0]
c[i] = j[1]
e[i] = j[2]
f_c_e.append(f)
f_c_e.append(c)
f_c_e.append(e)
print(f_c_e)
But I was wondering if there is a more efficient way rather than creating separate dicts and then appending them to a list.

Using zip:
[dict(zip(d, vals)) for vals in zip(*d.values())]
Result:
[{'n': 'a', 'a': 1, 'p': '123'}, {'n': 'b', 'a': 2, 'p': '321'}, {'n': 'x', 'a': 3, 'p': '456'}]

Use:
res = [dict(v) for v in zip(*[[(key, value) for value in values] for key, values in d.items()])]
print(res)
Output
[{'n': 'a', 'a': 1, 'p': '123'}, {'n': 'b', 'a': 2, 'p': '321'}, {'n': 'x', 'a': 3, 'p': '456'}]
A simpler alternative approach is to do:
result = [{} for _ in range(len(d))]
for key, values in d.items():
for i, value in enumerate(values):
result[i][key] = value
print(result)
Output (alternative)
[{'n': 'a', 'a': 1, 'p': '123'}, {'n': 'b', 'a': 2, 'p': '321'}, {'n': 'x', 'a': 3, 'p': '456'}]

Let's work through the problem step by step.
The core difficulty is that we have a sequence of lists:
['a', 'b', 'x']
[1, 2, 3]
['123', '321', '456']
And we want to produce sequences consisting of element 0 of each list, element 1 of each list, etc. (Each of these sequences contains all the values for an output dict.) Which is to say, we want to transpose the lists, which is exactly what the built-in zip is for:
# Feed the generator to `list` to unpack and view them
list(zip(
['a', 'b', 'x'],
[1, 2, 3],
['123', '321', '456']
))
Now we can sketch out a complete process:
Get the keys and values of the input (in the same order).
Transpose the values.
For each sequence in the transposed values, match that sequence up with the keys to make a new dict.
The first two parts are easy:
keys, value_lists = d.keys(), d.values()
# Since the .values() are a sequence, while `zip` accepts multiple
# arguments, we need to use the `*` operator to unpack them as
# separate arguments.
grouped_values = zip(*value_lists)
To finish up, let's first figure out how to create a single result dict from a single one of the new_values. It's easy to make a dict from a bunch of key-value pairs - we can feed that directly to dict. However, we have instead a pair of sequences - the original .keys(), and a result from the zip. Clearly, the solution is to zip again - we can make a trivial helper function to make sure everything is as clear as possible:
def new_dict(keys, values):
return dict(zip(keys, values))
Then all we need to do is repeatedly apply that function:
new_dicts = [new_dict(keys, values) for values in grouped_values]
Putting everything inline with short names for the sake of showing off gives:
new_dicts = [dict(zip(d.keys(), v)) for v in zip(*d.values())]
which is almost exactly Jab's answer (note that iterating over a dictionary gives the keys, so you can pass d directly rather than d.keys() to zip since zip will only be iterating over it).

Good Question.
Conversion from one data type to other is essential in either development or competitive programming.
It can be done in many methods, I'm explaining the two methods which I know, below:
Method #1 : Using list comprehension
We can use list comprehension as the one-liner alternative to perform various naive tasks providing readability with a more concise code. We can iterate through each of dictionary element and corresponding keep constructing the list of dictionary.
# Python3 code to demonstrate
# to convert dictionary of list to
# list of dictionaries
# using list comprehension
# initializing dictionary
test_dict = { "Rash" : [1, 3], "Manjeet" : [1, 4], "Akash" : [3, 4] }
# printing original dictionary
print ("The original dictionary is : " + str(test_dict))
# using list comprehension
# to convert dictionary of list to
# list of dictionaries
res = [{key : value[i] for key, value in test_dict.items()}
for i in range(2)]
# printing result
print ("The converted list of dictionaries " + str(res))
Method #2 : Using zip()
This approach used zip function two times, first time when we need to zip the particular index value of all lists as one and second to get all values of particular index and zip it with the corresponding keys.
# Python3 code to demonstrate
# to convert dictionary of list to
# list of dictionaries
# using zip()
# initializing dictionary
test_dict = { "Rash" : [1, 3], "Manjeet" : [1, 4], "Akash" : [3, 4] }
# printing original dictionary
print ("The original dictionary is : " + str(test_dict))
# using zip()
# to convert dictionary of list to
# list of dictionaries
res = [dict(zip(test_dict, i)) for i in zip(*test_dict.values())]
# printing result
print ("The converted list of dictionaries " + str(res))
Output
The original dictionary is : {‘Rash’: [1, 3], ‘Manjeet’: [1, 4], ‘Akash’: [3, 4]}
The converted list of dictionaries [{‘Rash’: 1, ‘Manjeet’: 1, ‘Akash’: 3}, {‘Rash’: 3, ‘Manjeet’: 4, ‘Akash’: 4}]

How could one reduce the usage of helper functions in lambda expressions?

In this example I'm taking letters from a set and append them to a dictionary where the letter becomes the key and the literal 1 becomes the value to each pair.
def base_dict_from_set(s):
return reduce(lambda d,e : addvalue(1, e, d), s, dict())
def addvalue(value, key, d):
d[key] = value
return d
>>> base_dict_from_set(set("Hello World!".lower()))
{'o': 1, '!': 1, 'l': 1, 'd': 1, 'w': 1, ' ': 1, 'r': 1, 'e': 1, 'h': 1}
I was wondering whether I could somehow be rid of the 'addvalue' helper function and add the element and reference the modified dictionary within the lambda function itself.
The routine within addvalue itself seams very simple to me, so I would prefer something that looks like this:
def base_dict_from_set(s):
reutrn reduce(lambda d,e : d[e] = 1, s, dict())
I don't have a lot of experience in python and I come from a functional programming perspective. My goal is to understand pythons functional capabilities but I am too unexperienced to properly phrase and google what I am looking for.

What you are trying to do is why dict.fromkeys exists: create a dict that maps each key to the same constant value.
>>> dict.fromkeys("Hello World!".lower(), 1)
{'h': 1, 'e': 1, 'l': 1, 'o': 1, ' ': 1, 'w': 1, 'r': 1, 'd': 1, '!': 1}
There's no need to convert the string to a set first, since any duplicates will just be overwritten by the following occurrences.
(If the constant value is mutable, you should use the dict comprehension to ensure that each key gets its own mutable value, rather than every key sharing a reference to the same mutable value.)

You can use a dict comprehension for the same result:
{l: 1 for l in set("Hello World!".lower())}

To answer exactly the question asked, yes you can get rid of the addvalue by replacing addvalue(1, e, d) with {**d, e:1}.
Nevertheless, your code is still faulty. It is not counting the occurrences, but creates a dict of key: 1 for every letter in the string and it should create a dict of key: number_of_occurences to achieve this you should replace addvalue(1, e, d) with {**d, e: 1 + (d[e] if e in d else 0)} and not convert the string to set as it eliminates duplicates

I'm a bit surprised that you tried to use reduce when your goal is to transform each item in an input collection (the letters in a string) to an output collection (a key/value pair where the key is the letter and the value is a constant number), independently of each other.
In my view, reduce is for when an operation needs to be done to items in a sequence and taking all previous items into account (for instance, when calculating a sum of values).
So in a functional style, using map here would be more appropriate than reduce, in my opinion. Python supports this:
def quant_dict_from_set(s):
return dict(map(lambda c: (c, 1), s.lower()))
Where map converts the string to key/value pairs and the dict constructor collects these pairs in a dictionary, while eliminating duplicate keys at the same time.
But more idiomatic approaches would be to use a dictionary comprehension or the dict.fromkeys constructor.

Hacky and hard to read, but closest to the lambda you were trying to write, and hopefully educational:
>>> f = lambda d, e: d.__setitem__(e, 1) or d
>>> d = {}
>>> output = f(d, 42)
>>> output
{42: 1}
>>> output is d
True
Using __setitem__ avoids the = assignment.
__setitem__ returns None, so the expression d.__setitem__(e, 1) or d always evaluates to d, which is returned by the lambda.

You can use collections.Counter, a subclass of dict specifically for counting occurrences of elements.
>>> import collections
>>> collections.Counter('Hello, World!'.lower())
Counter({'l': 3, 'o': 2, 'h': 1, 'e': 1, ',': 1, ' ': 1, 'w': 1, 'r': 1, 'd': 1, '!': 1})
>>> collections.Counter(set('Hello, World!'.lower()))
Counter({'w': 1, 'l': 1, 'r': 1, ',': 1, 'h': 1, 'd': 1, 'o': 1, 'e': 1, ' ': 1, '!': 1})
Note that Counter is appropriate if you want to count the elements, of if you want to initiate the values to the constant 1. If you want to initiate the values to another constant, then Counter will not be the solution and you should use a dictionary comprehension or the dict.fromkeys constructor.

Is there a way to randomly shuffle keys and values in a Python Dictionary, but the result can't have any of the original key value pairs?

I would like to shuffle the key value pairs in this dictionary so that the outcome has no original key value pairs. Starting dictionary:
my_dict = {'A':'a',
'K':'k',
'P':'p',
'Z':'z'}
Example of unwanted outcome:
my_dict_shuffled = {'Z':'a',
'K':'k', <-- Original key value pair
'A':'p',
'P':'z'}
Example of wanted outcome:
my_dict_shuffled = {'Z':'a',
'A':'k',
'K':'p',
'P':'z'}
I have tried while loops and for loops with no luck. Please help! Thanks in advance.

Here's a fool-proof algorithm I learned from a Numberphile video :)
import itertools
import random
my_dict = {'A': 'a',
'K': 'k',
'P': 'p',
'Z': 'z'}
# Shuffle the keys and values.
my_dict_items = list(my_dict.items())
random.shuffle(my_dict_items)
shuffled_keys, shuffled_values = zip(*my_dict_items)
# Offset the shuffled values by one.
shuffled_values = itertools.cycle(shuffled_values)
next(shuffled_values, None) # Offset the values by one.
# Guaranteed to have each value paired to a random different key!
my_random_dict = dict(zip(shuffled_keys, shuffled_values))
Disclaimer (thanks for mentioning, #jf328): this will not generate all possible permutations! It will only generate permutations with exactly one "cycle". Put simply, the algorithm will never give you the following outcome:
{'A': 'k',
'K': 'a',
'P': 'z',
'Z': 'p'}
However, I imagine you can extend this solution by building a random list of sub-cycles:
(2, 2, 3) => concat(zip(*items[0:2]), zip(*items[2:4]), zip(*items[4:7]))

A shuffle which doesn't leave any element in the same place is called a derangement. Essentially, there are two parts to this problem: first to generate a derangement of the keys, and then to build the new dictionary.
We can randomly generate a derangement by shuffling until we get one; on average it should only take 2-3 tries even for large dictionaries, but this is a Las Vegas algorithm in the sense that there's a tiny probability it could take a much longer time to run than expected. The upside is that this trivially guarantees that all derangements are equally likely.
from random import shuffle
def derangement(keys):
if len(keys) == 1:
raise ValueError('No derangement is possible')
new_keys = list(keys)
while any(x == y for x, y in zip(keys, new_keys)):
shuffle(new_keys)
return new_keys
def shuffle_dict(d):
return { x: d[y] for x, y in zip(d, derangement(d)) }
Usage:
>>> shuffle_dict({ 'a': 1, 'b': 2, 'c': 3 })
{'a': 2, 'b': 3, 'c': 1}

theonewhocodes, does this work, if you don't have a right answer, can you update your question with a second use case?
my_dict = {'A':'a',
'K':'k',
'P':'p',
'Z':'z'}
while True:
new_dict = dict(zip(list(my_dict.keys()), random.sample(list(my_dict.values()),len(my_dict))))
if new_dict.items() & my_dict.items():
continue
else:
break
print(my_dict)
print(new_dict)

Not getting the same result when inverting a dictionary twice

I'm trying to invert a simple dictionary like:
{'a' : 1, 'b' : 2, 'c' : 3, 'd' : 4}
I'm using this function:
def invert(d):
return dict([(x,y) for y,x in d.iteritems()])
Now when I invert my dictionary, everything works out fine. When I invert it twice however, I get:
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
which is not in the same order as the dictionary I started with. Is there a problem with my invert function? Sorry I'm kinda new to python, but thanks for any help!

That is correct, dictionaries are unordered in python
from another so answer answer:
CPython implementation detail: Keys and values are listed in an
arbitrary order which is non-random, varies across Python
implementations, and depends on the dictionary’s history of insertions
and deletions.
from the docs:
It is best to think of a dictionary as an unordered set of key: value
pairs, with the requirement that the keys are unique (within one
dictionary). A pair of braces creates an empty dictionary: {}. Placing
a comma-separated list of key:value pairs within the braces adds
initial key:value pairs to the dictionary; this is also the way
dictionaries are written on output.

Python dictionaries are unsorted by design.
You can use collections.OrderedDict instead if you really need this behaviour.

Try running this code:
d = {
'a' : 1, 'b' : 2,
'c' : 3, 'd' : 4
}
def invert(d):
return dict([(x,y) for y,x in d.iteritems()])
print d
d = invert(d)
print d
d = invert(d)
print d
This is the output:
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
{1: 'a', 2: 'b', 3: 'c', 4: 'd'}
{'a': 1, 'c': 3, 'b': 2, 'd': 4}
As you can see, it technically is the same dictionary, but when you declare it, it is unordered.

Are there dictionary comprehensions in Python? (Problem with function returning dict)

I know about list comprehensions, what about dictionary comprehensions?
Expected Output:
>>> countChar('google')
{'e': 1, 'g': 2, 'l': 1, 'o': 2}
>>> countLetters('apple')
{'a': 1, 'e': 1, 'l': 1, 'p': 2}
>>> countLetters('')
{}
Code (I'm a beginner):
def countChar(word):
l = []
#get a list from word
for c in word: l.append(c)
sortedList = sorted(l)
uniqueSet = set(sortedList)
return {item:word.count(item) for item in uniqueSet }
What is the problem with this code? Why do I get this SyntaxError?
return { item:word.count(item) for item in uniqueSet }
^
SyntaxError: invalid syntax

If you're on Python 2.7 or newer:
{item: word.count(item) for item in set(word)}
works fine. You don't need to sort the list before you set it. You also don't need to turn the word into a list. Also, you're on a new enough Python to use collections.Counter(word) instead.
If you're on an older version of Python, you can't use dict comprehensions, you need to use a generator expression with the dict constructor:
dict((item, word.count(item)) for item in set(word))
This still requires you to iterate over word len(set(word)) times, so try something like:
from collections import defaultdict
def Counter(iterable):
frequencies = defaultdict(int)
for item in iterable:
frequencies[item] += 1
return frequencies

edit: As agf pointed out in comments and the other answer, there is a dictionary comprehension for Python 2.7 or newer.
def countChar(word):
return dict((item, word.count(item)) for item in set(word))
>>> countChar('google')
{'e': 1, 'g': 2, 'o': 2, 'l': 1}
>>> countChar('apple')
{'a': 1, 'p': 2, 'e': 1, 'l': 1}
There is no need to convert word to a list or sort it before turning it into a set since strings are iterable:
>>> set('google')
set(['e', 'o', 'g', 'l'])
There is no dictionary comprehension with for Python 2.6 and below, which could be why you are seeing the syntax error. The alternative is to create a list of key-value tuples using a comprehension or generator and passing that into the dict() built-in.