Develop Reference

Failure converting complex code to a list comprehension

NB: None of the suggested answers work for me as they are for-loop driven (which I already have working) and don't explain what is wrong with the list comprehension version.
I am trying to transform
('name:80', 'desc:100')
into
{'name': 80, 'desc': 100}
A working for-loop:
new_wrap = {}
for item in wrap:
k, v = item.split(':')
new_wrap[k] = int(v)
wrap = new_wrap
a non-working list-comprehension:
wrap = dict([
(k, int(v))
for item in wrap
for (k, v) in item.split(':')
])
which gives
Traceback (most recent call last):
File ..., line ..., in ...
for (k, v) in item.split(':')
ValueError: too many values to unpack
What am I doing wrong?

You don't want to iterate over item.split(':'), you want to unpack it. To do that you can introduce a nested comprehension or generator expression. That gives you an opportunity to create the new variables k and v:
dict([
(k, int(v))
for k,v in (item.split(':') for item in wrap)
])
I would then get rid of the square brackets. It's more efficient to use a generator expression that generates entries on the fly rather than a list comprehension that builds a whole list in memory just to throw it away when the dict is created.
dict((k, int(v))
for k,v in (item.split(':') for item in wrap))
You simplify it even further by using a dict comprehension with curly braces:
{k: int(v) for k,v in (item.split(':') for item in wrap)}

If you would like to stick to a comprehension that's one-level deep and don't mind using Python 3.8+, you can use an assignment expression to hold the result of the split in a variable for indexing:
{(t := s.split(':'))[0]: int(t[1]) for s in wrap}

You should separate splitting from conversion to an int:
{k: int(v) for k, v in [w.split(":") for w in wrap]}
#{'name': 80, 'desc': 100}
But, as roganjosh correctly suggested, you gain nothing from this.

With the insight provided by John Kugelman I was able to figure out the problem and the solution I wanted.
The working code from the for loop:
k, v = item.split(':')
Breakdown:
item.split(':') == 'name:80'.split(':') == 'name', '80'
k, v = 'name', '80'
k == 'name' and v == 80
The non-working code in the list-comp:
for (k, v) in item.split(':')
Breakdown:
item.split(':') == 'name:80'.split(':') == 'name', '80'
for (k, v) in ('name', '80')
And there's the problem: in is not the same as =. in iterates over its arguments, returning one at a time, so:
k, v = 'n', 'a', 'm', 'e' is what is attempted, and because four items are trying to be stuffed into two we get the ValueError: too many values to unpack exception.
The simple solution is to encapsulate the item.split() into a tuple of its own:
for k, v in (item.split(':'), )
which becomes
for k, v in (('name', '80'), )
and since the first item in the tuple is a tuple we get
k, v = 'name', '80' for the first (and only) iteration of the second for loop
and the problem is solved without nesting another list-comp inside the primary list-comp.
The final, working list-comp:
[
(k, int(v))
for item in wrap
for (k, v) in (item.split(':'), )
]

i found this solution:
x = ('name:80', 'desc:100')
dict([tuple(a.split(':')) for a in x])

Condensing the following code, preferably using a dictionary comprehension

Is it possible (and, more importantly, practical) to write the following code as a dict comprehension?
I am creating a dictionary here and then checking for "blank" values (represented by a '-') and replacing it with another string value.
test_dict = dict(zip(list_one,list_two))
for k,v in test_dict.items():
if v == '-':
test_dict[k] = 'missing'

Why not do the replacement when you are creating the dictionary?
test_dict = dict(zip(list_one,
('missing' if x == '-' else x for x in list_two)))
Or, if you have an existing dictionary you can create a new one using:
{k: 'missing' if v == '-' else v for k, v in d.items()}
If you wish to alter the existing dictionary in-place then really there is nothing wrong with the existing code. You could use a list comprehension such as:
[d.__setitem__(k, 'missing') for k, v in d.items() if v == '-']
but that is rather ugly.

Python 2.7 Improve algorithm that checks if a dictionary key is not in a list and if so deletes that key

I was playing around with a few solutions but am looking for an optimal and clean way to do this in Python. Basically I have a dictionary called 'd', and a list 'l'. If the dictionary contains a value not in the list delete that value in the dictionary. So given:
d = {1 : "bob", 2 : "mary", 3 : "joe"}
l = [2,3]
The first key of d gets deleted.
d = {2 : "mary", 3 : "joe"}
l = [2,3]
I have a working solution but feel the use of flags is unnecessary.
flag = False
del_vals = list()
for key in d.iterkeys():
for i in l:
if key == i: flag = True
if flag == False:
del_vals.append(key)
flag = False
for k in del_vals:
d.pop(k, None)
Is there a way I can write it to improve performance and make it more clean? Whether the use of generators or anything else is involved.

A list comprehension will work starting from Python 2.7:
d = {k: v for k, v in d.iteritems() if k in l}
Use items() instead of iteritems() in Python 3:
d = {k: v for k, v in d.items() if k in l}

You can also use sets for achieving the same:
z={i:d[i] for i in set(d.keys()).intersection(l)}
or use the below
p={i:d[i] for i in d.keys() if i in l}
or do this
q={i:d[i] for i in set(d.keys()) & set(l)}
Alternatively, you could do this as well:
for i in set(d.keys()) - set(l):
del(d[i])

Method of expanding list of dictionaries into list of formatted strings

I've run across a need to call out to an external executable using the subprocess module. Everything is working fine, however I'd like to improve how I'm generating the commandline arguments.
The executable's command line options require formatting as follows:
--argname=argvalue
I currently have a list of dictionaries as follows:
[{arg1:value1},{arg2:value2}]
What is the best method of expanding these dictionaries into their proper string format? I'm currently iterating over the list, appending to a different list, however it feels there's a more pythonic method I should utilize.

Use items() as in http://docs.quantifiedcode.com/python-code-patterns/readability/not_using_items_to_iterate_over_a_dictionary.html
for key,val in d.items():
print("{} = {}".format(key, val))

' '.join('--{key}={value}'.format(key = k, value = v) for d in arg_list for k, v in d.items())
Essentially, this iterates over each dictionary in the list (for d in arg_list) and then iterates over the items in each dictionary (for k, v in d.items()). Each item is formatted into the proper form, and then all of those key-value pairs are combined.
It is equivalent to:
arg_list = [{arg1:value1},{arg2:value2}]
formatted_args = []
for d in arg_list:
for k, v in d.items():
# Format each item per dictionary
formatted_args.append('--{key}={value}'.format(key = k, value = v))
# Combine the arguments into one string
args_string = ' '.join(formatted_args)

Try this
','.join('{}={}'.format(k, v) for d in arg_list for k, v in d.items())

How about
def format_dict(d):
key = d.keys()[0]
return '--%s=%s' % (key, d[key])
ex = [{'a':'3'}, {'b':'4'}]
print ' '.join(map(format_dict, ex)) # --a=3 --b=4

Is it possible to use dict comprehension in a for loop without creating a new dict?

In Python 3.5, I am trying to iterate over a for loop using dict comprehension, yet it does not seem to work as it does with list. The following will be reported as a syntax error (not by PyCharm, only at runtime):
for k, v in (k, v for k, v in {"invalid": 1, "valid": 2}.items() if k == "valid"): # syntax error
print("Valid: " + (str(v)))
While the following works:
for e in (e for e in ["valid", "invalid"] if e == "valid"): # works
print(e)
I know creating a new dict would work (e.g. ), but I want to avoid the overhead as I am just doing operations on the elements.
for k, v in {k: v for k, v in my_dict.items() if k == "valid"}.items(): # works
print("Valid: " + (str(k)))
Of course, I could use a plain old if k == "valid": continue condition inside the loop, but I would like to understand why dict comprehension seems to be more limited than with list.

Python needs parentheses around the first k, v.
((k, v) for k, v in {"invalid": 1, "valid": 2}.items() if k == "valid")
Otherwise it looks like you're trying to make a tuple with two elements:
k
v for k, v in ...
The second isn't syntactically valid. (It would be if you put parentheses around it, making it a generator expression.)

You need to create a tuple with k, v i.e (k,v):
for k, v in ((k, v) for k, v in {"invalid": 1, "valid": 2}.items() if k == "valid"):
^^^
That is what is causing the syntax error. If you did the same thing with a list, set comprehension etc.. it would be exactly the same, you need to use a container if you want to add more that one element. The second example (e for e... works because you are returning a single element each time, the first two examples are also generator expressions only the latter is actually a dict comp.