Assuming, I have a list bites with constant length
>>> chunk = [b'\xd0', b'\x00', b'\xd5', b'\x00', b'\xdf', b'\x00', b'\xaa', b'U']
>>> print(type(chunk))
<class 'list'>
>>> print(chunk)
[b'\xd0', b'\x00', b'\xd5', b'\x00', b'\xdf', b'\x00', b'\xaa', b'U']
And I want to get a single int from it, which could be done this way:
>>> a = int.from_bytes(chunk[2], byteorder='little')
>>> b = int.from_bytes(chunk[3], byteorder='little')
>>> decoded = a * 256 + b
>>> print(decoded)
213
How do I use struct.unpack for that purpose?
Several ways I failed:
>>> print(struct.unpack('<xxHxxxx', chunk))
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: a bytes-like object is required, not 'list'
>>> tmp = bytes(chunk)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'bytes' object cannot be interpreted as an integer
>>> tmp = bytes([chunk])
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'list' object cannot be interpreted as an integer
>>> tmp = bytearray(chunk)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: 'bytes' object cannot be interpreted as an integer
The initial motivation was to check if struct.unpack is any faster then already working code. But now I want to figure out the right way to use it.
Oh, and if there is any more efficient way, I would be glad to know that too!
The issue is in how you convert a list of bytes to bytes. You can do this with b''.join(list) and then apply your struct unpack.
Example:
>>> import struct
# your list of bytes
>>> lst = [b'\xd0', b'\x00', b'\xd5', b'\x00', b'\xdf', b'\x00', b'\xaa', b'U']
# Convert into `bytes`
>>> bts = b''.join(lst)
# Unpack
>>> print(struct.unpack('<xxHxxxx', bts))
(213,)
Note that you can also unpack the returned tuple with:
>>> (a,) = struct.unpack('<xxHxxxx', bts)
>>> a
213
how to make the function test() work correctly?
Python 3.4.1
a function into string does not work well when this string is inside a function.
how define this function that inside a string?
def func(x):
return x+1
def test(a,b):
loc = {'a':a,'b':b}
glb = {}
exec('c = [func(a+i)for i in range(b)]', glb,loc)
c = loc['c']
print(c)
print('out the function test()')
a= 1
b= 4
c = [func(a+i)for i in range(b)]
print(c)
'''results:
out the function test()
[2, 3, 4, 5]
>>> test(1,4)
Traceback (most recent call last):
File "<pyshell#1>", line 1, in <module>
test(1,4)
File "C:\Users\Rosania\Documents\Edilon\Python examples\apagar.py", line 6, in test
exec('c = [func(a+i)for i in range(b)]', glb,loc)
File "<string>", line 1, in <module>
File "<string>", line 1, in <listcomp>
NameError: name 'func' is not defined
'''
This is kind of evil to eval a string.
Assuming you know what you're doing...
Put "func" in the locals dict too. Your eval environments must know about everything you expect to reference in your eval'd string.
loc = {'a':a, 'b':b, 'func':func}
I need to have a dictionary which might have same names for some keys and return a list of values when referencing the key in that case.
For example
print mydict['key']
[1,2,3,4,5,6]
For consistency, you should have the dictionary map keys to lists (or sets) of values, of which some can be empty. There is a nice idiom for this:
from collections import defaultdict
d = defaultdict(set)
d["key"].add(...)
(A defaultdict is like a normal dictionary, but if a key is missing it will call the argument you passed in when you instantiated it and use the result as the default value. So this will automatically create an empty set of values if you ask for a key which isn't already present.)
If you need the object to look more like a dictionary (i.e. to set a value by d["key"] = ...) you can do the following. But this is probably a bad idea, because it goes against the normal Python syntax, and is likely to come back and bite you later. Especially if someone else has to maintain your code.
class Multidict(defaultdict):
def __init__(self):
super(Multidict, self).__init__(set)
def __setitem__(self, key, value):
if isinstance(value, (self.default_factory)): # self.default_factory is `set`
super().__setitem__(key, value)
else:
self[key].append(value)
I haven't tested this.
You can also try paste.util.multidict.MultiDict
$ easy_install Paste
Then:
from paste.util.multidict import MultiDict
d = MultiDict()
d.add('a', 1)
d.add('a', 2)
d.add('b', 3)
d.mixed()
>>> {'a': [1, 2], 'b': 3}
d.getall('a')
>>> [1, 2]
d.getall('b')
>>> [3]
Web frameworks like Pylons are using this library to handle HTTP query string/post data, which can have same-name keys.
You can use:
myDict = {'key': []}
Then during runtime:
if newKey in myDict:
myDict[newKey].append(value)
else:
myDict[newKey] = [value]
Edited as per #Ben's comment:
myDict = {}
myDict.setdefault(newKey, []).append(value)
This is an ideal place to use a defaultdict object from the collections library
from collections import defaultdict
mydict = defaultdict(set)
mydict['key'] += set([1,2,3,4])
mydict['key'] += set([4,5,6])
print(mydict['key'])
returns [1,2,3,4,5,6]
In the case where a key is referenced that has not been implicitly assigned, an empty set is returned.
print(mydict['bad_key'])
returns []
Using setdefault on a dict from the standard library would require a significant change in your syntax when assigning values and can get rather messy. I've never used Multidict, but it also looks like a significant change in the way assignments are made. Using this method, you simply assume that there may already be a value associated with this key in the dictionary and slightly modify your assignment operator by using the '+=' operator when assigning key values.
FYI - I am a big fan of using the NoneType as the default which results in any access of an invalid key returning None. This behaves properly in most cases including iterating and json dumps, but for your specific need the default should be of type set unless you want to enable having duplicate values stored in the key. Then use a list. In fact, anytime you have a homogenous dictionary the default should be of that type.
mydict = defaultdict(lambda: None)
I'm unsatisfied with all the proposed solutions, so this is my solution. This is for Python 3. Code is below.
EXAMPLES
(code is below)
>>> a = MultiDict({0: [0]})
>>> a
MultiDict({0: [0]})
>>> a[0] = (1, 7)
>>> a
MultiDict({0: [1, 7]})
>>> a.add(0, 2)
>>> a
MultiDict({0: [1, 7, 2]})
>>> a.add(1, 2)
>>> a
MultiDict({0: [1, 7, 2], 1: [2]})
>>> a.getfirst(0)
1
>>> a.getfirst(3)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 61, in getfirst
File "<stdin>", line 17, in __getitem__
KeyError: 3
>>> len(a)
2
>>> tuple(a.items())
((0, [1, 7, 2]), (1, [2]))
>>> tuple(a.values())
([1, 7, 2], [2])
>>> a.get(0)
[1, 7, 2]
>>> tuple(a.multiitems())
((0, 1), (0, 7), (0, 2), (1, 2))
>>> tuple(a.multikeys())
(0, 0, 0, 1)
>>> tuple(a.multivalues())
(1, 7, 2, 2)
>>> a.remove(0, 1)
>>> a
MultiDict({0: [7, 2], 1: [2]})
>>> a.remove(3, 5)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 53, in remove
File "<stdin>", line 17, in __getitem__
KeyError: 3
>>> a.remove(0, 5)
Traceback (most recent call last):
File "<stdin>", line 53, in remove
ValueError: list.remove(x): x not in list
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 56, in remove
ValueError: No element with value 5 for key 0
>>> b = MultiDict({0: [7, 2], 1: [2]})
>>> b == a
True
>>> c = MultiDict(a)
>>> c
MultiDict({0: [7, 2], 1: [2]})
>>> d = MultiDict({0: 0})
Traceback (most recent call last):
File "<stdin>", line 30, in __init__
TypeError: 'int' object is not iterable
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 33, in __init__
TypeError: Values must be iterables, found 'int' for key 0
>>> a.pop(0)
[7, 2]
>>> a
MultiDict({1: [2]})
>>> c.popitem()
(0, [7, 2])
>>> c.setdefault(0, [1])
[1]
>>> c
MultiDict({0: [1], 1: [2]})
>>> c.setdefault(0, [2])
[1]
>>> c
MultiDict({0: [1], 1: [2]})
>>> c.setdefault(3)
[]
>>> c
MultiDict({0: [1], 1: [2], 3: []})
>>> c.getfirst(3)
Traceback (most recent call last):
File "<stdin>", line 61, in getfirst
IndexError: list index out of range
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 63, in getfirst
IndexError: No values in key 3
>>> c.clear()
>>> c
MultiDict({})
>>> c.update(b)
>>> c
MultiDict({0: [7, 2], 1: [2]})
>>> d = c.copy()
>>> d == c
True
>>> id(d) == id(c)
False
>>> MultiDict.fromkeys((0, 1), [5])
MultiDict({0: [5], 1: [5]})
>>> MultiDict.fromkeys((0, 1))
MultiDict({0: [], 1: []})
CODE
try:
from collections.abc import MutableMapping
except ImportError: # python < 3.3
from collections import MutableMapping
class MultiDict(MutableMapping):
#classmethod
def fromkeys(cls, seq, value=None, *args, **kwargs):
if value is None:
v = []
else:
v = value
return cls(dict.fromkeys(seq, v, *args, **kwargs))
def __setitem__(self, k, v):
self._dict[k] = list(v)
def __getitem__(self, k):
return self._dict[k]
def __iter__(self):
for k in self._dict:
yield k
def __init__(self, *args, **kwargs):
self._dict = dict(*args, **kwargs)
for k, v in self._dict.items():
try:
self._dict[k] = list(v)
except TypeError:
err_str = "Values must be iterables, found '{t}' for key {k}"
raise TypeError(err_str.format(k=k, t=type(v).__name__))
def __delitem__(self, k):
del self._dict[k]
def __len__(self):
return len(self._dict)
def add(self, k, v):
if not k in self:
self[k] = []
self[k].append(v)
def remove(self, k, v):
try:
self[k].remove(v)
except ValueError:
err_str = "No element with value {v} for key {k}"
raise ValueError(err_str.format(v=v, k=k))
def getfirst(self, k):
try:
res = self[k][0]
except IndexError:
raise IndexError("No values in key {k}".format(k=k))
return self[k][0]
def multiitems(self):
for k, v in self.items():
for vv in v:
yield (k, vv)
def multikeys(self):
for k, v in self.items():
for vv in v:
yield k
def multivalues(self):
for v in self.values():
for vv in v:
yield vv
def setdefault(self, k, default=None):
if default is None:
def_val = []
else:
def_val = default
if k not in self:
self[k] = def_val
return self[k]
def copy(self):
return self.__class__(self)
def __repr__(self):
return (
self.__class__.__name__ +
"({{{body}}})".format(body=self._dict)
)
SOME VERBOSE EXPLAINATION
For simplicity, the constructor is the same as dict. All values passed to the constructor, or assigned directly to a key, must be iterables.
All the values of my MultiDict are lists, even if value is only one. This is to avoid confusion.
I added also a remove method to delete a single entry from the MultiDict. Furthermore I added a multiitems, that iters over the couple (key, value) over all the values of the dictionary. multikeys and multivalues are similar.
ALTERNATIVES
You can also use aiohttp, WebOp or Werkzeug implementations of MultiDict.
def toMultiDict(items):
def insertMulti(d, kv):
k, v = kv
d.setdefault(k, []).append(v)
return d
return reduce(insertMulti, [{}] + items)
should create a dict from key to a list of values:
In [28]: toMultiDict(zip([1,2,1], [4,5,6]))
Out[28]: {1: [4, 6], 2: [5]}
I couldn't put insertMulti into a lambda, because the lambda needs to return the dict again.