I want to map numbers to list in redis
I currently have: r.set(r_id, [d_id,l_id,in_state], ex=10)
But it says this while running: Invalid input of type: 'list'. Convert to a bytes, string, int or float first.
Here I want to map: r_id -> [d_id,l_id,in_state]
how do i do this?
As you notice, SET doesn't accept list as value, but only simple values.
To work with lists, you should use the list-based commands.
Specifically, to map a key to a list, you can use the RPUSH command (note that it appends elements if the list already exists).
You haven't mentioned what client you use. Assuming it's redis-py, you would do:
r.rpush(r_id, d_id, l_id, in_state)
See documentation here.
You can find more information about Redis lists here.
Related
I need a list of all objects with same property/type. Is there a way or an API(just like waitForObject) where I give the type/property and get list of all objects.
I am using Python script for Squish.
You can findAllObjects(objectName):
This function finds and returns a list of object references identified by the symbolic or real (multi-property) name objectName. Because it can return multiple object references, the name must not contain the occurrence property. The given name can also be a native script dictionary (resp. hash) value.
An other (more complicated but more generic) way might be using the object.children() to the container of the objects you are looking for. The list of children needs to be filtered afterwards.
I am trying to filter some data using the Python Core API, which is through Apache Spark, but I am coming into this error, and I am unable to solve it in terms of the data I have:
TypeError: tuple indices must be integers or slices, not str
Now, this is a sample of my data structure:
This is the code I am using to filter my data, but it keeps giving me that error. I am simply trying to return the business_id, city and stars from my dataset.
(my_rdd
.filter(lambda x: x['city']=='Toronto')
.map(lambda x: (x['business_id'], x['city'], x['stars']))
).take(5)
Any guidance on how to filter my data would be helpful.
Thanks.
Sinc your data is nested in tuples, you need to specify the tuple indices in your filter and map:
result = (my_rdd
.filter(lambda x: x[1][1]['city']=='Toronto')
.map(lambda x: (x[1][1]['business_id'], x[1][1]['city'], x[1][1]['stars']))
)
print(result.collect())
[('7v91woy8IpLrqXsRvxj_vw', 'Toronto', 3.0)]
I think you are mistaking in the use of filter and map here. Both of them are used to update lists, and returns lists.
Both of them take a function as parameter (that's the case in the object version, you can also find a functional version which takes the input list as second parameter) and apply it on each item of the input list to build the output list. What differs though is their usage of the function:
filter uses it to, well, filter the input list. The function should return a boolean which indicates whether or not to include the item in the output list.
map uses it to build a new list of the same length as the old one, but with values updated using the provided function.
Now that being said, I believe you have the error TypeError: tuple indices must be integers or slices, not str when you try to filter the list.
On the first loop, the filter function will try to run the function against the first element of the list. This first element is the tuple ('7v91woy8IpLrqXsRvxj_vw', (({'average_stars': 3.41, 'compliment_cool': 9, ...}))). The problem is that you are trying to access a value of this tuple using a string, as if it was a dictionary, which is not permitted in python (and doesn't make much sense).
To extract the data you need, I would go with something much more simple:
item = my_rdd[0]
(item[1][1]['business_id'], item[1][1]['city'], item[1][1]['stars'])
Is the program below guaranteed to always produce the same output?
s = 'fgvhlsdagfcisdghfjkfdshfsal'
for c in s:
print(c)
Yes, it is. This is because the str type is an immutable sequence. Sequences represent a finite ordered set of elements (see Sequences in the Data model chapter of the Reference guide).
Iteration through a given string (any Sequence) is guaranteed to always produce the same results in the same order for different runs of the CPython interpreter, versions of CPython and implementations of Python.
Yes. Internally the string you have there is stored in an c style array (depending on interpreter implementation), being a sequential array of data, one can create an iterator. In order to use for ... in ... syntax, you need to be able to iterate over the object after the in. A string supplies its own iterator which allows it to be parsed via for in syntax in sequential order as do all python sequences.
The same is true for lists, and even custom objects that you create. However not all iterable python objects will necessarily be in order or represent the values they store, a clear example of this is the dictionary. Dictionary iteration yields keys which may or may not be in the order you added them in (depending on the version of python you use among other things, so don't assume its ordered unless you use OrderedDict) instead of sequential values like list tuple and string.
Yes, it is. Over a string, a for-loop iterates over the characters in order. This is also true for lists and tuples -- a for-loop will iterate over the elements in order.
You may be thinking of sets and dictionaries. These don't specify a particular order, so:
for x in {"a","b","c"}: # over a set
print(x)
for key in {"x":1, "y":2, "z":3}: # over a dict
print(key)
will iterate in some arbitrary order that you can't easily predict in advance.
See this Stack Overflow answer for some additional information on what guarantees are made about the order for dictionaries and sets.
Yes. The for loop is sequential.
Yes, the loop will always print each letter one by one starting from the first character and ending with the last.
I want to use BDB as a time-series data store, and planning to use the microseconds since epoch as the key values. I am using BTREE as the data store type.
However, when I try to store integer keys, bsddb3 gives an error saying TypeError: Integer keys only allowed for Recno and Queue DB's.
What is the best workaround? I can store them as strings, but that probably will make it unnecessarily slower.
Given BDB itself can handle any kind of data, why is there a restriction? can I sorta hack the bsddb3 implementation? has anyone used anyother methods?
You can't store integers since bsddb doesn't know how to represent integers and which kind of integer it is.
If you convert your integer to a string you will break the lexicographic ordering of keys of bsddb: 10 > 2 but as strings "10" < "2".
You have to use python struct to convert your integers into a string (or in python 3 into bytes) to store then store them in bsddb. You have to use bigendian packing or ordering will not be correct.
Then you can use bsddb's Cursor.set_range(key) to query for information in a given slice of time.
For instance, Cursor.set_range(struct.unpack('>Q', 123456789)) will set the cursor at the key of the even happening at 123456789 or the first that happens after.
Well, there's no workaround. But you can use two approaches
Store the integers as string using str or repr. If the ints are big, you can even use string formatting
use cPickle/pickle module to store and retrieve data. This is a good way if you have data types other than basic types. For basics ints and floats this actually is slower and takes more space than just storing strings
I want to use a tuple (1,2,3) as a key using the shelve module in Python. I can do this with dictionaries:
d = {}
d[(1,2,3)] = 4
But if i try it with shelve:
s = shelve.open('myshelf')
s[(1,2,3)] = 4
I get: "TypeError: String or Integer object expected for key, tuple found"
Any suggestions?
How about using the repr() of the tuple:
s[repr((1,2,3))] = 4
As per the docs,
the values (not the keys!) in a shelf
can be essentially arbitrary Python
objects
My emphasis: shelf keys must be strings, period. So, you need to turn your tuple into a str; depending on what you'll have in the tuple, repr, some separator.join, pickling, marshaling, etc, may be fruitfully employed for that purpose.
Why not stick with dictionaries if you want to have arbitray keys ? The other option is to build a wrapper class around your tuple with a repr or str method to change it to a string. I am thinking of a library(natural response to shelves) - your tuple can be the elements in the Dewey decimal and the str creates a concatenated complete representation.
This may be an old question, but I had the same problem.
I often use shelve's and frequently want to use non-string keys.
I subclassed shelve-modules class into a shelf which automatically converts non-string-keys into string-keys and returns them in original form when queried.
It works well for Python's standard immutable objects: int, float, string, tuple, boolean.
It can be found in: https://github.com/North-Guard/simple_shelve