Develop Reference

TypeError: tuple indices must be integers or slices, not str using Python Core API?

I am trying to filter some data using the Python Core API, which is through Apache Spark, but I am coming into this error, and I am unable to solve it in terms of the data I have:
TypeError: tuple indices must be integers or slices, not str
Now, this is a sample of my data structure:
This is the code I am using to filter my data, but it keeps giving me that error. I am simply trying to return the business_id, city and stars from my dataset.
(my_rdd
.filter(lambda x: x['city']=='Toronto')
.map(lambda x: (x['business_id'], x['city'], x['stars']))
).take(5)
Any guidance on how to filter my data would be helpful.
Thanks.

Sinc your data is nested in tuples, you need to specify the tuple indices in your filter and map:
result = (my_rdd
.filter(lambda x: x[1][1]['city']=='Toronto')
.map(lambda x: (x[1][1]['business_id'], x[1][1]['city'], x[1][1]['stars']))
)
print(result.collect())
[('7v91woy8IpLrqXsRvxj_vw', 'Toronto', 3.0)]

I think you are mistaking in the use of filter and map here. Both of them are used to update lists, and returns lists.
Both of them take a function as parameter (that's the case in the object version, you can also find a functional version which takes the input list as second parameter) and apply it on each item of the input list to build the output list. What differs though is their usage of the function:
filter uses it to, well, filter the input list. The function should return a boolean which indicates whether or not to include the item in the output list.
map uses it to build a new list of the same length as the old one, but with values updated using the provided function.
Now that being said, I believe you have the error TypeError: tuple indices must be integers or slices, not str when you try to filter the list.
On the first loop, the filter function will try to run the function against the first element of the list. This first element is the tuple ('7v91woy8IpLrqXsRvxj_vw', (({'average_stars': 3.41, 'compliment_cool': 9, ...}))). The problem is that you are trying to access a value of this tuple using a string, as if it was a dictionary, which is not permitted in python (and doesn't make much sense).
To extract the data you need, I would go with something much more simple:
item = my_rdd[0]
(item[1][1]['business_id'], item[1][1]['city'], item[1][1]['stars'])

trying add two numbers ,by reading them from list and print their index no. from their actual list

Given an array of integers, return indices of the two numbers such that they add up to a specific target.
,in each input would have not use the same element twice.
class solution():
def __init__(self,array_num,target_num):
self.array_num=array_num
self.target_num=target_num
for t in self.array_num:
for b in self.array_num:
e=t+b
w=self.array_num.index(t),self.array_num.index(b)
y=list(w)
if e==self.target_num:
if y==[0,0]:
break
else:
print(y)
p=[3,3]
so=solution(p,6)
output
[] or nothing
expected output
[0,1]

The problem is that you are asking the list to give you the index if a number like this:
self.array_num.index(t)
This will always give you the first occurrence, which is 0 here, although the loop is actually at the second position with index 1.
To avoid that, reverse the logic: create the loop for the index (use len() and range()), then get the number at that position.
As this question sounds like homework or school assignment, I'll not post a full solution. It should be possible to solve the problem now.
More hints to make your teacher happy:
[0, 0] is not the only solution that results in 6. You want to exclude other invalid combinations as well. Pro tip: there's a nice solution that doesn't require any check and will run faster. It's easy to find once you switched the logic.
Currently you do all work in the constructor of the object. Maybe you want a method that does the actual calculation.
Your variable names are not self-explaining. Don't use so many single letter variables.

Matlab cell array to python list conversion

I have Matlab cell array of size (1,260), but I am unable to convert it into a Python list. My code is as follow:
i=sio.loadmat('bispec.mat')
k=i['a']
After executing the code, I get these entities in the variable explorer.
I am trying to convert the Matlab cell array named k to the Python list by the following code.
val = np.asarray(k, dtype='float').tolist()
But I get the following error.

As shown in one of the images, you have tried:
val = np.asarray(k, dtype='double').tolist().
There should not be inverted commas around double. You should be actually using:
val = np.asarray(k, dtype=np.longdouble).tolist().
Adding further, one more reason for this error to occur is trying to use a string as an element in an array type of double. If you really want to have a NumPy array containing both strings and doubles, you could use the dtype object, which enables the array to hold arbitrary Python objects, as shown.
val = np.asarray(k, dtype=object).tolist().

Transforming type Int64Index into an integer index in Python

I'm quite new with python, however, I have to accomplish some assignment and I am struggling now on a problem. I try to get the index of the element in a table A when some other parameter from this table A corresponds to a value in a list B. The table A also already contains a column "index" where all elements are numerated from 0 till the end. Moreover, the values in tableA.parameter1 and listB can coincide only once, multiple matches are not possible. So to derive the necessary index I use a line
t=tableA.index[tableA.parameter1==listB[numberObservation]]
However, what I get as a result is something like:
t Int64Index([2], dtype='int64')
If I use the variable t in this format Int64Index, it doesn't suit for the further code I have to work with. Actually, I need only 2 as an integer number, without all this redundant rest.
Can somebody please help me to circumvent my problem? I am in total despair and would be grateful for any help.

Try .tolist()
t=tableA.index[tableA.parameter1==listB[numberObservation]].tolist()
This should return
t = [2]
a list "without all the redundant rest" :)

What package is giving you Int64Index? This looks vaguely numpy-ish, but numpy arrays define __index__ so a single element array of integer values will seamlessly operate as indices for sequence lookup.
Regardless, assuming t is supposed to be exactly one value, and it's a sequence type itself, you can just do:
t, = tableA.index[tableA.parameter1==listB[numberObservation]]
That trailing comma changes the line from straight assignment to iterable unpacking; it expects the right hand side to produce an iterable with exactly one value, and that one value is unpacked into t. If the iterable has 0 or 2+ values, you'll get a ValueError.

How to access the specific locations of an integer list in Python?

I have an integer list which should be used as indices of another list to retrieve a value. Lets say we have following array
a = [1,2,3,4,5,6,7,8,9]
We can get the specific elements using following code
import operator
operator.itemgetter(1,2,3)(a)
It will return the 2nd, 3rd and 4th item.
Lets say i have another list
b=[1,2,3]
But if I try to run the following code it gets an error
operator.itemgetter(b)(a)
I am wondering if someone could help me please. I think its just the problem that I have to convert the b to comma seprated indices butnot very sure.
Thanks a lot

Use *:
operator.itemgetter(*b)(a)
The * in a function call means, unpack this value, and use its elements as the arguments to the function.

Since you have tagged your question with the numpy tag, you could also consider making a an array so this works:
from numpy import array
a = array([1,2,3,4,5,6,7,8,9])
b = [1,2,3]
a[b]

The first argument to itemgetter needs to be a tuple. You can do this with:
apply(operator.itemgetter, tuple(b))(a)
There might be a cleaner/more idomatic way of doing this, but this does work for your example.

You can also try:
map(a.__getitem__, b)
The code returns a list in Python 2 or an iterator in Python 3. If you need to convert it to tuple, just put it in a tuple().