0 is not in range in pandas - python

So I am trying to load my data from jupyter into mysql workbench using the one "multiple row" insert statement. I will achieve that with a for loop and i am receiving some error messages.
First, a little background:
So I had my csv file which contains data set for preprocessing and I split into 2 here:
Before_handwashing=copy_monthly_df.iloc[:76]
After_handwashig=copy_monthly_df.iloc[76:]
I have successfully structured and loaded the first data set Before_handwashing into mysql work bench using this for loop below.
for x in range(Before_handwashing.shape[0]):
insert_query+='('
for y in range(Before_handwashing.shape[1]):
insert_query+= str(Before_handwashing[Before_handwashing.columns.values[y]][x])+', '
insert_query=insert_query[:-2]+'), '
Now I want to structure and load my second part of the dataset which is After_handwashig into mysql workbench using a similar code structure here.
for x in range(After_handwashig.shape[0]):
insert_query+='('
for y in range(After_handwashig.shape[1]):
insert_query+=str(After_handwashig[After_handwashig.columns.values[y]][x])+', '
insert_query=insert_query[:-2]+'), '
And I am recieving the following error messages
error message: ValueError
Traceback (most recent call last)
~\anaconda3\lib\site-packages\pandas\core\indexes\range.py in get_loc(self, key, method, tolerance)
384 try:
--> 385 return self._range.index(new_key)
386 except ValueError as err:
ValueError: 0 is not in range
The above exception was the direct cause of the following exception:
KeyError
Traceback (most recent call last) ~\AppData\Local\Temp/ipykernel_4316/2677185951.py in <module>
2 insert_query+='('
3 for y in range(After_handwashig.shape[1]):
----> 4 insert_query+=str(After_handwashig[After_handwashig.columns.values[y]][x])+', '
5 insert_query=insert_query[:-2]+'), '
~\anaconda3\lib\site-packages\pandas\core\series.py in __getitem__(self, key)
940
941 elif key_is_scalar:
--> 942 return self._get_value(key)
943
944 if is_hashable(key):
~\anaconda3\lib\site-packages\pandas\core\series.py in
_get_value(self, label, takeable)
1049
1050 # Similar to Index.get_value, but we do not fall back to positional
-> 1051 loc = self.index.get_loc(label)
1052 return self.index._get_values_for_loc(self, loc, label)
1053
~\anaconda3\lib\site-packages\pandas\core\indexes\range.py in
get_loc(self, key, method, tolerance)
385 return self._range.index(new_key)
386 except ValueError as err:
--> 387 raise KeyError(key) from err
388 raise KeyError(key)
389 return super().get_loc(key, method=method, tolerance=tolerance)
KeyError: 0
Can someone help me out in answering this problem?


OK, it took me a moment to find this. Consider these statements:
Before_handwashing=copy_monthly_df.iloc[:76]
After_handwashig=copy_monthly_df.iloc[76:]
When these are done, Before contains lines with indexes 0 to 75. After contains lines starting with index 76. There is no line with index 0, so your attempt to access it causes a key error.
There are two solutions. One is to use iloc to reference lines by ordinal instead of by index:
insert_query+=str(After_handwashig[After_handwashig.columns.values[y]]./iloc(x))+', '
The other is to reset the indexes to start from 0:
After_handwashing.reset_index(drop=True, inplace=True)
There's really no point in splitting the dataframe like that. Just have your first loop do range(76): and the second do range(76,copy_monthly_dy.shape[1]):.

Related

I keep getting the error message ValueError: Wrong number of items passed 2, placement implies 1 [duplicate]

I am receiving the error:
ValueError: Wrong number of items passed 3, placement implies 1, and I am struggling to figure out where, and how I may begin addressing the problem.
I don't really understand the meaning of the error; which is making it difficult for me to troubleshoot. I have also included the block of code that is triggering the error in my Jupyter Notebook.
The data is tough to attach; so I am not looking for anyone to try and re-create this error for me. I am just looking for some feedback on how I could address this error.
KeyError Traceback (most recent call last)
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\indexes\base.py in get_loc(self, key, method, tolerance)
1944 try:
-> 1945 return self._engine.get_loc(key)
1946 except KeyError:
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4154)()
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4018)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12368)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12322)()
KeyError: 'predictedY'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in set(self, item, value, check)
3414 try:
-> 3415 loc = self.items.get_loc(item)
3416 except KeyError:
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\indexes\base.py in get_loc(self, key, method, tolerance)
1946 except KeyError:
-> 1947 return self._engine.get_loc(self._maybe_cast_indexer(key))
1948
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4154)()
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4018)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12368)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12322)()
KeyError: 'predictedY'
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-95-476dc59cd7fa> in <module>()
26 return gp, results
27
---> 28 gp_dailyElectricity, results_dailyElectricity = predictAll(3, 0.04, trainX_dailyElectricity, trainY_dailyElectricity, testX_dailyElectricity, testY_dailyElectricity, testSet_dailyElectricity, 'Daily Electricity')
<ipython-input-95-476dc59cd7fa> in predictAll(theta, nugget, trainX, trainY, testX, testY, testSet, title)
8
9 results = testSet.copy()
---> 10 results['predictedY'] = predictedY
11 results['sigma'] = sigma
12
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\frame.py in __setitem__(self, key, value)
2355 else:
2356 # set column
-> 2357 self._set_item(key, value)
2358
2359 def _setitem_slice(self, key, value):
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\frame.py in _set_item(self, key, value)
2422 self._ensure_valid_index(value)
2423 value = self._sanitize_column(key, value)
-> 2424 NDFrame._set_item(self, key, value)
2425
2426 # check if we are modifying a copy
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\generic.py in _set_item(self, key, value)
1462
1463 def _set_item(self, key, value):
-> 1464 self._data.set(key, value)
1465 self._clear_item_cache()
1466
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in set(self, item, value, check)
3416 except KeyError:
3417 # This item wasn't present, just insert at end
-> 3418 self.insert(len(self.items), item, value)
3419 return
3420
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in insert(self, loc, item, value, allow_duplicates)
3517
3518 block = make_block(values=value, ndim=self.ndim,
-> 3519 placement=slice(loc, loc + 1))
3520
3521 for blkno, count in _fast_count_smallints(self._blknos[loc:]):
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in make_block(values, placement, klass, ndim, dtype, fastpath)
2516 placement=placement, dtype=dtype)
2517
-> 2518 return klass(values, ndim=ndim, fastpath=fastpath, placement=placement)
2519
2520 # TODO: flexible with index=None and/or items=None
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in __init__(self, values, placement, ndim, fastpath)
88 raise ValueError('Wrong number of items passed %d, placement '
89 'implies %d' % (len(self.values),
---> 90 len(self.mgr_locs)))
91
92 #property
ValueError: Wrong number of items passed 3, placement implies 1
My code is as follows:
def predictAll(theta, nugget, trainX, trainY, testX, testY, testSet, title):
gp = gaussian_process.GaussianProcess(theta0=theta, nugget =nugget)
gp.fit(trainX, trainY)
predictedY, MSE = gp.predict(testX, eval_MSE = True)
sigma = np.sqrt(MSE)
results = testSet.copy()
results['predictedY'] = predictedY
results['sigma'] = sigma
print ("Train score R2:", gp.score(trainX, trainY))
print ("Test score R2:", sklearn.metrics.r2_score(testY, predictedY))
plt.figure(figsize = (9,8))
plt.scatter(testY, predictedY)
plt.plot([min(testY), max(testY)], [min(testY), max(testY)], 'r')
plt.xlim([min(testY), max(testY)])
plt.ylim([min(testY), max(testY)])
plt.title('Predicted vs. observed: ' + title)
plt.xlabel('Observed')
plt.ylabel('Predicted')
plt.show()
return gp, results
gp_dailyElectricity, results_dailyElectricity = predictAll(3, 0.04, trainX_dailyElectricity, trainY_dailyElectricity, testX_dailyElectricity, testY_dailyElectricity, testSet_dailyElectricity, 'Daily Electricity')

In general, the error ValueError: Wrong number of items passed 3, placement implies 1 suggests that you are attempting to put too many pigeons in too few pigeonholes. In this case, the value on the right of the equation
results['predictedY'] = predictedY
is trying to put 3 "things" into a container that allows only one. Because the left side is a dataframe column, and can accept multiple items on that (column) dimension, you should see that there are too many items on another dimension.
Here, it appears you are using sklearn for modeling, which is where gaussian_process.GaussianProcess() is coming from (I'm guessing, but correct me and revise the question if this is wrong).
Now, you generate predicted values for y here:
predictedY, MSE = gp.predict(testX, eval_MSE = True)
However, as we can see from the documentation for GaussianProcess, predict() returns two items. The first is y, which is array-like (emphasis mine). That means that it can have more than one dimension, or, to be concrete for thick headed people like me, it can have more than one column -- see that it can return (n_samples, n_targets) which, depending on testX, could be (1000, 3) (just to pick numbers). Thus, your predictedY might have 3 columns.
If so, when you try to put something with three "columns" into a single dataframe column, you are passing 3 items where only 1 would fit.

Not sure if this is relevant to your question but it might be relevant to someone else in the future: I had a similar error. Turned out that the df was empty (had zero rows) and that is what was causing the error in my command.

Another cause of this error is when you apply a function on a DataFrame where there are two columns with the same name.

Starting with pandas 1.3.x it's not allowed to fill objects (e.g. like an eagertensor from an embedding) into columns.
https://github.com/pandas-dev/pandas/blame/master/pandas/core/internals/blocks.py

So ValueError: The wrong number of items passed 3, placement implies 1 occurs when you're passing to many arguments but method supports only a few. for example -
df['First_Name', 'Last_Name'] = df['Full_col'].str.split(' ', expand = True)
In the above code, I'm trying to split Full_col into two sub-columns names as -First_Name & Last_Name, so here I'll get the error because instead list of columns the columns I'm passing only a single argument.
So to avoid this - use another sub-list
df[['First_Name', 'Last_Name']] = df['Full_col'].str.split(' ', expand = True)

Just adding this as an answer: nesting methods and misplacing closed brackets will also throw this error, ex:
march15_totals= march15_t.assign(sum_march15_t=march15_t[{"2021-03-15","2021-03-16","2021-03-17","2021-03-18","2021-03-19","2021-03-20","2021-03-21"}]).sum(axis=1)
Versus the (correct) version:
march15_totals= march15_t.assign(sum_march15_t=march15_t[{"2021-03-15","2021-03-16","2021-03-17","2021-03-18","2021-03-19","2021-03-20","2021-03-21"}].sum(axis=1))
This is probably common sense to most of you but I was quite puzzled until I realized my mistake.

I got this error when I was trying to convert a one-column dataframe, df, into a Series, pd.Series(df).
I resolved this with
pd.Series(df.values.flatten())
The problem was that the values in the dataframe were lists:
my_col
0 ['a']
1 ['b']
2 ['c']
3 ['d']
When I was printing the dataframe it wasn't showing the brackets which made it hard to track down.

for i in range(100):
try:
#Your code here
break
except:
continue
This one worked for me.

Error while one hot encoding a column of list of strings

I have a dataframe that contains information about cuisines and their respective ingredients. The ingredients are stored in a column with type of list of strings, ['ingredients'], as shown in the image below:
I tried to one hot encode each ingredient so I used the answer in this post for reference.
However, I got an error message shown below:
code:
train_w_stemming_df = pd.DataFrame(mlb.fit_transform(train_w_stemming_df['ingredients']),
columns=mlb.classes_,
index = train_w_stemming_df.index)
error message:
ValueError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/pandas/core/internals/managers.py in create_block_manager_from_blocks(blocks, axes)
1670 blocks = [
-> 1671 make_block(values=blocks[0], placement=slice(0, len(axes[0])))
1672 ]
6 frames
ValueError: Wrong number of items passed 1, placement implies 43
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/pandas/core/internals/managers.py in create_block_manager_from_blocks(blocks, axes)
1679 blocks = [getattr(b, "values", b) for b in blocks]
1680 tot_items = sum(b.shape[0] for b in blocks)
-> 1681 raise construction_error(tot_items, blocks[0].shape[1:], axes, e)
1682
1683
ValueError: Shape of passed values is (29774, 1), indices imply (29774, 43)
How can I fix this error?

KeyError when selecting and modifying an element in Dataframe

I have 2 lists that I am iterating over, english_tweets_2 and truncated_trigrams_list.
english_tweets_2 contains tweets, stored as strings.
truncated_trigrams_list contains trigrams, also stored as strings.
I check if a trigram occurs in a tweet. If so, I use the trigram name to go to the corresponding column, and the tweet to go to the corresponding row. Then I increment that single value by 1, and repeat for all other combinations of tweets/trigrams.
# Create new columns, fill with 0 initially
for trigram in truncated_trigrams_list:
tweet_features_en[trigram] = 0
# Increment columns depending on occurrence of trigram in tweet
for tweet in english_tweets_2:
for trigram_name in truncated_trigrams_list:
if trigram_name in tweet:
tweet_features_en.loc[tweet][trigram_name] += 1
This gives me the following error:
---------------------------------------------------------------------------
KeyError Traceback (most recent call last)
~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
2894 try:
-> 2895 return self._engine.get_loc(casted_key)
2896 except KeyError as err:
pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()
pandas\_libs\index.pyx in pandas._libs.index.IndexEngine.get_loc()
pandas\_libs\index_class_helper.pxi in pandas._libs.index.Int64Engine._check_type()
KeyError: 'I love it when professors draw a big question mark next to my answer on an exam because I’m always like yeah I don’t either ¯\\_(?)_/¯'
The above exception was the direct cause of the following exception:
KeyError Traceback (most recent call last)
<ipython-input-123-b400deacdb1b> in <module>
17 for trigram_name in truncated_trigrams_list:
18 if trigram_name in tweet:
---> 19 tweet_features_en.loc[tweet][trigram_name] += 1
~\anaconda3\lib\site-packages\pandas\core\indexing.py in __getitem__(self, key)
877
878 maybe_callable = com.apply_if_callable(key, self.obj)
--> 879 return self._getitem_axis(maybe_callable, axis=axis)
880
881 def _is_scalar_access(self, key: Tuple):
~\anaconda3\lib\site-packages\pandas\core\indexing.py in _getitem_axis(self, key, axis)
1108 # fall thru to straight lookup
1109 self._validate_key(key, axis)
-> 1110 return self._get_label(key, axis=axis)
1111
1112 def _get_slice_axis(self, slice_obj: slice, axis: int):
~\anaconda3\lib\site-packages\pandas\core\indexing.py in _get_label(self, label, axis)
1057 def _get_label(self, label, axis: int):
1058 # GH#5667 this will fail if the label is not present in the axis.
-> 1059 return self.obj.xs(label, axis=axis)
1060
1061 def _handle_lowerdim_multi_index_axis0(self, tup: Tuple):
~\anaconda3\lib\site-packages\pandas\core\generic.py in xs(self, key, axis, level, drop_level)
3489 loc, new_index = self.index.get_loc_level(key, drop_level=drop_level)
3490 else:
-> 3491 loc = self.index.get_loc(key)
3492
3493 if isinstance(loc, np.ndarray):
~\anaconda3\lib\site-packages\pandas\core\indexes\base.py in get_loc(self, key, method, tolerance)
2895 return self._engine.get_loc(casted_key)
2896 except KeyError as err:
-> 2897 raise KeyError(key) from err
2898
2899 if tolerance is not None:
KeyError: 'I love it when professors draw a big question mark next to my answer on an exam because I’m always like yeah I don’t either ¯\\_(?)_/¯'
The string in the keyerror 'I love it when professors draw a big question mark next to my answer on an exam because I’m always like yeah I don’t either ¯\\_(?)_/¯' is one of the entries in the english_tweets_2 list.
How do I get around this error? It is likely that my syntax is wrong, would love some help. Thank you!

I do not know the structure of the dataframe, but I will assume that it is worth referring to the index with iloc:
for index, tweet in enumerate(english_tweets_2):
for trigram_name in truncated_trigrams_list:
if trigram_name in tweet:
tweet_features_en.iloc[index][trigram_name] += 1

ValueError: Wrong number of items passed - Meaning and suggestions?

I am receiving the error:
ValueError: Wrong number of items passed 3, placement implies 1, and I am struggling to figure out where, and how I may begin addressing the problem.
I don't really understand the meaning of the error; which is making it difficult for me to troubleshoot. I have also included the block of code that is triggering the error in my Jupyter Notebook.
The data is tough to attach; so I am not looking for anyone to try and re-create this error for me. I am just looking for some feedback on how I could address this error.
KeyError Traceback (most recent call last)
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\indexes\base.py in get_loc(self, key, method, tolerance)
1944 try:
-> 1945 return self._engine.get_loc(key)
1946 except KeyError:
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4154)()
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4018)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12368)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12322)()
KeyError: 'predictedY'
During handling of the above exception, another exception occurred:
KeyError Traceback (most recent call last)
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in set(self, item, value, check)
3414 try:
-> 3415 loc = self.items.get_loc(item)
3416 except KeyError:
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\indexes\base.py in get_loc(self, key, method, tolerance)
1946 except KeyError:
-> 1947 return self._engine.get_loc(self._maybe_cast_indexer(key))
1948
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4154)()
pandas\index.pyx in pandas.index.IndexEngine.get_loc (pandas\index.c:4018)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12368)()
pandas\hashtable.pyx in pandas.hashtable.PyObjectHashTable.get_item (pandas\hashtable.c:12322)()
KeyError: 'predictedY'
During handling of the above exception, another exception occurred:
ValueError Traceback (most recent call last)
<ipython-input-95-476dc59cd7fa> in <module>()
26 return gp, results
27
---> 28 gp_dailyElectricity, results_dailyElectricity = predictAll(3, 0.04, trainX_dailyElectricity, trainY_dailyElectricity, testX_dailyElectricity, testY_dailyElectricity, testSet_dailyElectricity, 'Daily Electricity')
<ipython-input-95-476dc59cd7fa> in predictAll(theta, nugget, trainX, trainY, testX, testY, testSet, title)
8
9 results = testSet.copy()
---> 10 results['predictedY'] = predictedY
11 results['sigma'] = sigma
12
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\frame.py in __setitem__(self, key, value)
2355 else:
2356 # set column
-> 2357 self._set_item(key, value)
2358
2359 def _setitem_slice(self, key, value):
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\frame.py in _set_item(self, key, value)
2422 self._ensure_valid_index(value)
2423 value = self._sanitize_column(key, value)
-> 2424 NDFrame._set_item(self, key, value)
2425
2426 # check if we are modifying a copy
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\generic.py in _set_item(self, key, value)
1462
1463 def _set_item(self, key, value):
-> 1464 self._data.set(key, value)
1465 self._clear_item_cache()
1466
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in set(self, item, value, check)
3416 except KeyError:
3417 # This item wasn't present, just insert at end
-> 3418 self.insert(len(self.items), item, value)
3419 return
3420
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in insert(self, loc, item, value, allow_duplicates)
3517
3518 block = make_block(values=value, ndim=self.ndim,
-> 3519 placement=slice(loc, loc + 1))
3520
3521 for blkno, count in _fast_count_smallints(self._blknos[loc:]):
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in make_block(values, placement, klass, ndim, dtype, fastpath)
2516 placement=placement, dtype=dtype)
2517
-> 2518 return klass(values, ndim=ndim, fastpath=fastpath, placement=placement)
2519
2520 # TODO: flexible with index=None and/or items=None
C:\Users\brennn1\AppData\Local\Continuum\Anaconda3\lib\site-packages\pandas\core\internals.py in __init__(self, values, placement, ndim, fastpath)
88 raise ValueError('Wrong number of items passed %d, placement '
89 'implies %d' % (len(self.values),
---> 90 len(self.mgr_locs)))
91
92 #property
ValueError: Wrong number of items passed 3, placement implies 1
My code is as follows:
def predictAll(theta, nugget, trainX, trainY, testX, testY, testSet, title):
gp = gaussian_process.GaussianProcess(theta0=theta, nugget =nugget)
gp.fit(trainX, trainY)
predictedY, MSE = gp.predict(testX, eval_MSE = True)
sigma = np.sqrt(MSE)
results = testSet.copy()
results['predictedY'] = predictedY
results['sigma'] = sigma
print ("Train score R2:", gp.score(trainX, trainY))
print ("Test score R2:", sklearn.metrics.r2_score(testY, predictedY))
plt.figure(figsize = (9,8))
plt.scatter(testY, predictedY)
plt.plot([min(testY), max(testY)], [min(testY), max(testY)], 'r')
plt.xlim([min(testY), max(testY)])
plt.ylim([min(testY), max(testY)])
plt.title('Predicted vs. observed: ' + title)
plt.xlabel('Observed')
plt.ylabel('Predicted')
plt.show()
return gp, results
gp_dailyElectricity, results_dailyElectricity = predictAll(3, 0.04, trainX_dailyElectricity, trainY_dailyElectricity, testX_dailyElectricity, testY_dailyElectricity, testSet_dailyElectricity, 'Daily Electricity')

In general, the error ValueError: Wrong number of items passed 3, placement implies 1 suggests that you are attempting to put too many pigeons in too few pigeonholes. In this case, the value on the right of the equation
results['predictedY'] = predictedY
is trying to put 3 "things" into a container that allows only one. Because the left side is a dataframe column, and can accept multiple items on that (column) dimension, you should see that there are too many items on another dimension.
Here, it appears you are using sklearn for modeling, which is where gaussian_process.GaussianProcess() is coming from (I'm guessing, but correct me and revise the question if this is wrong).
Now, you generate predicted values for y here:
predictedY, MSE = gp.predict(testX, eval_MSE = True)
However, as we can see from the documentation for GaussianProcess, predict() returns two items. The first is y, which is array-like (emphasis mine). That means that it can have more than one dimension, or, to be concrete for thick headed people like me, it can have more than one column -- see that it can return (n_samples, n_targets) which, depending on testX, could be (1000, 3) (just to pick numbers). Thus, your predictedY might have 3 columns.
If so, when you try to put something with three "columns" into a single dataframe column, you are passing 3 items where only 1 would fit.

Not sure if this is relevant to your question but it might be relevant to someone else in the future: I had a similar error. Turned out that the df was empty (had zero rows) and that is what was causing the error in my command.

Another cause of this error is when you apply a function on a DataFrame where there are two columns with the same name.

Starting with pandas 1.3.x it's not allowed to fill objects (e.g. like an eagertensor from an embedding) into columns.
https://github.com/pandas-dev/pandas/blame/master/pandas/core/internals/blocks.py

So ValueError: The wrong number of items passed 3, placement implies 1 occurs when you're passing to many arguments but method supports only a few. for example -
df['First_Name', 'Last_Name'] = df['Full_col'].str.split(' ', expand = True)
In the above code, I'm trying to split Full_col into two sub-columns names as -First_Name & Last_Name, so here I'll get the error because instead list of columns the columns I'm passing only a single argument.
So to avoid this - use another sub-list
df[['First_Name', 'Last_Name']] = df['Full_col'].str.split(' ', expand = True)

Just adding this as an answer: nesting methods and misplacing closed brackets will also throw this error, ex:
march15_totals= march15_t.assign(sum_march15_t=march15_t[{"2021-03-15","2021-03-16","2021-03-17","2021-03-18","2021-03-19","2021-03-20","2021-03-21"}]).sum(axis=1)
Versus the (correct) version:
march15_totals= march15_t.assign(sum_march15_t=march15_t[{"2021-03-15","2021-03-16","2021-03-17","2021-03-18","2021-03-19","2021-03-20","2021-03-21"}].sum(axis=1))
This is probably common sense to most of you but I was quite puzzled until I realized my mistake.

I got this error when I was trying to convert a one-column dataframe, df, into a Series, pd.Series(df).
I resolved this with
pd.Series(df.values.flatten())
The problem was that the values in the dataframe were lists:
my_col
0 ['a']
1 ['b']
2 ['c']
3 ['d']
When I was printing the dataframe it wasn't showing the brackets which made it hard to track down.

for i in range(100):
try:
#Your code here
break
except:
continue
This one worked for me.

using pool.map to apply function to list of strings in parallel?

I have a large list of http user agent strings (taken from a pandas dataframe) that I am trying to parse using the python implementation of ua-parser. I can parse the list fine when only using a single thread, but based on some preliminary speed testing, it'd take me well over 10 hours to run the whole dataset.
I am trying to use pool.map() to decrease processing time but can't quite seem to figure out how to get it to work. I've read about a dozen 'tutorials' that I found online and have searched SO (likely a duplicate of some sort, as there are a lot of similar questions), but none of the dozens of attempts have worked for one reason or another. I'm assuming/hoping it's an easy fix.
Here is what I have so far:
from ua_parser import user_agent_parser
http_str = df['user_agents'].tolist()
def uaparse(http_str):
for i, item in enumerate(http_str):
return user_agent_parser.Parse(http_str[i])
pool = mp.Pool(processes=10)
parsed = pool.map(uaparse, range(0,len(http_str))
Right now I'm seeing the following error message:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
<ipython-input-25-701fbf58d263> in <module>()
7
8 pool = mp.Pool(processes=10)
----> 9 results = pool.map(uaparse, range(0,len(http_str)))
/home/ubuntu/anaconda/lib/python2.7/multiprocessing/pool.pyc in map(self, func, iterable, chunksize)
249 '''
250 assert self._state == RUN
--> 251 return self.map_async(func, iterable, chunksize).get()
252
253 def imap(self, func, iterable, chunksize=1):
/home/ubuntu/anaconda/lib/python2.7/multiprocessing/pool.pyc in get(self, timeout)
565 return self._value
566 else:
--> 567 raise self._value
568
569 def _set(self, i, obj):
TypeError: 'int' object is not iterable
Thanks in advance for any assistance/direction you can provide.

It seems like all you need is:
http_str = df['user_agents'].tolist()
pool = mp.Pool(processes=10)
parsed = pool.map(user_agent_parser.Parse, http_str)

Categories

Resources