Saving files in specific folders using a for loop - python

I am trying to save various excel files that are produced in the code below to a specific folder, but keep getting "SyntaxError: invalid syntax".
#df = some random data
dfinvest = df[df['INVEST'] >= 0.1]
groupedinvest = dfinvest.groupby("SEDOL")
keysI = groupedinvest.groups.keys()
save_path = "C:/Users/Documents/Python Scripts/"
for key in keysI:
splitdf = groupedinvest.get_group(key)
splitdf.to_excel(os.path.join(save_path((str(key)+ str(datetime.datetime.now().strftime(" %d-%m-%Y - Invest") )+ ".xlsx"), engine='xlsxwriter')))
Any help or pointing in the right direction would be much appreciated.
Error:
TypeError Traceback (most recent call last)
<ipython-input-48-5616ac4f50b7> in <module>
1 for key in keysI: #looping through each key
2 splitdf = groupedinvest.get_group(key) # creating a temporary dataframe with only the values of the current key.
----> 3 splitdf.to_excel(os.path.join(save_path((str(key)+ str(datetime.datetime.now().strftime(" %d-%m-%Y - Invest") )+ ".xlsx"), engine='xlsxwriter')))
TypeError: 'str' object is not callable

Related

TypeError : 'int' object is not iterable

I'm trying to preprocess video frames for anomaly event detection. The model is already trained but I can't figure out the issue with the following code (I'm a beginner). Please help with this respected developers.
def Fit_Preprocessing(path: object, frames_ext: object) -> object:
if frames_ext is None:
raise TypeError(
'Invalid Value for argument `frames_ext`, it cannot be None. Give proper extensions of the frames e.g: `.tif` or `.png` etc!')
print('\n\nProcessing Images in this Dataset Path: {0}\n'.format(path))
file_names: List[Union[str, List[str]]]
onlyfiles, file_names, dirs = ReadFileNames(path, frames_ext)
img_list = [1, 2, 3, 4]
for img in tqdm(range(len(onlyfiles))):
images = onlyfiles[img]
count = 0
for images in onlyfiles[img]:
img.split('/')
img_name = dirs[i] + '_' + file_names[i][count]
write_path = 'ProcessedImages/' + path.split('/')[1]
gray = ProcessImg(img_name, read_path=img, write=True,
write_path=write_path, res_shape=(227, 227))
img_list.append(gray)
count += 1
return img_list
Getting this error:
Processing Images in this Dataset Path: C:/Users/Public/Downloads/Surveillance with deep learning/Datasets/UCSD_Anomaly_Dataset.v1p2/UCSDped1/Test
0%| | 0/47 [00:00<?, ?it/s]
Traceback (most recent call last):
File "C:/Users/Public/Downloads/Surveillance-with-deep-learning/preprocessing.py", line 171, in <module>
img_list: object = Fit_Preprocessing(path, frames_ext='.Fit')
File "C:/Users/Public/Downloads/Surveillance-with-deep-learning/preprocessing.py", line 154, in Fit_Preprocessing
for images in onlyfiles[img]:
TypeError: 'int' object is not iterable
Process finished with exit code 1
I tried using images = img_list to fix the loop but it did not work
The reason you are getting an error is because in the for loop:
for images in onlyfiles[img]:
You are getting the value using the index to access it, and onlyfiles[img] will, according to your code, return an int value. Since I don't know what you are clearly doing, my suggestion is to turn onlyfiles[img] into a list, so it iterates only through one thing or more, using:
for images in [onlyfiles[img]]:
Example:
my_int = 123
for i in my_int:
print(i)
Gets the error:
Traceback (most recent call last):
File "main.py", line 2, in <module>
for i in my_int:
TypeError: 'int' object is not iterable
So if you turn it into a list:
my_int = 123
for i in [my_int]:
print(i)
Gives:
123
Or if you want to iterate through the digits, turn it into a string:
my_int = 123
for i in str(my_int):
print(i)
Gives:
1
2
3

How to merge chosen text file that saved in google drive and saved as a single csv file in python?

My data are stored in different directories on google drive. I want to extract one certain text file from each directory and store them as a single csv file. The csv file called model keeps all the different file names that I need to get. And this is the only part I need to change for searching the files that are qualified.
To be more specific:the model csv file contains the following :['ENS','ENS_hr','ENS_lr','MM5','MM5G','MPAS25','NMM3GFS','NMM3NAM','WRF2GFS','WRF2GFS81','WRF2NAM','WRF2NAM81','WRF3ARPEGE','WRF3GEM','WRF3GFS','WRF3GFSgc01','WRF3NAM','WRF3NAVGEM','WRF4ICON','WRF4NAM','WRF4RDPS']
here is my code:
md = []
model = pd.read_csv(verification_path + 'model_name.csv')
#find file for the correct model
for m in model.iterrows():
model_file = verification_path + m +'/MAE_monthly_APCP6_60hr_small.txt'
new = pd.read_csv(model_file)
md.append(new)
But I got the error shows:
TypeError Traceback (most recent call last)
<ipython-input-5-981115533dbc> in <module>
6 #find file for the correct model
7 for m in model.iterrows():
----> 8 model_file = verification_path + m +'/MAE_monthly_APCP6_60hr_small.txt'
9 new = pd.read_csv(model_file)
10 md.append(new)
TypeError: can only concatenate str (not "tuple") to str
Does anyone have any idea how to solve it? Is there another better way?
Thx!
I tried to convert the tuple by the following code and got the new error:
The code for converting tuple:
import functools
import operator
def convertTuple(tup):
str = functools.reduce(operator.add, (tup))
return str
The updated code:
for m in model.iterrows():
model_file = verification_path + convertTuple(m) +'/MAE_monthly_APCP6_60hr_small.txt'
new = pd.read_csv(model_file)
md.append(new)
The error message:
---------------------------------------------------------------------------
TypeError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/pandas/core/ops/array_ops.py in _na_arithmetic_op(left, right, op, is_cmp)
165 try:
--> 166 result = func(left, right)
167 except TypeError:
12 frames
TypeError: unsupported operand type(s) for +: 'int' and 'str'
During handling of the above exception, another exception occurred:
TypeError Traceback (most recent call last)
/usr/local/lib/python3.7/dist-packages/pandas/core/roperator.py in radd(left, right)
7
8 def radd(left, right):
----> 9 return right + left
10
11
TypeError: unsupported operand type(s) for +: 'int' and 'str'

Traceback and TypeError on Python import of excel

I am getting the following error on the code below :
Traceback (most recent call last):
File "D:/Personal Files/Technical Development/PycharmProjects/Call Center Headcount Model/Call Center Headcount Model.py", line 12, in
historical_start_date = work_rules.iloc(4, 2)
TypeError: call() takes from 1 to 2 positional arguments but 3 were given
I am trying to assign a cell value in excel to a variable in Python through pandas. i.e. Historical_start_date = work_rules.loc(4,2)
Any idea why this is?
Code:
work_rules = pd.read_excel(
'D:\\Personal Files\\Technical Development\\PycharmProjects\\Call Center Headcount Model\\Call Center Work Rules.xlsx',
sheet_name='Inputs')
historical_start_date = work_rules.iloc(4, 2)
print(historical_start_date)
Please refer to the Iloc documentation. You're supplying variables as if it is a method. I'm not sure what you want to do exactly but it seems to me that you need to replace the parenthesis after iloc with brackets.

'numpy.int32' object has no attribute 'apply'

Trying to extract numeric part from column-dtype('O') and have the extracted data in a new column or replace the existing column.
In this process the apply method didnt work for the code below. Provided the traceback also
code
def dpr_size(data_size):
split_size = re.split('(\d*.*\d)', data_size)
size = float(split_size[0])
return float(size)
df1['size_new'] = df1.size.astype('str').apply(dpr_size)
Traceback
AttributeError Traceback (most recent call last)
in ()
5 return float(size)
6
----> 7 df1['size_new'] = df1.size.apply(dpr_size)
AttributeError: 'numpy.int32' object has no attribute 'apply'
Tried many other alternatives, and now the below code works.
code
df1['size'] = df1['size'].astype('str')
for i in range(len(df1['size'])):
split_size = re.split('(\d*.*\d)', df1['size'][i])[1:]
df1['size'][i] = split_size[0]
However want to find out why 'apply' doesnt work
Data

Python error: Valueerror-need-more-than-1-value-to-unpack

In python when I run this code:
lat, lon = f.variables['latitude'], f.variables['longitude']
latvals = lat[:]; lonvals = lon[:]
def getclosest_ij(lats,lons,latpt,lonpt):
dist_sq = (lats-latpt)**2 + (lons-lonpt)**2
minindex_flattened = dist_sq.argmin()
return np.unravel_index(minindex_flattened, lats.shape)
iy_min, ix_min = getclosest_ij(latvals, lonvals, 46.1514, 20.0846)
It get the following error:
ValueError Traceback (most recent call last)
ipython-input-104-3ba92bea5d48 in module()
11 return np.unravel_index(minindex_flattened, lats.shape)
12 iy_min, ix_min = getclosest_ij(latvals, lonvals, 46.1514, 20.0846)
ValueError: need more than 1 value to unpack
What does it mean? How could I fix it?
I would read a NetCDF file, it is consist of total coloumn water data with dimensions: time(124), latitude(15), and longitude(15). I would appropriate the amount of tcw for specific point (lat,lon), and time. I tried to use the code above to solve the first part of my task to evaluate the tcw for specific coorinates, but didn't work.
Thank your help in advance.
in python you can write
var1, var2 = (1, 2) # = iterable with 2 items
that will store 1 in var1 and 2 in var2.
This feature is called unpacking.
So the error your code throws means, that the function getclosest_ij returned one value instead of the 2 values you would need to unpack them into iy_min and ix_min

Categories

Resources