Develop Reference

python: error reading files into data frame

I'm trying to import multiple csv files in one folder into one data frame. This is my code. It can iterate through the files and print them successfully and it can read one file into a data frame but combining them is printing an error. I saw many questions similar but the responses are complex, I thought the 'pythonic' way is to be simple because I am new to this. Thanks in advance for any help. The error message is always: No such file or directory: 'some file name' which makes no sense because it successfully printed the file name in the print step.
import pandas as pd
# this works
df = pd.read_csv("headlines/2017-1.csv")
print(df)
path = 'C:/.../... /.../headlines/' <--- full path I shortened it here
files = os.listdir(path)
print(files) <-- prints all file names successfully
for filename in files:
print(filename) # <-- successfully prints all file names
df = pd.read_csv(filename) # < -- error here
df2.append(df) # append to data frame

It seems like your current working directory is different from your path. Please use
os.chdir(path) before attempting to read your csv.

Data set file not found in python

I am coding a python program to read a data set file writing this line:
df = pd.read_csv (r'C:\Users\user118\Desktop\StudentsPerformance.csv')
This line works, but I have to upload this project as an assignment , so the computer path must be changed. I think about putting the csv file in the project folder and i did and wrote this line:
df = pd.read_csv ("StudentsPerformance.csv")
but it gave me an error saying that the file isn't found. Where to correctly put the file in the project folder? Or what I should do?

To read your csv file with a df = pd.read_csv ("StudentsPerformance.csv") line you should put it right next to executing .py file.
To do that you can just read your csv file using full path like this:
df = pd.read_csv (r'C:\Users\user118\Desktop\StudentsPerformance.csv')
and than write df.to_csv('StudentsPerformance.csv')
After that you would be able to read your csv as you wanted using
df = pd.read_csv('StudentsPerformance.csv')

seaching all excel files with os.walk (), then call the function to read this excel, How can I do it?

I created a function to read excel sheet read_Excel_file(path) that return a list contains some data from a specific Column.
and in the main code I search all the excel files (where the name start Design) and this excel file should be saved in a folder Design. If I find the excel file, I call the function read_Excel_file.
Please find below the code:
import openpyxl as opx
import os
for r, d, f in os.walk('.'):
for file in f:
if '.xlsx' and 'design' in file:
#print(r)
if r.endswith('\Design'):
print(file)
read_Excel_file(file)
but I get the error :
No such file or directory
even if I am sure that I have this file in my directory
Do you think that I have path problem?
PS: I add print(file) just to check the name of the file, but when read_Excel_file(file) after that I have the error.
Can you help me please?

File is just the name of the file. You are missing the complete adress.
You need to add the root part of the address.
Just do:
filepath = os.path.join(r, file)
read_Excel_file(filepath)

How to write script that edits every excel file inside a folder

I want to make a script that writes a certain text to the A1 cell of every excel file inside a folder. I'm not sure how to get Python to open every file one by one, make changes to A1, and then overwrite save the original file.
import os
import openpyxl
os.chdir('C:/Users/jdal/Downloads/excelWorkdir')
folderList = os.listdir()
for file in in os.walk():
for name in file:
if name.endswith(".xlsx" and ".xls")
wb = openpyxl.load_workbook()
sheet = wb.get_sheet_by_name('Sheet1')
sheet['A1'] = 'input whatever here!'
sheet['A1'].value
wb.save()

I see following errors in your code:
You have an error in .endswith, it should be
name.endswith((".xlsx",".xls"))
i.e. it needs to be feed with tuple of allowed endings.
Your if lacks : at end and your indentation seems to be broken.
You should deliver one argument to .load_workbook and one argument to .save, i.e. name of file to read/to write.

I would iterate through the folder and use pandas to read the files as temporary data frames. From there, they are easily manipulable.
Assuming you are in the relevant directory:
import pandas as pd
import os
files = os.listdir()
for i in range(len(files)):
if files[i].endswith('.csv'):
# Store file name for future replacement
name = str(files[i])
# Save file as dataframe and edit cell
df = pd.read_csv(files[i])
df.iloc[0,0] = 'changed value'
# Replace file with df
df.to_csv(name, index=False)

pandas - reading dynamically named file within a .zip

I am creating a new dataframe in pandas as below:
df = pd.read_excel(zipfile.open('zipfilename 2017-06-28.xlsx'), header=1, names=cols)
The single .xlsx within the .zip is dynamically named (so changes based on the date).
This means I need to change the name of the .xlsx in my code each time I open the .zip to account for the dynamically named .xlsx.
Is there a way to make pandas read the file within the .zip, regardless of the name of the file? Or to return the name of the .xlsx within the line of code somehow?
Thanks

Read your file using,
df = pd.read_excel('zipfilename 2017-06-28.xlsx',compression='zip', header=1, names=cols)