Language: Python 3.8
Platform: MacOS 11 | Windows 10
Filetypes: .xlsx | .csv.
Task: File/Format Conversion
Synopsis: My excel file has cells with functions/formulas. I want to save this file as a .csv while preserving the value of the formulas (not the actual string of the function, itself)
What works: Pause script, prompt user to open Excel > 'Save As' .csv // Excel processes the functions within the cells and preserves the values before saving as .csv
What hasn't worked: Using pandas or openpyxl to convert the excel file to a .csv (such as 'wb.save' and 'df.to_csv' // The produced .csv file does not process the function cells and instead outputs nothing within those cells.
Question: Anyway of leveraging excel's 'process the function and save the values' function within the Python script?
Thank you!
Sample Code - Pandas
df = pd.read_excel('file.xlsx')
df.to_csv('file.csv')
Sample Code - Openpyxl
wb = openpyxl.load_workbook('file.xlsx', data_only=True)
sheet = wb.active
with open('file.csv', 'w', newline="") as f:
c = csv.writer(f)
for r in sheet.iter_rows():
c.writerow([cell.value for cell in r])
wb.save('file.csv')
Sample Problem
Excel Columns:
A: ['First Initial']
B: ['Last Name']
C: ['Email']
Formula in all rows within column C:
C1: [=CONCATENATE(A1,".",B1,"#domain.net")]
C2: [=CONCATENATE(A2,".",B2,"#domain.net")]
C3: [=CONCATENATE(A3,".",B3,"#domain.net")]
etc.
Output of 'file.xlsx' through excel & 'file.csv' (via excel > 'Save As' .csv):
A1: ['j']
B1: ['doe']
C1: ['j.doe#domain.net']
Output of 'file.csv' after following the Pandas Sample Code:
A1: ['j']
B1: ['doe']
C1: ['']
if a cell does not contain a formula, the conversion outputs correct values within the cells. For the cells with formulas, the cells are empty (since .csv is just plain-text). Is there a way to replicate excel's behavior of running the functions first > save output value into cell > save as .csv?
UPDATE:
So I found the issue, although not sure how to go about solving this. Pandas works as intended when I created a fresh .xlsx and tried the sample code. But it didn't work with the .xlsx in my script - and I narrowed it down to this step
The following is a snippet from my script that copies values from one excel file into another:
wb1 = xl.load_workbook('/file1.xlsx')
ws1 = wb1.worksheets[0]
wb2 = xl.load_workbook('/file2.xlsx')
ws2 = wb2.active
mr = ws1.max_row
mc = ws1.max_column
for i in range (1, mr + 1):
for j in range (1, mc + 1):
c = ws1.cell(row = i, column = j)
ws2.cell(row = i, column = j).value = c.value
wb2.save('file2.xlsx')
The file ('file2.xlsx'), while seemingly opens and functions just like a regular excel file, DOES NOT preserve its values within cells that have formulas after converting it to a .csv via pandas.
The file ('file1.xlsx') however, does this just fine.
BUT, if I open 'file2.xlsx' and just simply save it (without changing anything), and then try converting it via pandas - it DOES end up preserving the values within formulas.
So there's definitely something wrong in my code (surprise, surprise) that does this. Not sure why, though.
SOLVED
I was able to solve my own question - posting it here for anyone else who has a similar issue (searching this problem led me believe ya'll exist, so here you go.)
Note: This only works on a Windows system, with Excel installed
import win32com.client as win32
from win32com.client import Dispatch
from win32com.client import constants as c
excel = Dispatch('Excel.Application') # Calls Excel
excel.DisplayAlerts = False # Disables prompts, such as asking to overwrite files
wb = excel.Workbooks.Open("/file.xlsx") # Input File
wb.SaveAs("/file.csv"), c.xlCSV) # Output File
excel.Application.Quit() # Close Excel
excel.DisplayAlerts = True # Turn alerts back on
This can be done using Pandas library.
Here ,this might help :
https://www.geeksforgeeks.org/convert-excel-to-csv-in-python/
Related
I am very new in the programming space and tried a lot of things to solve my problem.
It feels like I exactly knew what I have to do, but I can not communicate it in Python.
I have two Excel sheets: wb 1 and wb2.
wb1 is a self updating sheet from some downloaded CSV data with one table, and extending number of rows on and fix number of columns.
I want to insert the date from wb1 (column A-J) and the max. number of rows into wb2 (sheet 3) sheet 3 is a huge table with a lot of data but just the specific part out of wb1 (same positioning as in the wb1 file) needs to be updated.
So I basically want to copy a part out of an workbook and insert int into a specific position into another workbook.
The data in workbook2 is formatted as table so I guess I haive to format the date in workbook1 as table as well before I can copy it. I tried it so far with openpyxl what seems to work great but I can't specify the location in wb2.
I knew this is a problem what can be solved within seconds but I am really in a dead end right now.
Workbook1 = "download.xlsx"
Workbook2 = "Git.xlsx"
wb1 = xl.load_workbook(filename = Workbook1)
ws1 = wb1.worksheets[0]
wb2 = xl.load_workbook(filename = Workbook2)
ws2 = wb2.get_sheet_by_name("Table3")
?
wb2.save(path2)
First, you've to read all the data from the first sheet which download.xlsx, then appened the output value into a list of tuples, after that write the values using the same package openpyxl into the second file Git.xlsx.
I will not be able to write the whole code because my phone battery is running off. So, i will try to share with you an important links you can follow:-
Reading using openpyxl
Writing using openpyxl
Lists in Python
Python List of tuples
I was using openpyxl but found it can not handle macro-based Excel files. I need to know how to copy certain cells from one excel file to another with xlwings, which can handle macros. The syntax is not the same and i am not sure how to go about doing that cell by cell accounting or rows and columns. Any sample code would be much appreciated!
for the ease to read and write data, you can use the below function to read and write
import xlwings as xw
def readData(rownum, column_name):
rownum1 = str(rownum)
cell = column_name+rownum1
return sheet.range(cell).value
def writeData(rownum, column_name, data):
rownum1 = str(rownum)
cell = column_name+ str(rownum)
sheet2.range(cell).value= data
first make an object and open your excel file from where you need to read and write and define worksheet
wb_reader = xw.Book(path_file_ of_read)
sheet = workbook.sheets["Sheet1"] #for reading
wb_writer = xw.Book(path_file_ of_write)
sheet2 = workbook.sheets["Sheet2"] #for writing
and now just do
x = readData(2,3)
write(2,3,x)
Here is a sample code where i am trying to print the number of rows in the excel file each time a new row is inserted.The code does not work ,because i believe it's not interacting with the excel file at run time.
import xlrd
loc = r'C:\Users\dell\Desktop\sample2.xlsx'
wb = xlrd.open_workbook(loc)
sheet = wb.sheet_by_index(0)
k = sheet.nrows
while(True):
wb = xlrd.open_workbook(loc)
sheet = wb.sheet_by_index(0)
k1 = sheet.nrows
if(k1 > k):
print(k1)
k=k1
I think you misunderstand how the library xlrd works. It provides you with an interface to Excel files, not to an Excel session of an Excel instance somebody is working on in parallel. Everything you do in Excel is not written to the according file until you save the workbook. Hence, this is the moment when your code reads updated cells, not already when cells are changed.
Right now, I am sorting data with pandas and then adding those values into the excel file and it works great. The problem is that the excel file already has the sort coded in and when I write data to the xlsx it will remove the sort. My colleagues want to be able to use the built-in excel sort and I can't figure out why openpyxl will remove that sort when I write data to the file.
Is there a parameter I'm missing or is this just how openpyxl operates? I can include code if it is needed.
Using the latest version of python and openpyxl.
This happens to me on mac. Here is the code for writing from a dataframe to a sheet. Afterwards, the sheet will delete the pre-formatted sort in the .xlsx
def add_sheet(self, df, sheet, min_col, max_col, sheet_dim):
index_positions = list(df.index.values)
r = 1
for x in index_positions:
_row = df.loc[x].values
c = min_col
r += 1
for val in _row:
if c <= max_col:
sheet.cell(row=r, column=c).value = val
c += 1
Maybe it's how I save the excel file? Some parameter I'm missing?
test.save("test.xlsx")
Nevermind, here is code that will strip all sorts from my excel file on Mac.
test = test.load_workbook("test.xlsx")
test.save("test_save.xlsx")
The file goes from 376KBs to 259KBs. This doesn't make any sense.
I'm able to open my pre-existing workbook, but I don't see any way to open pre-existing worksheets within that workbook. Is there any way to do this?
You cannot append to an existing xlsx file with xlsxwriter.
There is a module called openpyxl which allows you to read and write to preexisting excel file, but I am sure that the method to do so involves reading from the excel file, storing all the information somehow (database or arrays), and then rewriting when you call workbook.close() which will then write all of the information to your xlsx file.
Similarly, you can use a method of your own to "append" to xlsx documents. I recently had to append to a xlsx file because I had a lot of different tests in which I had GPS data coming in to a main worksheet, and then I had to append a new sheet each time a test started as well. The only way I could get around this without openpyxl was to read the excel file with xlrd and then run through the rows and columns...
i.e.
cells = []
for row in range(sheet.nrows):
cells.append([])
for col in range(sheet.ncols):
cells[row].append(workbook.cell(row, col).value)
You don't need arrays, though. For example, this works perfectly fine:
import xlrd
import xlsxwriter
from os.path import expanduser
home = expanduser("~")
# this writes test data to an excel file
wb = xlsxwriter.Workbook("{}/Desktop/test.xlsx".format(home))
sheet1 = wb.add_worksheet()
for row in range(10):
for col in range(20):
sheet1.write(row, col, "test ({}, {})".format(row, col))
wb.close()
# open the file for reading
wbRD = xlrd.open_workbook("{}/Desktop/test.xlsx".format(home))
sheets = wbRD.sheets()
# open the same file for writing (just don't write yet)
wb = xlsxwriter.Workbook("{}/Desktop/test.xlsx".format(home))
# run through the sheets and store sheets in workbook
# this still doesn't write to the file yet
for sheet in sheets: # write data from old file
newSheet = wb.add_worksheet(sheet.name)
for row in range(sheet.nrows):
for col in range(sheet.ncols):
newSheet.write(row, col, sheet.cell(row, col).value)
for row in range(10, 20): # write NEW data
for col in range(20):
newSheet.write(row, col, "test ({}, {})".format(row, col))
wb.close() # THIS writes
However, I found that it was easier to read the data and store into a 2-dimensional array because I was manipulating the data and was receiving input over and over again and did not want to write to the excel file until it the test was over (which you could just as easily do with xlsxwriter since that is probably what they do anyway until you call .close()).
After searching a bit about the method to open the existing sheet in xlxs, I discovered
existingWorksheet = wb.get_worksheet_by_name('Your Worksheet name goes here...')
existingWorksheet.write_row(0,0,'xyz')
You can now append/write any data to the open worksheet.
You can use the workbook.get_worksheet_by_name() feature:
https://xlsxwriter.readthedocs.io/workbook.html#get_worksheet_by_name
According to https://xlsxwriter.readthedocs.io/changes.html the feature has been added on May 13, 2016.
"Release 0.8.7 - May 13 2016
-Fix for issue when inserting read-only images on Windows. Issue #352.
-Added get_worksheet_by_name() method to allow the retrieval of a worksheet from a workbook via its name.
-Fixed issue where internal file creation and modification dates were in the local timezone instead of UTC."
Although it is mentioned in the last two answers with it's documentation link, and from the documentation it seems indeed there are new methods to work with the "worksheets", I couldn't able to find this methods in the latest package of "xlsxwriter==3.0.3"
"xlrd" has removed support for anything other than xls files now.
Hence I was able to workout with "openpyxl" this gives you the expected functionality as mentioned in the first answer above.