Develop Reference

pandas read_csv not working when running via cron on Mac OS

I have a python script that is being run by cron. The script imports the pandas module and uses read_csv to load a csv to a data frame and then later saves it to another csv. 'apath' is the absolute path to the file:
statedata_raw=pd.read_csv(apath+'statedata.csv')
statedata_raw.to_csv(apath+'state_data.csv',index=False)
The permissions on the csv file are set correctly -rwxr-xr-x
when I run it in the command line, everything works fine. When I run it via cron I get the following error:
Traceback (most recent call last):
File "/users/maderman/wdtest.py", line 21, in <module>
statedata_raw=pd.read_csv(apath+'statedata.csv')
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 676, in parser_f
return _read(filepath_or_buffer, kwds)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 448, in _read
parser = TextFileReader(fp_or_buf, **kwds)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 880, in __init__
self._make_engine(self.engine)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1114, in _make_engine
self._engine = CParserWrapper(self.f, **self.options)
File "/opt/miniconda3/lib/python3.7/site-packages/pandas/io/parsers.py", line 1891, in __init__
self._reader = parsers.TextReader(src, **kwds)
File "pandas/_libs/parsers.pyx", line 374, in pandas._libs.parsers.TextReader.__cinit__
File "pandas/_libs/parsers.pyx", line 678, in pandas._libs.parsers.TextReader._setup_parser_source
OSError: Initializing from file failed
I verified that pandas itself is loading and that the to_csv is working by replacing the read_csv. When I replaced the read_csv with the following code to manually create a dataframe, everything worked fine, running in command line and running in cron:
cat=['a','a','a','a','a','b','b','b','b','b']
val=[1,2,3,4,5,6,7,8,9,10]
columns=['cat','val']
data=[cat,val]
dict={key:value for key,value in zip(columns,data)}
statedata_raw=pd.DataFrame(data=dict)
I found another post that suggested passing the argument engine='python' to the read_csv, but that didn't do anything.
So I know that:
cron is running python fine
it can import pandas and run a couple of different pandas functions.
the file permissions are fine
The issue seems to be specifically related to the read_csv commmand.
Any suggestions would be appreciated.

The framing on this question was wrong and it boiled down to a permissions issue. A better question was posted and answered here: stackoverflow.com/questions/62353610

Openpyxl ['MergedCell' object attribute 'hyperlink' is read-only] Problem

I met a problem while loading a xlsx File. In the worksheet there is a Hyperlink in a merged cell. While loading the file, error ocured. Can anybody
Code is just like this.
workbook = openpyxl.load_workbook(report_filepath)
Error Info:
File "F:\mainfunc_new.py", line 733, in read_report
workbook = openpyxl.load_workbook(report_filepath)
File "C:\Users\10225167\AppData\Local\Programs\Python\Python36\lib\site-packages\openpyxl\reader\excel.py", line 312, in load_workbook
reader.read()
File "C:\Users\10225167\AppData\Local\Programs\Python\Python36\lib\site-packages\openpyxl\reader\excel.py", line 274, in read
self.read_worksheets()
File "C:\Users\10225167\AppData\Local\Programs\Python\Python36\lib\site-packages\openpyxl\reader\excel.py", line 228, in read_worksheets
ws_parser.bind_all()
File "C:\Users\10225167\AppData\Local\Programs\Python\Python36\lib\site-packages\openpyxl\worksheet\_reader.py", line 389, in bind_all
self.bind_hyperlinks()
File "C:\Users\10225167\AppData\Local\Programs\Python\Python36\lib\site-packages\openpyxl\worksheet\_reader.py", line 355, in bind_hyperlinks
cell.hyperlink = link
AttributeError: 'MergedCell' object attribute 'hyperlink' is read-only
Thanks.

Use version 2.5.14 instead of yours. It worked for me.
pip install openpyxl==2.5.14

Double check if the cells that you're merging are empty. I don't think you can merge cells if that has values.

Maybe this error was caused by Excel format. I deleted the current excel file and replaced it with a good excel file, the error disappeared.

aifc module Error: Python won't open file

I'm just getting started with python so please bear with me ;)
While following a basic tutorial I ran into a problem while opening a file, here's the traceback:
File "/home/nick/Dropbox/workspace/pytest/schlange.py", line 55, in <module>
f=open("file.csv","r")
File "/usr/lib/python2.6/aifc.py", line 922, in open
return Aifc_read(f)
File "/usr/lib/python2.6/aifc.py", line 335, in __init__
self.initfp(f)
File "/usr/lib/python2.6/aifc.py", line 288, in initfp
raise Error, 'file does not start with FORM id'
aifc.Error: file does not start with FORM id
Does that mean that 'aifc.py' in my python installation is broken or did I miss something important here?

You've decided to import * from aifc, and its open() has shadowed the built-in open(). This is why we don't import *. Import the module itself instead and use the reference to get to its names when required, e.g. aifc.open().

pandas HDFStore - how to reopen?

I created a file by using:
store = pd.HDFStore('/home/.../data.h5')
and stored some tables using:
store['firstSet'] = df1
store.close()
I closed down python and reopened in a fresh environment.
How do I reopen this file?
When I go:
store = pd.HDFStore('/home/.../data.h5')
I get the following error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/misc/apps/linux/python-2.6.1/lib/python2.6/site-packages/pandas-0.10.0-py2.6-linux-x86_64.egg/pandas/io/pytables.py", line 207, in __init__
self.open(mode=mode, warn=False)
File "/misc/apps/linux/python-2.6.1/lib/python2.6/site-packages/pandas-0.10.0-py2.6-linux-x86_64.egg/pandas/io/pytables.py", line 302, in open
self.handle = _tables().openFile(self.path, self.mode)
File "/apps/linux/python-2.6.1/lib/python2.6/site-packages/tables/file.py", line 230, in openFile
return File(filename, mode, title, rootUEP, filters, **kwargs)
File "/apps/linux/python-2.6.1/lib/python2.6/site-packages/tables/file.py", line 495, in __init__
self._g_new(filename, mode, **params)
File "hdf5Extension.pyx", line 317, in tables.hdf5Extension.File._g_new (tables/hdf5Extension.c:3039)
tables.exceptions.HDF5ExtError: HDF5 error back trace
File "H5F.c", line 1582, in H5Fopen
unable to open file
File "H5F.c", line 1373, in H5F_open
unable to read superblock
File "H5Fsuper.c", line 334, in H5F_super_read
unable to find file signature
File "H5Fsuper.c", line 155, in H5F_locate_signature
unable to find a valid file signature
End of HDF5 error back trace
Unable to open/create file '/home/.../data.h5'
What am I doing wrong here? Thank you.

In my hands, following approach works best:
df = pd.DataFrame(...)
"write"
with pd.HDFStore('test.h5', mode='w') as store:
store.append('df', df, data_columns= df.columns, format='table')
"read"
with pd.HDFStore('test.h5', mode='r') as newstore:
df_restored = newstore.select('df')

You could try doing instead:
store = pd.io.pytables.HDFStore('/home/.../data.h5')
df1 = store['firstSet']
or use the read method directly:
df1 = pd.read_hdf('/home/.../data.h5', 'firstSet')
Either way, you should have pandas 0.12.0 or higher...

I had the same problem and finally fixed it by installing the pytables module (next to the pandas modules which I was using):
conda install pytables
which got me numexpr-2.4.3 and pytables-3.2.0
After that it worked. I am using pandas 0.16.2 under python 2.7.9

Error when trying to convert .sxw file to .rml in OpenERP 6.0.3

[2012-06-01 15:33:10,638][molisamples] ERROR:web-services:Uncaught exception
Traceback (most recent call last):
File "osv\osv.pyo", line 122, in wrapper
File "osv\osv.pyo", line 176, in execute
File "osv\osv.pyo", line 167, in execute_cr
File "C:\Program Files (x86)\OpenERP 6.0\Server\addons\base_report_designer\base_report_designer.py", line 42, in sxwtorml
File "C:\Program Files (x86)\OpenERP 6.0\Server\addons\base_report_designer\openerp_sxw2rml\openerp_sxw2rml.py", line 309, in sxw2rml
File "C:\Program Files (x86)\OpenERP 6.0\Server\addons\base_report_designer\openerp_sxw2rml\openerp_sxw2rml.py", line 294, in unpackNormalize
File "C:\Program Files (x86)\OpenERP 6.0\Server\addons\base_report_designer\openerp_sxw2rml\openerp_sxw2rml.py", line 269, in oo_read
File "zipfile.pyo", line 346, in init
File "zipfile.pyo", line 366, in _GetContents
File "zipfile.pyo", line 378, in _RealGetContents
BadZipfile: File is not a zip file
I get the above error when I try to convert a report I just designed to .rml (using Open Office Writer). Please what could be the issue. I am seriously confused here

You can convert your .sxw to .rml using base_report_designer module.
Try following steps:
Open terminal -> go to openerp_sxw2rml folder like this:
cd addons/base_report_designer/openerp_sxw2rml
Then run this command: python openerp_sxw2rml.py absolute path of sxw > absolute path of rml
Like this:
python openerp_sxw2rml.py /home/arya/my_module/report/my_report.sxw > /home/arya/my_module/report/my_report.rml
This will convert sxw file into rml and you can find your file at given path of rml.
Thank you.

The error says that the file is not a zip file, so it's probably expecting the compressed format of sxw file. Any chance you saved the file in OpenOffice's uncompressed format?

Make sure when you save in Openoffice Writer, you select the old format, the one with SXW extension.
Don't just type .sxw, make sure the program puts it there by itself by selecting the correct entry in the fileformat selectionbox (i forget the full title and cannot check atm)

I figured it out. I had some errors in the python parser file for the report. That's what was causing the issue. It's been fixed now. Thanks y'all for the help