I am new to python. I have a file data.pkl. What I would like to do is get the data from the file. I looked at http://docs.python.org/library/pickle.html, 11.1.7 example and tried exactly that.
My code looks like this:
import pprint, pickle
pkl_file = open('data.pkl', 'rb')
data1 = pickle.load(pkl_file)
pprint.pprint(data1)
pkl_file.close()
But it is giving me error:
Traceback (most recent call last):
File "/home/sadiksha/workspace/python/test.py", line 5, in <module>
data1 = pickle.load(pkl_file)
File "/usr/lib/python2.7/pickle.py", line 1378, in load
return Unpickler(file).load()
File "/usr/lib/python2.7/pickle.py", line 858, in load
dispatch[key](self)
File "/usr/lib/python2.7/pickle.py", line 966, in load_string
raise ValueError, "insecure string pickle"
Can anyone please tell me what am I doing wrong here?
It seems that your pickle file was either not written correctly (specifying 'wb') or the file was somehow corrupted. Try creating your own pickle file and reading that back in. That should do the trick.
As for the pickle file specified, it is definitely corrupted.
Related
I am trying to load a pickle file which i created using joblib.dump()
The dumping code looks like:
from sklearn.externals import joblib
with open('sample.pickle','wb') as f:
joblib.dump([x,y],f)
This works fine and my sample.pickle is saved successfully. But when i try to load this file:
with open('sample.pickle', 'rb') as f:
x, y = joblib.load(f)
I get the following error:
ValueError: EOF: reading array data, expected 1200 bytes got 0
The full error log looks like this:
Traceback (most recent call last):
File "model.py", line 16, in <module>
vec_x, vec_y = joblib.load(f)
File "C:\Users\acer_pc\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\externals\joblib\numpy_pickle.py", line 568, in load
obj = _unpickle(fobj)
File "C:\Users\acer_pc\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\externals\joblib\numpy_pickle.py", line 508, in _unpickle
obj = unpickler.load()
File "C:\Users\acer_pc\AppData\Local\Programs\Python\Python36\lib\pickle.py", line 1050, in load
dispatch[key[0]](self)
File "C:\Users\acer_pc\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\externals\joblib\numpy_pickle.py", line 341, in load_build
self.stack.append(array_wrapper.read(self))
File "C:\Users\acer_pc\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\externals\joblib\numpy_pickle.py", line 184, in read
array = self.read_array(unpickler)
File "C:\Users\acer_pc\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\externals\joblib\numpy_pickle.py", line 135, in read_array
read_size, "array data")
File "C:\Users\acer_pc\AppData\Local\Programs\Python\Python36\lib\site-packages\sklearn\externals\joblib\numpy_pickle_utils.py", line 646, in _read_bytes
raise ValueError(msg % (error_template, size, len(data)))
ValueError: EOF: reading array data, expected 1200 bytes got 0
By the way, i am very new to dumping and pickling.
Any help will be appreciated. Thanks in advance.
Currently I cannot
pkl_file = open(item, 'rb')
meta = pickle.load(pkl_file)
print type(meta)
I get:
Traceback (most recent call last):
File "blob-pickles.py", line 16, in <module>
meta = pickle.load(pkl_file);print type(meta)
File "C:\Anaconda\lib\pickle.py", line 1378, in load
return Unpickler(file).load()
File "C:\Anaconda\lib\pickle.py", line 858, in load
dispatch[key](self)
KeyError: '\x00'
pointing to the pickle.load() line
Originally i saved the pickle like this :
output = open(fileName+'.pkl', 'wb')
pickle.dump(meta, output)
where meta was a list with 4 strings
So I tried
pickle.loads()
And I get EOF error. The files are not blank...
When I wrote the pickles i used wb mode and also to read them..
What went wrong or what can i do to retrieve the information within which is just a small list?
pickle left a bad taste in my mouth. Next time I'll try dill. Maybe their code is more kosher.
I'm getting the following error when i try to run my py file on a windows machine, everything works fine from the Mac, but i cant get the two machines to talk to one another. do i have to 'dumps' in ascii; if so how do i do that? any suggestions? THANKS!!
Traceback (most recent call last):
File "C:\Users\Keith\Desktop\rcserver\tcpserver_simpleReceive.py", line 23, in <module>
new_data = pickle.loads(data)
File "C:\Python27\lib\pickle.py", line 1382, in loads
return Unpickler(file).load()
File "C:\Python27\lib\pickle.py", line 858, in load
dispatch[key](self)
File "C:\Python27\lib\pickle.py", line 1165, in load_put
self.memo[self.readline()[:-1]] = self.stack[-1]
IndexError: list index out of range
The dumps:
if e.type == pygame.JOYAXISMOTION:
client_socket.sendall (pickle.dumps(e.dict,0))
this is the loads:
data = client_socket.recv(1024)
new_data = pickle.loads(data)
I created a file by using:
store = pd.HDFStore('/home/.../data.h5')
and stored some tables using:
store['firstSet'] = df1
store.close()
I closed down python and reopened in a fresh environment.
How do I reopen this file?
When I go:
store = pd.HDFStore('/home/.../data.h5')
I get the following error.
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/misc/apps/linux/python-2.6.1/lib/python2.6/site-packages/pandas-0.10.0-py2.6-linux-x86_64.egg/pandas/io/pytables.py", line 207, in __init__
self.open(mode=mode, warn=False)
File "/misc/apps/linux/python-2.6.1/lib/python2.6/site-packages/pandas-0.10.0-py2.6-linux-x86_64.egg/pandas/io/pytables.py", line 302, in open
self.handle = _tables().openFile(self.path, self.mode)
File "/apps/linux/python-2.6.1/lib/python2.6/site-packages/tables/file.py", line 230, in openFile
return File(filename, mode, title, rootUEP, filters, **kwargs)
File "/apps/linux/python-2.6.1/lib/python2.6/site-packages/tables/file.py", line 495, in __init__
self._g_new(filename, mode, **params)
File "hdf5Extension.pyx", line 317, in tables.hdf5Extension.File._g_new (tables/hdf5Extension.c:3039)
tables.exceptions.HDF5ExtError: HDF5 error back trace
File "H5F.c", line 1582, in H5Fopen
unable to open file
File "H5F.c", line 1373, in H5F_open
unable to read superblock
File "H5Fsuper.c", line 334, in H5F_super_read
unable to find file signature
File "H5Fsuper.c", line 155, in H5F_locate_signature
unable to find a valid file signature
End of HDF5 error back trace
Unable to open/create file '/home/.../data.h5'
What am I doing wrong here? Thank you.
In my hands, following approach works best:
df = pd.DataFrame(...)
"write"
with pd.HDFStore('test.h5', mode='w') as store:
store.append('df', df, data_columns= df.columns, format='table')
"read"
with pd.HDFStore('test.h5', mode='r') as newstore:
df_restored = newstore.select('df')
You could try doing instead:
store = pd.io.pytables.HDFStore('/home/.../data.h5')
df1 = store['firstSet']
or use the read method directly:
df1 = pd.read_hdf('/home/.../data.h5', 'firstSet')
Either way, you should have pandas 0.12.0 or higher...
I had the same problem and finally fixed it by installing the pytables module (next to the pandas modules which I was using):
conda install pytables
which got me numexpr-2.4.3 and pytables-3.2.0
After that it worked. I am using pandas 0.16.2 under python 2.7.9
I recently found out about pickle, which is amazing. But it errors on me when used for my actual script, testing it with a one item dictionary it worked fine. My real script is thousands of lines of code storing various objects within maya into it. I do not know if it has anything to do with the size, I have read around a lot of threads here but none are specific to my error.
I have tried writing with all priorities. No luck.
This is my output code:
output = open('locatorsDump.pkl', 'wb')
pickle.dump(l.locators, output, -1)
output.close()
This is my read code:
jntdump = open('locatorsDump.pkl', 'rb')
test = pickle.load(jntdump)
jntdump.close()
This is the error:
# Error: Error in maya.utils._guiExceptHook:
# File "C:\Program Files\Autodesk\Maya2011\Python\lib\site-packages\pymel-1.0.0-py2.6.egg\maya\utils.py", line 277, in formatGuiException
# exceptionMsg = excLines[-1].split(':',1)[1].strip()
# IndexError: list index out of range
#
# Original exception was:
# Traceback (most recent call last):
# File "<maya console>", line 3, in <module>
# File "C:\Program Files\Autodesk\Maya2011\bin\python26.zip\pickle.py", line 1370, in load
# return Unpickler(file).load()
# File "C:\Program Files\Autodesk\Maya2011\bin\python26.zip\pickle.py", line 858, in load
# dispatch[key](self)
# File "C:\Program Files\Autodesk\Maya2011\bin\python26.zip\pickle.py", line 880, in load_eof
# raise EOFError
# EOFError #
Try using pickle.dumps() and pickle.loads() as a test.
If you don't recieve the same error, you know it is related to the file write.