I'm using python3.3 and I'm having a cryptic error when trying to pickle a simple dictionary.
Here is the code:
import os
import pickle
from pickle import *
os.chdir('c:/Python26/progfiles/')
def storvars(vdict):
f = open('varstor.txt','w')
pickle.dump(vdict,f,)
f.close()
return
mydict = {'name':'john','gender':'male','age':'45'}
storvars(mydict)
and I get:
Traceback (most recent call last):
File "C:/Python26/test18.py", line 31, in <module>
storvars(mydict)
File "C:/Python26/test18.py", line 14, in storvars
pickle.dump(vdict,f,)
TypeError: must be str, not bytes
The output file needs to be opened in binary mode:
f = open('varstor.txt','w')
needs to be:
f = open('varstor.txt','wb')
Just had same issue. In Python 3, Binary modes 'wb', 'rb' must be specified whereas in Python 2x, they are not needed. When you follow tutorials that are based on Python 2x, that's why you are here.
import pickle
class MyUser(object):
def __init__(self,name):
self.name = name
user = MyUser('Peter')
print("Before serialization: ")
print(user.name)
print("------------")
serialized = pickle.dumps(user)
filename = 'serialized.native'
with open(filename,'wb') as file_object:
file_object.write(serialized)
with open(filename,'rb') as file_object:
raw_data = file_object.read()
deserialized = pickle.loads(raw_data)
print("Loading from serialized file: ")
user2 = deserialized
print(user2.name)
print("------------")
pickle uses a binary protocol, hence only accepts binary files. As the document said in the first sentence, "The pickle module implements binary protocols for serializing and de-serializing".
Related
I want to read a .pyc file. However, I cannot find any documentation on the format.
The only one I found does not work for Python 3 (although it does for Python 2):
>>> f = open('__pycache__/foo.cpython-34.pyc', 'rb')
>>> f.read(4)
b'\xee\x0c\r\n'
>>> f.read(4)
b'\xf8\x17\x08W'
>>> marshal.load(f)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
ValueError: bad marshal data (unknown type code)
marshal only consumes one byte: \x00, which indeed is not a valid first character for marshall (as a comparison, the first byte of Python 2 bytecode for the same empty module is c)
So, how can I decode what comes after the header?
Try this. It worked a while back. They added another int32 in v3.
def load_file(self, source):
if isinstance(source, str):
import os.path
if not os.path.exists(source):
raise IOError("Cannot load_file('"
+ source
+ "'): does not exist")
with open(source, "rb") as fh:
header_bytes = fh.read(12)
# ignore header
self.code = marshal.load(fh)
return self.code
Have you looked at the dissembler?
https://docs.python.org/3/library/dis.html
I have to work with a txt file and to do that I used the following code:
inputFile = open("C:/Abaqus_JOBS/Job-M1-3_4.inp", "r") #CAE INPUT FILE
However I get this error when I ran this line in a specific application for running python scripts available in another program. I don't get any error when I ran it in Spyder.
TypeError: an integer is required
I don't have a clue why this error occurs....
EDIT:
lines of code until line in question
import os
from os import *
from abaqus import *
from odbAccess import *
from abaqusConstants import *
import time
import itertools
os.chdir('C:\\Abaqus_JOBS')
LCKf = 'C:\\Abaqus_JOBS\\Job-M1-3_2.lck'
STAf = 'C:\\Abaqus_JOBS\\Job-M1-3_2.sta'
def get_num_part(s):
for i in xrange(len(s)):
if s[i:].isdigit():
return s[i:]
return ''
if not path.exists(LCKf):
time.sleep(1)
while path.exists(LCKf) and path.isfile(LCKf) and access(LCKf, R_OK):
variableX = 0
else:
odb = openOdb(path='Job-M1-3_2.odb')
#get CF
#session.odbs[name].steps[name].frames[i].FieldOutput
myAssembly = odb.rootAssembly
myAssemblyName = odb.rootAssembly.name
nsteps=len(odb.steps.values())
step1 = odb.steps.values()[nsteps-1]
step1Name = odb.steps.values()[nsteps-1].name
myInstanceName = odb.rootAssembly.instances.values()[0].name
dCF3=[]
dCF3v=[]
coordFv=[]
fileData = [] #array with the input file
nodes = [] #array with the content of *NODES
inputFile = open("C:/Abaqus_JOBS/Job-M1-3_4.inp", "r") #CAE INPUT FILE
#fileData = variable with all the lines of the inp file
for line in inputFile:
fileData.append([x.strip() for x in line.split(',')])
the error is:
Traceback (most recent call last):
File "c:/Abaqus_JOBS/results.py", line 47, in <module>
inputFile = open("C:/Abaqus_JOBS/Job-M1-3_4.inp", "r") #CAE INPUT FILE
TypeError: an integer is required
With the
from os import *
You're importing all os stuff in the global namespace, including os.open(). Don't do this.
The second argument, flags, is defined as integer constants while you're providing a single-character string r. This is basically what DSM was telling you and what Lattyware said.
open() included in Python by default in the global namespace, which you were expecting apparently, is different:
Note: This function is intended for low-level I/O. For normal usage,
use the built-in function open(), which returns a “file object” with
read() and write() methods (and many more). To wrap a file descriptor
in a “file object”, use fdopen().
I'm using python3.3 and I'm having a cryptic error when trying to pickle a simple dictionary.
Here is the code:
import os
import pickle
from pickle import *
os.chdir('c:/Python26/progfiles/')
def storvars(vdict):
f = open('varstor.txt','w')
pickle.dump(vdict,f,)
f.close()
return
mydict = {'name':'john','gender':'male','age':'45'}
storvars(mydict)
and I get:
Traceback (most recent call last):
File "C:/Python26/test18.py", line 31, in <module>
storvars(mydict)
File "C:/Python26/test18.py", line 14, in storvars
pickle.dump(vdict,f,)
TypeError: must be str, not bytes
The output file needs to be opened in binary mode:
f = open('varstor.txt','w')
needs to be:
f = open('varstor.txt','wb')
Just had same issue. In Python 3, Binary modes 'wb', 'rb' must be specified whereas in Python 2x, they are not needed. When you follow tutorials that are based on Python 2x, that's why you are here.
import pickle
class MyUser(object):
def __init__(self,name):
self.name = name
user = MyUser('Peter')
print("Before serialization: ")
print(user.name)
print("------------")
serialized = pickle.dumps(user)
filename = 'serialized.native'
with open(filename,'wb') as file_object:
file_object.write(serialized)
with open(filename,'rb') as file_object:
raw_data = file_object.read()
deserialized = pickle.loads(raw_data)
print("Loading from serialized file: ")
user2 = deserialized
print(user2.name)
print("------------")
pickle uses a binary protocol, hence only accepts binary files. As the document said in the first sentence, "The pickle module implements binary protocols for serializing and de-serializing".
i have encountered a very strange issue: i use json.dump to write a file and then use json.load to read the file.
The same code can run succeed on windows 7 but it can not do on mac os x 10.7
Below is the code:
class Result:
def __init__(self,name,result):
self.name = name
self.result = result
def __repr__(self):
return 'Result name : %s , result : %s' % (self.name,self.result)
class MyEncoder(json.JSONEncoder):
def default(self,obj):
#convert object to a dict
d = {'CaseResult':{}}
d['CaseResult'][obj.name] = obj.result
return d
def save(name,result):
filename = 'basic.json'
obj = Result(name,result)
obj_json = MyEncoder().encode(obj)
with open(filename, mode='ab+') as fp:
json.dump(obj_json,fp)
s=json.load(fp)
save('aaa','bbb')
in mac os it give an error "ValueError:NO JSON object could be decoded"
who can tell me why this happen and how can i resolve it
This problem is unrelated to being run on a Mac; this code should never work:
with open(filename, mode='ab+') as fp:
json.dump(obj_json,fp)
s=json.load(fp)
This is because after json.dump, your file pointer is at the end of the file. You must call fp.seek to reset it to the initial position, like this:
import os
with open(filename, mode='rb+') as fp:
fp.seek(0, os.SEEK_END)
pos = fp.tell()
json.dump(obj_json,fp)
fp.seek(pos)
s=json.load(fp)
I'm not sure how this actually works on Windows, but you're missing a seek back to the beginning of the file before you read the object back. change your save/load to
with open(filename, mode='ab+') as fp:
json.dump(obj_json,fp)
fp.seek(0)
s=json.load(fp)
and it runs just fine on MacOS too. Note that you're appending to the file, so only the first run succeeds in loading the object back, the next one will find extra data after the end of the object.
So, I'm trying to save some objects to disk on Windows 7 using Python's pickle. I'm using the code below, which fails on pretty much any arbitrary object (the contents of saveobj aren't important, it fails regardless). Below is my test code:
import pickle, os, time
outfile = "foo.pickle"
f = open(outfile, 'wb')
p = pickle.Pickler(f, -1)
saveobj = ( 2,3,4,5,["hat", {"mat": 6}])
p.save(saveobj)
#pickle.dump(saveobj, f)
print "done pickling"
f.close()
g = open(outfile, 'rb')
tup = pickle.load(g)
g.close()
print tup
When I run it, I get the following output/error:
done pickling
Traceback (most recent call last):
File "C:\Users\user\pickletest2.py", line 13, in <module>
tup = pickle.load(g)
File "C:\Python26\lib\pickle.py", line 1370, in load
return Unpickler(file).load()
File "C:\Python26\lib\pickle.py", line 858, in load
dispatch[key](self)
File "C:\Python26\lib\pickle.py", line 880, in load_eof
raise EOFError
EOFError
However, if I use pickle.dump() instead of a Pickler object, it works just fine. My reason for using Pickler is that I would like to subclass it so I can perform operations on each object before I pickle it.
Does anybody know why my code is doing this? My searching has revealed that not having 'wb' and 'rb' commonly cause this, as does not having f.close(), but I have both of those. Is it a problem with using -1 as the protocol? I'd like to keep it, as it can handle objects which define their own __slots__ methods without defining a __getstate__ method.
Pickler.save() is a lower level method, that you're not supposed to call directly.
If you call p.dump(saveobj) instead of p.save(saveobj), it works as expected.
Perhaps it should be called _save to avoid confusion. But dump is the method described in the documentation, and it neatly matches up with the module-level pickle.dump.
In general it is better to use cPickle for performance reasons (since cPickle is written in C).
Anyway, using dump it works just fine:
import pickle
import os, time
outfile = "foo.pickle"
f = open(outfile, 'wb')
p = pickle.Pickler(f, -1)
saveobj = ( 2,3,4,5,["hat", {"mat": 6}])
p.dump(saveobj)
#pickle.dump(saveobj, f)
f.close()
print "done pickling"
#f.close()
g = open(outfile, 'rb')
u = pickle.Unpickler(g) #, -1)
tup = u.load()
#tup = pickle.load(g)
g.close()
print tup