So I've got a problem - I'm writing a game prototype in Python, using Pygame, and I want to save my games. All of the game-related data is in three instances of certain classes, and I want to save these three instances to a file. However, I've tried pickling these instances, and it doesn't work. Instead, I get "TypeError: can't pickle Surface objects". This is a problem, because I want to store Surface objects.
I'm open to any alternatives to pickling that there may be, using any other kind of data type. The important thing is that these instances get stored, and that their data is then retrievable later on. So what can I do to overcome this issue? Please keep in mind, I'm not a very experienced programmer, having learned Python in my spare time a year ago, and I can't write much of any other language, though I'm slowly learning C++.
The basic point of pickling is that you should be able to serialise the object somehow. An SDL surface is an in memory object holding a lot of run time state. Trying to serialise it is not totally sensible.
What you should do is to decouple the state of your game from the rendering components so that you can serialise just those (pickling or whatever).
It's like trying to save the state of a video by somehow saving the memory buffers holding the decoded video. This will not work. Instead, how you save it is to serialise the location of the video file and the time offset. Then you can continue playback upon load the next time you restore your application.
Reading http://docs.python.org/library/pickle.html#pickle-protocol, you need to either have Surface objects export a reduce method or use the copy_reg module to tell pickle how to handle that data as documented in http://docs.python.org/library/copy_reg.html#module-copy_reg.
Either way what pickle needs is a function that will turn a blob it can't handle into (some_class, [arguments here]). And then when you unpickle it will construct a new thing of that class with those arguments.
Related
This question already has answers here:
Best method of saving data
(5 answers)
Closed 3 years ago.
I occasionally have Python programs that take a long time to run, and that I want to be able to save the state of and resume later. Does anyone have a clever way of saving the state either every x seconds, or when the program is exiting?
Put all of your "state" data in one place and use a pickle.
The pickle module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as “serialization”, “marshalling,” 1 or “flattening”, however, to avoid confusion, the terms used here are “pickling” and “unpickling”.
If you want to save everything, including the entire namespace and the line of code currently executing to be restarted at any time, there is not a standard library module to do that.
As another poster said, the pickle module can save pretty much everything into a file and then load it again, but you would have to specifically design your program around the pickle module (i.e. saving your "state" -- including variables, etc -- in a class).
If you ok with OOP, consider creating a method for each class that output a serialised version ( using pickle ) to file. Then add a second method to load in the instance the data, and if the pickled file is there you call the load method instead of the processing one.
I use this approach for ML and it really seed up my workflow.
In the traditional programming approach the obvious way to save a state of variables or objects after some point of execution is serialization.
So if you want to execute the program after some heavy already computed state we need to start only from the deserialization part.
These steps will be mostly needed mostly in data science modelling where we load a huge CSV or some other data and compute it and we do not want to recompute every time we run a program.
http://jupyter.org/ - A tool which does these serialization/deserialization automatically without you doing any manual work.
All you need to do is do execute a selected portion of your python code let us say line 10-15, which are dependent on pervious lines 1-9. Jupyter saves the state of 1-9 for you. Explore a tutorial it and give it a try.
This question already has answers here:
Best method of saving data
(5 answers)
Closed 3 years ago.
I occasionally have Python programs that take a long time to run, and that I want to be able to save the state of and resume later. Does anyone have a clever way of saving the state either every x seconds, or when the program is exiting?
Put all of your "state" data in one place and use a pickle.
The pickle module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as “serialization”, “marshalling,” 1 or “flattening”, however, to avoid confusion, the terms used here are “pickling” and “unpickling”.
If you want to save everything, including the entire namespace and the line of code currently executing to be restarted at any time, there is not a standard library module to do that.
As another poster said, the pickle module can save pretty much everything into a file and then load it again, but you would have to specifically design your program around the pickle module (i.e. saving your "state" -- including variables, etc -- in a class).
If you ok with OOP, consider creating a method for each class that output a serialised version ( using pickle ) to file. Then add a second method to load in the instance the data, and if the pickled file is there you call the load method instead of the processing one.
I use this approach for ML and it really seed up my workflow.
In the traditional programming approach the obvious way to save a state of variables or objects after some point of execution is serialization.
So if you want to execute the program after some heavy already computed state we need to start only from the deserialization part.
These steps will be mostly needed mostly in data science modelling where we load a huge CSV or some other data and compute it and we do not want to recompute every time we run a program.
http://jupyter.org/ - A tool which does these serialization/deserialization automatically without you doing any manual work.
All you need to do is do execute a selected portion of your python code let us say line 10-15, which are dependent on pervious lines 1-9. Jupyter saves the state of 1-9 for you. Explore a tutorial it and give it a try.
Python docs mention this word a lot and I want to know what it means.
It simply means it can be serialized by the pickle module. For a basic explanation of this, see What can be pickled and unpickled?. Pickling Class Instances provides more details, and shows how classes can customize the process.
Things that are usually not pickable are, for example, sockets, file(handler)s, database connections, and so on. Everything that's build up (recursively) from basic python types (dicts, lists, primitives, objects, object references, even circular) can be pickled by default.
You can implement custom pickling code that will, for example, store the configuration of a database connection and restore it afterwards, but you will need special, custom logic for this.
All of this makes pickling a lot more powerful than xml, json and yaml (but definitely not as readable)
These are all great answers, but for anyone who's new to programming and still confused here's the simple answer:
Pickling an object is making it so you can store it as it currently is, long term (to often to hard disk). A bit like Saving in a video game.
So anything that's actively changing (like a live connection to a database) can't be stored directly (though you could probably figure out a way to store the information needed to create a new connection, and that you could pickle)
Bonus definition: Serializing is packaging it in a form that can be handed off to another program. Unserializing it is unpacking something you got sent so that you can use it
Pickling is the process in which the objects in python are converted into simple binary representation that can be used to write that object in a text file which can be stored. This is done to store the python objects and is also called as serialization. You can infer from this what de-serialization or unpickling means.
So when we say an object is picklable it means that the object can be serialized using the pickle module of python.
This question already has answers here:
Best method of saving data
(5 answers)
Closed 3 years ago.
I occasionally have Python programs that take a long time to run, and that I want to be able to save the state of and resume later. Does anyone have a clever way of saving the state either every x seconds, or when the program is exiting?
Put all of your "state" data in one place and use a pickle.
The pickle module implements a fundamental, but powerful algorithm for serializing and de-serializing a Python object structure. “Pickling” is the process whereby a Python object hierarchy is converted into a byte stream, and “unpickling” is the inverse operation, whereby a byte stream is converted back into an object hierarchy. Pickling (and unpickling) is alternatively known as “serialization”, “marshalling,” 1 or “flattening”, however, to avoid confusion, the terms used here are “pickling” and “unpickling”.
If you want to save everything, including the entire namespace and the line of code currently executing to be restarted at any time, there is not a standard library module to do that.
As another poster said, the pickle module can save pretty much everything into a file and then load it again, but you would have to specifically design your program around the pickle module (i.e. saving your "state" -- including variables, etc -- in a class).
If you ok with OOP, consider creating a method for each class that output a serialised version ( using pickle ) to file. Then add a second method to load in the instance the data, and if the pickled file is there you call the load method instead of the processing one.
I use this approach for ML and it really seed up my workflow.
In the traditional programming approach the obvious way to save a state of variables or objects after some point of execution is serialization.
So if you want to execute the program after some heavy already computed state we need to start only from the deserialization part.
These steps will be mostly needed mostly in data science modelling where we load a huge CSV or some other data and compute it and we do not want to recompute every time we run a program.
http://jupyter.org/ - A tool which does these serialization/deserialization automatically without you doing any manual work.
All you need to do is do execute a selected portion of your python code let us say line 10-15, which are dependent on pervious lines 1-9. Jupyter saves the state of 1-9 for you. Explore a tutorial it and give it a try.
Python docs mention this word a lot and I want to know what it means.
It simply means it can be serialized by the pickle module. For a basic explanation of this, see What can be pickled and unpickled?. Pickling Class Instances provides more details, and shows how classes can customize the process.
Things that are usually not pickable are, for example, sockets, file(handler)s, database connections, and so on. Everything that's build up (recursively) from basic python types (dicts, lists, primitives, objects, object references, even circular) can be pickled by default.
You can implement custom pickling code that will, for example, store the configuration of a database connection and restore it afterwards, but you will need special, custom logic for this.
All of this makes pickling a lot more powerful than xml, json and yaml (but definitely not as readable)
These are all great answers, but for anyone who's new to programming and still confused here's the simple answer:
Pickling an object is making it so you can store it as it currently is, long term (to often to hard disk). A bit like Saving in a video game.
So anything that's actively changing (like a live connection to a database) can't be stored directly (though you could probably figure out a way to store the information needed to create a new connection, and that you could pickle)
Bonus definition: Serializing is packaging it in a form that can be handed off to another program. Unserializing it is unpacking something you got sent so that you can use it
Pickling is the process in which the objects in python are converted into simple binary representation that can be used to write that object in a text file which can be stored. This is done to store the python objects and is also called as serialization. You can infer from this what de-serialization or unpickling means.
So when we say an object is picklable it means that the object can be serialized using the pickle module of python.