OpenAI's baselines use the following code to return a LazyFrames instead of a concatenated numpy array to save memory. The idea is to take the advantage of the fact that a numpy array can be saved at different lists at the same time as lists only save a reference not object itself. However, in the implementation of LazyFrames, it further saves the concatenated numpy array in self._out, in that case if every LazyFrames object has been invoked at least once, it will always save a concatenated numpy array within it, which does not seem to save any memory at all. Then what's the point of LazeFrames? Or do I misunderstand anything?
class FrameStack(gym.Wrapper):
def __init__(self, env, k):
"""Stack k last frames.
Returns lazy array, which is much more memory efficient.
See Also
--------
baselines.common.atari_wrappers.LazyFrames
"""
gym.Wrapper.__init__(self, env)
self.k = k
self.frames = deque([], maxlen=k)
shp = env.observation_space.shape
self.observation_space = spaces.Box(low=0, high=255, shape=(shp[:-1] + (shp[-1] * k,)), dtype=env.observation_space.dtype)
def reset(self):
ob = self.env.reset()
for _ in range(self.k):
self.frames.append(ob)
return self._get_ob()
def step(self, action):
ob, reward, done, info = self.env.step(action)
self.frames.append(ob)
return self._get_ob(), reward, done, info
def _get_ob(self):
assert len(self.frames) == self.k
return LazyFrames(list(self.frames))
class LazyFrames(object):
def __init__(self, frames):
"""This object ensures that common frames between the observations are only stored once.
It exists purely to optimize memory usage which can be huge for DQN's 1M frames replay
buffers.
This object should only be converted to numpy array before being passed to the model.
You'd not believe how complex the previous solution was."""
self._frames = frames
self._out = None
def _force(self):
if self._out is None:
self._out = np.concatenate(self._frames, axis=-1)
self._frames = None
return self._out
def __array__(self, dtype=None):
out = self._force()
if dtype is not None:
out = out.astype(dtype)
return out
def __len__(self):
return len(self._force())
def __getitem__(self, i):
return self._force()[i]
def count(self):
frames = self._force()
return frames.shape[frames.ndim - 1]
def frame(self, i):
return self._force()[..., I]
I actually came here trying to understand how this saved any memory at all! But you mention that the lists store references to the underlying data, while the numpy arrays store copies of that data, and I think you are correct about that.
To answer your question, you are right! When _force is called, it populates the self._out item with a numpy array, thereby expanding the memory. But until you call _force (which is called in any of the API functions of the LazyFrame), self._out is None. So the idea is to not call _force (and therefore, don't call any of the LazyFrames methods) until you need the underlying data, hence the warning in its doc string of "This object should only be converted to numpy array before being passed to the model".
Note that when self._out gets populated by the array, it also clears the self._frames, so that it doesn't store duplicate information (thereby harming its whole purpose of storing only as much as it needs).
Also, in the same file, you'll find the ScaledFloatFrame which carries this doc string:
Scales the observations by 255 after converting to float.
This will undo the memory optimization of LazyFrames,
so don't use it with huge replay buffers.
Related
I have created an immutable data type in Python where millions of objects are created and released every second. After profiling the code thoroughly, it looks like the constructor is where most of the time is spent.
The solution I thought of was to use an object pool so that reference counting and memory allocation is done all at once. I have looked at solutions like this, where acquire and release methods need to be called explicitly.
However, the class I have implemented is similar to Decimal class in Python where objects are created and released automatically by numpy. For example, a short piece of code that uses my class will look like this (I use Decimal instead of my own class):
import numpy as np
from decimal import Decimal
x = np.array([[Decimal(1), Decimal(2)], [Decimal(3), Decimal(4)]]
y = np.array([[Decimal(5), Decimal(6)], [Decimal(7), Decimal(8)]]
z = (x * y) + (2 * x) - (y ** 2) + (x ** 3)
Because the class is immutable, numpy needs to create a new object for each operation and this slows down the whole code. Additionally, because numpy is the code that is creating these objects, I do not think that I can explicitly call methods such as acquire or release.
Is there a better implementation of an object pool or some other method where a lot of objects are created all at once and later, the released objects are automatically placed back in the pool? In other words, is there another solution that avoids frequent object creation and destruction?
P.S. I understand that this is not a good way of using numpy. This is one of the first steps in my design and hopefully, numpy will be used more efficiently in next steps.
Would something like this work?
class Pool():
def __init__(self, type_, extra_alloc=1):
self._objects = []
self.type = type_
self.extra_alloc = extra_alloc
def allocate(self, size):
self._objects.extend(object.__new__(self.type) for _ in range(size))
def get_obj(self):
print("Getting object")
if not self._objects:
self.allocate(self.extra_alloc)
return self._objects.pop()
def give_obj(self, obj):
print("Object released")
self._objects.append(obj)
class Thing(): # This can also be used as a base class
pool = None
def __new__(self, *args):
return self.pool.get_obj()
def __del__(self):
self.pool.give_obj(self)
thing_pool = Pool(Thing)
Thing.pool = thing_pool
Thing()
# Getting object
# Object released
x = Thing()
# Getting object
del x
# Object released
In the snippet below, how do I avoid computing the following numpy variables mask, zbar, te , ro and rvol in the procedures Get_mask, Get_K_Shell_Te etc? These variables are large arrays and I have to define at least six more procedures that reuse them. It looks like what I am doing is not a good idea and is slow.
import numpy as np
# this computes various quantities related to the shell in a object oriented way
class Shell_Data:
def __init__(self, data):
self.data = data
def Get_mask(self):
zbar=self.data['z2a1']
y=self.data['y']*1000
mask= np.logical_and(zbar >= 16 ,zbar<= 19 )
return self.mask
def Get_Shell_Te(self):
self.mask =self.Get_mask()
te =self.data['te'][self.mask]
ro =self.data['ro'][self.mask]
rvol =self.data['rvol'][self.mask]
self.Shell_Te=np.sum(te*ro/rvol)/(np.sum(ro/rvol))
print "Shell Temperature= %0.3f eV" % (self.Shell_Te)
return self.Shell_Te
def Get_Shell_ro(self):
mask =self.Get_mask()
te =self.data['te'][mask]
ro =self.data['ro'][mask]
rvol =self.data['rvol'][mask]
radk =self.data['radk'][mask]
self.Shell_ro=np.sum(radk*ro/rvol)/np.sum(radk/rvol)
return self.Shell_ro
zbar depends on self.data. If you update self.data, you likely have to re-compute it.
If you can make your data immutable, you can compute these values once, e.g. in the constructor.
If you want to avoid calculating the mask data until it's actually required, you can cache the value, like so:
class Shell_Data(...):
def __init__(self,...):
self.cached_mask = None
...
# #property makes an access to self.mask
# to actually return the result of a call to self.mask()
#property
def mask(self):
if self.cached_mask is None: # Not yet calculated.
self.cached_mask = self.computeMask()
return self.cached_mask
def computeMask(self):
zbar = ...
...
return np.logical_and(...)
def someComputation(self):
# The first access to self.mask will compute it.
te = self.data['te'][self.mask]
# The second access will just reuse the same result.
ro = self.data['ro'][self.mask]
...
If you have to mutate self.data, you can cache the computed data, and re-calculate it only when self.data changes. E.g. if you had a setData() method for that, you could recalculate the mask in it, or just set self.cached_mask to None.
(Also, read about instance variables again.
Every method receives the parameter named self, the instance of the object for which it is called. You can access all its instance variables as self.something, exactly the way you access instance variables (and methods) when an object is not called self. If you set an instance variable in one method, you can just access it an another (e.g. self.mask), no need to return it. If you return something from a method, likely it's not worth storing as an instance variable, like self.mask.)
My code is something like this:
def something():
"""this function will call the `get_array` multiple times until the
end of the array and is fixed (cannot change this function)"""
#skip some functions and initialization there
#only used to get the result of `get_array`
a.get_array()
In a separate file:
class a:
def __init__(self, count=-1):
self.count = count
def cal_array(self, parameters):
"""calculate array"""
return array
def get_array(self, parameters):
if self.count == -1:
self.cal_array(parameters)
# some methods to store array there
else:
self.count += 1
return array[self.count,:]
I want to know if there is a way to calculate the numpy array once and save it for future use (calling from another function).
I know I can save the numpy array to the disk or use a global dict to save the array, but I wonder if there is a more efficient way to do this.
I am quite new to Python and I have been facing a problem for which I could not find a direct answer here on stackoverflow (but I guess I am just not experienced enough to google for the correct terms). I hope you can help 😊
Consider this:
import numpy as np
class Data:
def __init__(self, data):
self.data = data
def get_dimensions(self):
return np.shape(self.data)
test = Data(np.random.random((20, 15)))
print(test.get_dimensions())
This gives me
(20, 15)
just as I wanted.
Now here is what I want to do:
During my data processing I will need to get the shape of my datasets quite often, especially within the class itself. However, I do not want to call numpy every time I do
self.get_dimensions()
as I think this would always go though the process of analysing the array. Is there a way to calculate the shape variable just once and then share it within the class so I save computation time?
My Problem is more complicated, as I need to first open files, read them and the from this get the shape of the data, so I really want to avoid doing this every time I want to get the shape...
I hope you see my problem
thanks!! 😊
EDIT:
My question has already been answered, however I wanted to ask a follow up question if this would also be efficient:
import numpy as np
class Data:
def __init__(self, data):
self.data = data
self.dimensions = self._get_dimensions()
def _get_dimensions(self):
return np.shape(self.data)
test = Data(np.random.random((20, 15)))
print(test.dimensions)
I ask this, because with the method you guys described I have to calculate it at least once somewhere, before I can get the dimensions. Would this way also always go through the calculation process, or store it just once?
Thanks again! 😊
Sure, you could do it like this:
class Data:
def __init__(self, data):
self.data = data
self.dimensions = None
def get_dimensions(self):
self.dimensions = (np.shape(self.data) if
self.dimensions is None else
self.dimensions)
return self.dimensions
If you ever need to modify self.data and recalculate self.dimensions, you could be served better with a keyword argument to specify whether you'd like to recalculate the result. Ex:
def get_dimensions(self, calculate=False):
self.dimensions = (np.shape(self.data)
if calculate or self.dimensions is None
else self.dimensions)
return self.dimensions
You can cache the result as a member variable (if I understood the question correctly):
import numpy as np
class Data:
def __init__(self, data):
self.data = data
self.result = None
def get_dimensions(self):
if not self.result:
self.result = np.shape(self.data)
return self.result
test = Data(np.random.random((20, 15)))
print(test.get_dimensions())
The shape of an array is directly stored on an array and is not a computed value. The shape of the array has to stored as the backing memory is a flat array. Thus (4, 4), (2, 8) and (16,) would have the same backing array. Without storing the shape, the array cannot perform indexing operations. numpy.shape is only really useful for acquiring the shape of array-like objects (such as lists or tuples).
shape = self.data.shape
I missed the last bit where you were concerned about some other large expensive computation that you haven't shown. The best solution is to cache the computed value the first time and return the cached value on later method calls.
To cope with additional computation
from random import random
class Data:
#property
def dimensions(self):
# Do a try/except block as the exception will only every be thrown the first
# time. Secondary invocations will work quicker and not require any checking.
try:
return self._dimensions
except AttributeError:
pass
# some complex computation
self._dimensions = random()
return self._dimensions
d = Data()
assert d.dimensions == d.dimensions
if your dimension is not changing you can do it more simple.
import numpy as np
class Data:
def __init__(self, data):
self.data = data
self.dimension=np.shape(self.data)
def get_dimensions(self):
return self.dimension
test = Data(np.random.random((20, 15)))
print(test.get_dimensions())
I am developing an application where I have defined a "variable" object containing data in the form of a numpy array. These variables are linked to (netcdf) data files, and I would like to dynamically load the variable values when needed instead of loading all data from the sometimes huge files at the start.
The following snippet demonstrates the principle and works well, including access to data portions with slices. For example, you can write:
a = var() # empty variable
print a.values[7] # values have been automatically "loaded"
or even:
a = var()
a[7] = 0
However, this code still forces me to load the entire variable data at once. Netcdf (with the netCDF4 library) would allow me to directly access data slices from the file. Example:
f = netCDF4.Dataset(filename, "r")
print f.variables["a"][7]
I cannot use the netcdf variable objects directly, because my application is tied to a web service which cannot remember the netcdf file handler, and also because the variable data don't always come from netcdf files, but may originate from other sources such as OGC web services.
Is there a way to "capture" the slicing expression in the property or setter methods and use them? The idea would be to write something like:
#property
def values(self):
if self._values is None:
self._values = np.arange(10.)[slice] # load from file ...
return self._values
instead of the code below.
Working demo:
import numpy as np
class var(object):
def __init__(self, values=None, metadata=None):
if values is None:
self._values = None
else:
self._values = np.array(values)
self.metadata = metadata # just to demonstrate that var has mor than just values
#property
def values(self):
if self._values is None:
self._values = np.arange(10.) # load from file ...
return self._values
#values.setter
def values(self, values):
self._values = values
First thought: Should I perhaps create values as a separate class and then use __getitem__? See In python, how do I create two index slicing for my own matrix class?
No, you cannot detect what will be done to the object after returning from .values. The result could be stored in a variable and only (much later on) be sliced, or sliced in different places, or used in its entirety, etc.
You indeed should instead return a wrapper object and hook into object.__getitem__; it would let you detect slicing and load data as needed. When slicing, Python passes in a slice() object.
Thanks to the guidance of Martijn Pieters and with a bit more reading, I came up with the following code as demonstration. Note that the Reader class uses a netcdf file and the netCDF4 library. If you want to try out this code yourself you will either need a netcdf file with variables "a" and "b", or replace Reader with something else that will return a data array or a slice from a data array.
This solution defines three classes: Reader does the actual file I/O handling, Values manages the data access part and invokes a Reader instance if no data have been stored in memory, and var is the final "variable" which in real life will contain a lot more metadata. The code contains a couple of extra print statements for educational purposes.
"""Implementation of a dynamic variable class which can read data from file when needed or
return the data values from memory if they were read already. This concepts supports
slicing for both memory and file access."""
import numpy as np
import netCDF4 as nc
FILENAME = r"C:\Users\m.schultz\Downloads\data\tmp\MACC_20141224_0001.nc"
VARNAME = "a"
class Reader(object):
"""Implements the actual data access to variable values. Here reading a
slice from a netcdf file.
"""
def __init__(self, filename, varname):
"""Final implementation will also have to take groups into account...
"""
self.filename = filename
self.varname = varname
def read(self, args=slice(None, None, None)):
"""Read a data slice. Args is a tuple of slice objects (e.g.
numpy.index_exp). The default corresponds to [:], i.e. all data
will be read.
"""
with nc.Dataset(self.filename, "r") as f:
values = f.variables[self.varname][args]
return values
class Values(object):
def __init__(self, values=None, reader=None):
"""Initialize Values. You can either pass numerical (or other) values,
preferrably as numpy array, or a reader instance which will read the
values on demand. The reader must have a read(args) method, where
args is a tuple of slices. If no args are given, all data should be
returned.
"""
if values is not None:
self._values = np.array(values)
self.reader = reader
def __getattr__(self, name):
"""This is only be called if attribute name is not present.
Here, the only attribute we care about is _values.
Self.reader should always be defined.
This method is necessary to allow access to variable.values without
a slicing index. If only __getitem__ were defined, one would always
have to write variable.values[:] in order to make sure that something
is returned.
"""
print ">>> in __getattr__, trying to access ", name
if name == "_values":
print ">>> calling reader and reading all values..."
self._values = self.reader.read()
return self._values
def __getitem__(self, args):
print "in __getitem__"
if not "_values" in self.__dict__:
values = self.reader.read(args)
print ">>> read from file. Shape = ", values.shape
if args == slice(None, None, None):
self._values = values # all data read, store in memory
return values
else:
print ">>> read from memory. Shape = ", self._values[args].shape
return self._values[args]
def __repr__(self):
return self._values.__repr__()
def __str__(self):
return self._values.__str__()
class var(object):
def __init__(self, name=VARNAME, filename=FILENAME, values=None):
self.name = name
self.values = Values(values, Reader(filename, name))
if __name__ == "__main__":
# define a variable and access all data first
# this will read the entire array and save it in memory, so that
# subsequent access with or without index returns data from memory
a = var("a", filename=FILENAME)
print "1: a.values = ", a.values
print "2: a.values[-1] = ", a.values[-1]
print "3: a.values = ", a.values
# define a second variable, where we access a data slice first
# In this case the Reader only reads the slice and no data are stored
# in memory. The second access indexes the complete array, so Reader
# will read everything and the data will be stored in memory.
# The last access will then use the data from memory.
b = var("b", filename=FILENAME)
print "4: b.values[0:3] = ", b.values[0:3]
print "5: b.values[:] = ", b.values[:]
print "6: b.values[5:8] = ",b.values[5:8]