I want to save a model that I have trained.
Since it is using shared variables (like Weights, bias and so on) and since it should be readable on machines without Theano installed, I wanted to use the theano.misc.pkl_utils.dump() function.
However, it seems as if that is only installed in bleeding edge installations (the current github file looks different than my local one).
Is that really the case? And why is the description in the docs then?
I am using theano 0.7.0 and I'm seriously confused about this.
If that feature is not yet available (I can't install bleeding edge right now), what are other ways? I'm sure that I am not the only one trying to save a trained model the easiest way possible ;-)
Thank you a lot,
Roman
If you train your model using Theano, the parameters of the model will be eventually shared variables (probably a list of shared variables if the network consists of several layers). It is possible to pickle the list of shared variables and unpickle it later. However you might have problems to unpickle such variables in another machine, e.g. with no Theano installation or if you train in a GPU-capable machine that generates CudaNdarrays and then you want to load back the model in a non-GPU-capable machine. What I recommend you is the following: convert every shared variable of the list of parameters into a numpy ndarray:
params_numpy = [numpy.asarray(p.get_value()) for p in params]
where params is a list of shared variables. Then you can safely pickle/unpickle params_numpy.
Related
For my use case I do not have the luxury of having access to the R version of the LightGBM package. I'd like to implement a trained model into a platform that can use R but does NOT have access to the LightGBM package.
For example, something analogous to this would be handy: https://cran.r-project.org/web/packages/gbm2sas/gbm2sas.pdf
Which one can basically creates a brute-force "if-else-then" script logic to aggregate up the marginal predictions into SAS code. Is there something like a "lightgbm2R" package out there that creates an "R script" for the "if-then-else" logic?
I've also looked into outputting the model into a JSON and importing it into R that way, too, but there doesn't seem to be a way to pass predictions into the model without access to LightGBM's predict function using the R's lightgbm package.
I do have access to R's gbm package, however. Is there a way this package could possibly import a LightGBM trained model file? The platform also has access to Python (but again no access to the LightGBM package). I know you can output the model file to a dataframe (i.e. "trees_to_dataframe()", but I have no idea how to pass the model features to this dataframe to get a prediction. If there's an example out there that shows how to do this (again, without leveraging the LightGBM package) that would also work.
I REALLY do not want to manually code in "if-then-else" logic for the tree. I was hoping there would be some sort of "standard" for exporting/importing serialized model objects across platforms (such as how JSON is a standard that is recognized across many platforms), but I can't seem to find such a thing for tree models.
In summary, the question is this: Is there a way to get a prediction from a LightGBM model file in either R or Python WITHOUT access to the LightGBM package that does not involve manually coding in the "if-then-else" tree logic?
Thank you for your time,
SYL
I'm implementing Memory Transformer and I need to keep the memory between calls to the model. I've tried to use tf.Variable for this task and it works perfectly on a single GPU.
But under MirroredStrategy on multiple GPUs this approach fails because mirrored strategy wants to synchronize the variables written on multiple replicas. Which is not needed in my case, I need to create a personal set of memory variables on each 'tower' like in TransformerXL implementation.
I think I can use with tf.device(): to create those variables but I'm not sure how to get current replica's device inside build method.
I know that it is possible, for example using TensorFlow but also in PyTorch or whatever, to store an instance of a trained (or in training) model in a way that it can be loaded in future, or loaded by another machine, or just to use it as a checkpoint during the training.
What I wonder is if there is any way, such as the above mentioned one, to store the difference (maybe not exactly the algebric subtraction but a similar concept, always referring to operation on tensors) between two instances of the same neural network (same architecture, different weights) for efficiency purposes.
If you are wondering why this should be convenient, consider an hypothetical setting where there are different entities and all of them know a model instance (a "shared model"), so using the "difference" calculated with respect to this shared model could be useful in terms of storage space or in terms of bandwidth (if the local model parameters should be sent via Internet to another machine).
The hypotesis is that it is possible to reconstruct a model knowing the shared model and the "difference" with the model to reconstruct.
Summarizing my questions:
There is any built-in features in TensorFlow, Pytorch, etc.. to do this?
It could be convenient in your opinion to do something like that? If not, why?
PS: In literature, this concept exists and it has been recently explored within the "Federated Learning" topic, and the "difference" I mentioned is called update.
I've been following this tutorial for creating LMDBs in Python. The code in the tutorial depends on Caffe, namely the caffe.proto.caffe_pb2.Datum() object for serializing the data.
However, those not using Caffe as a deep learning framework might find it tedious to install Caffe just for using the protobuf. What other ways exist for creating LMDBs?
LMDB is a general purpose DB that can be used for many applications in many ways. However, if you want to use it with caffe you are restricted to the interface written in caffe's "Data" layer. This interface is expecting a "Datum" elements in the LMDB. Other stored elements in the LMDB will simply results with a read error by caffe failing to interpret the stored elements.
Therefore, if you are going to use LMDB as an input for a caffe "Data" layer, you must store elements as "Datum" in the LMDB.
However, if you are going to use LMDB for other purposes, you may store any type of objects you wish, as long as you know how to read them correctly.
I've read caffe2 tutorials and tried pre-trained models. I knew caffe2 will leverge GPU to run the model/net. But the input data seems always be given from CPU(ie. Host) memory. For example, in Loading Pre-Trained Models, after model is loaded, we can predict an image by
result = p.run([img])
However, image "img" should be read in CPU scope. What I look for is a framework that can pipline the images (which is decoded from a video and still resides in GPU memory) directly to the prediction model, instead of copying it from GPU to CPU scope, and then transfering to GPU again to predict result. Is Caffe or Caffe2 provides such functions or interfaces for python or C++? Or should I need to patch Caffe to do so? Thanks at all.
Here is my solution:
I'd found in tensor.h, function ShareExternalPointer() can exactly do what I want.
Feed gpu data this way,
pInputTensor->ShareExternalPointer(pGpuInput, InputSize);
then run the predict net through
pPredictNet->Run();
where pInputTensor is the entrance tensor for the predict net pPredictNet
I don't think you can do it in caffe with python interface.
But I think that it can be accomplished using the c++: In c++ you have access to the Blob's mutable_gpu_data(). You may write code that run on device and "fill" the input Blob's mutable_gpu_data() directly from gpu. Once you made this update, caffe should be able to continue its net->forward() from there.
UPDATE
On Sep 19th, 2017 PR #5904 was merged into master. This PR exposes GPU pointers of blobs via the python interface.
You may access blob._gpu_data_ptr and blob._gpu_diff_ptr directly from python at your own risk.
As you've noted, using a Python layer forces data in and out of the GPU, and this can cause a huge hit to performance. This is true not just for Caffe, but for other frameworks too. To elaborate on Shai's answer, you could look at this step-by-step tutorial on adding C++ layers to Caffe. The example given should touch on most issues dealing with layer implementation. Disclosure: I am the author.