I want LSTM to learn with newer data. It needs to update itself depending on the trend in the new data and I wish to save this say in a file. Then I wish to call this pre-fed training file into any other X,Y,Z fresh files where testing is done. So I wish to 're-fit' [update NOT re-train] the model with new data such that model parameters are just updated and not re-initialized. I understand this is online learning but how to implement it through Keras? Can someone please advise how to successfully implement it?
Related
I have finished training a model, what I want to do next is hand over my model to my colleages who know nothing about deep learning, I just want to give them a function that they can run without installing tensorflow or Python on their machines, may be just python (ideally I would love it to run on Matlab). Is this doable? How can I abstract away all or codify everything from them?
I read about deployment of models but it's all about servers and stuff, this is not what I want.
PS: assume a TF model for now.
Basically, there's onnx that aims at assisting in deploying trained models.
I do not have much experience with it, but I know it's not always a straight forward procedure.
I have model saved in graph (.pb file). But now the model is inaccurate and I would like to develop it. I have pictures of additional data to learn, but I don't if it's possible or if it's how to do it? The result must be the modified of new data pb graph.
It's a good question. Actually it would be nice, if someone could explain how to do this. But in addition i can say you, that it would come to "catastrophic forgetting", so it wouldn't work out. You had to train all your data again.
But anyway, i also would like to know that espacially for ssd, just for test reasons.
The mozzila/DeepSpeech community has contributed a way to initialize training from a frozen graph(.pb). It does not restores optimizer parameters, so adjusting the learning rate is necessary.
You could find the code at:
https://github.com/mozilla/DeepSpeech/blob/master/DeepSpeech.py#L1562
Hope this helps!
I am new with machine learning and want to do following implementation
Want to create a custom .mlmodel with input of "xls or csv or nsdata of this files" and output should be double or array.
Pythone file should be able to read input data because i am going to use train_data from this input.
Pythone will do some calculation on this input data and provide prediction on this (i will do this calculation using sklearn,LinearRegression)
Can any one please help me how i can do this ?
You can use python to train your model with SKLearn as you suggested. This is a good post on getting started with that (make sure you use Sklearn and not Statsmodels).
https://towardsdatascience.com/simple-and-multiple-linear-regression-in-python-c928425168f9
When you have trained your model, you can convert it using Apple's coremltools:
https://github.com/apple/coremltools
When you've converted it you can add your .mlmodel file to your xcode project. You'll then need to write some code to get all of the your model inputs collected from your app and pass them as inputs to the model.
Good luck!
I'm a tensorflow beginner. So, excuse my question if it is stupied
I checked a github code for implementing CNN using MNIST data and tensorflow.
the link below:
https://github.com/aymericdamien/TensorFlow-Examples/blob/master/examples/3_NeuralNetworks/convolutional_network.py
However, I need to save the model generated by this code, but don't know how to do it, as this code does not involve the use of sessions, how to incoperate session on it?
Would appreciate your response.
The linked code is using tf.estimator.Estimator to train the model. Its documentation includes how to save the model using export_savedmodel. A saved model can be imported by specifying its location through the model_dir argument of the tf.estimator.Estimator initialiser.
How are machine learning models updated in web apps?
Take SKLearn for example, after training a huge model (let's say on 10gb of data) how might you update the model based on the current day's new data?
Presumably you wouldn't want to update real time, maybe something like once at the end of each day– but I can't find a way to do this in SKLearn. Do you just have to re-train the entire thing on the entire ever growing data set every day?
Number of estimators in sklearn implement partial_fit that allows incremental (online) learning. Check this article.