I am training and storing logistic regression models in python3 using sklearn. For storing, I use the pickle module as show below
filename = 'models/logistic_regression/protocol_2/test/{}_pixels.p'.format(i//2)
pickle.dump(clf, open(filename, 'wb'), protocol=2)
Then, in another script, I am loading the models in python2. The loading is achieved with the following code
f = open(model_names[i//2-1], 'rb')
clf = pickle.load(f)
However, I get the error
ImportError: No module named _logistic
Could someone tell me why I cannot load the model? Thanks in advance
Related
I built one model with sklearn RandomForestClassifier in an old server and now I need to migrate it to another server. How can I transfer the model to the new server? Which Python package should I use? Pickle? joblib? Thanks!
Use "joblib".
Suppose your model is in a variable "my_model".
Then the 'joblib' code would go like this:
# On your development machine
from joblib import dump
dump(my_model, 'model.joblib')
# On your new machine, following code would go to load the model
from joblib import load
my_model = load('model.joblib')
Note: Replace "model.joblib" with path to the model.joblib file.
pickle is the way to go
from sklearn import model_selection
from sklearn.linear_model import LogisticRegression
import pickle
# Fit the model on training set
model = LogisticRegression()
model.fit(X_train, Y_train) # fit on some data ...
# save the model to disk
filename = 'finalized_model.sav'
pickle.dump(model, open(filename, 'wb'))
# load the model from disk
loaded_model = pickle.load(open(filename, 'rb'))
result = loaded_model.score(X_test, Y_test) # predict some test data
print(result)
I'm trying to open a pickled XGBoost model I created in AWS Sagemaker to look at feature importances in the model. I'm trying to follow the answers in this post. However, I get an the error shown below. When I try to call Booster.save_model, I get an error saying 'Estimator' object has no attribute 'save_model'. How can I resolve this?
# Build initial model
sess = sagemaker.Session()
s3_input_train = sagemaker.s3_input(s3_data='s3://{}/{}/train/'.format(bucket, prefix), content_type='csv')
xgb_cont = get_image_uri(region, 'xgboost', repo_version='0.90-1')
xgb = sagemaker.estimator.Estimator(xgb_cont, role, train_instance_count=1, train_instance_type='ml.m4.4xlarge',
output_path='s3://{}/{}'.format(bucket, prefix), sagemaker_session=sess)
xgb.set_hyperparameters(eval_metric='rmse', objective='reg:squarederror', num_round=100)
ts = strftime("%Y-%m-%d-%H-%M-%S", gmtime())
xgb_name = 'xgb-initial-' + ts
xgb.set_hyperparameters(eta=0.1, alpha=0.5, max_depth=10)
xgb.fit({'train': s3_input_train}, job_name=xgb_name)
# Load model to get feature importances
model_path = 's3://{}/{}//output/model.tar.gz'.format(bucket, prefix, xgb_name)
fs = s3fs.S3FileSystem()
with fs.open(model_path, 'rb') as f:
with tarfile.open(fileobj=f, mode='r') as tar_f:
with tar_f.extractfile('xgboost-model') as extracted_f:
model = pickle.load(extracted_f)
XGBoostError: [19:16:42] /workspace/src/learner.cc:682: Check failed: header == serialisation_header_:
If you are loading a serialized model (like pickle in Python) generated by older
XGBoost, please export the model by calling `Booster.save_model` from that version
first, then load it back in current version. There's a simple script for helping
the process. See:
https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html
for reference to the script, and more details about differences between saving model and
serializing.
Which version of XGBoost are you using in the notebook? The model format has changed in XGBoost 1.0. See https://xgboost.readthedocs.io/en/latest/tutorials/saving_model.html. Short version: if you're using 1.0 in the notebook, you can't load a pickled model.
Here's a working example using XGBoost in script mode (which is much more flexible than the built in algo):
https://gitlab.com/juliensimon/dlnotebooks/-/blob/master/sagemaker/09-XGBoost-script-mode.ipynb
https://gitlab.com/juliensimon/dlnotebooks/-/blob/master/sagemaker/xgb.py
I am using theano, sklearn and numpy in Python. I found this code for saving my trained network and predict on my new dataset in this link https://github.com/lzhbrian/RBM-DBN-theano-DL4J/blob/master/src/theano/code/logistic_sgd.py. the part of the code I am using is this :
"""
An example of how to load a trained model and use it
to predict labels.
"""
def predict():
# load the saved model
classifier = pickle.load(open('best_model.pkl'))
# compile a predictor function
predict_model = theano.function(
inputs=[classifier.input],
outputs=classifier.y_pred)
# We can test it on some examples from test test
dataset='mnist.pkl.gz'
datasets = load_data(dataset)
test_set_x, test_set_y = datasets[2]
test_set_x = test_set_x.get_value()
predicted_values = predict_model(test_set_x[:10])
print("Predicted values for the first 10 examples in test set:")
print(predicted_values)
if __name__ == '__main__':
sgd_optimization_mnist()
The code for the neural network model I want to save and load and predict with is https://github.com/aseveryn/deep-qa. I could save and load the model with cPickle but I continuously get errors in # compile a predictor function part:
predict_model = theano.function(inputs=[classifier.input],outputs=classifier.y_pred)
Actually I am not certain what I need to put in the inputs according to my code. Which one is right?
inputs=[main.predict_prob_batch.batch_iterator], outputs=test_nnet.layers[-1].
y_pred)
inputs=[predict_prob_batch.batch_iterator],
outputs=test_nnet.layers[-1].y_pred)
inputs=[MiniBatchIteratorConstantBatchSize.dataset],
outputs=test_nnet.layers[-1].y_pred)
inputs=[
sgd_trainer.MiniBatchIteratorConstantBatchSize.dataset],
outputs=test_nnet.layers[-1].y_pred)
or none of them???
Each of them I tried I got the errors:
ImportError: No module named MiniBatchIteratorConstantBatchSize
or
NameError: global name 'predict_prob_batch' is not defined
I would really appreciate if you could help me.
I also used these commands for running the code but still the errors.
python -c 'from run_nnet import predict; from sgd_trainer import MiniBatchIteratorConstantBatchSize; from MiniBatchIteratorConstantBatchSize import dataset; print predict()'
python -c 'from run_nnet import predict; from sgd_trainer import *; from MiniBatchIteratorConstantBatchSize import dataset; print predict()'
Thank you and let me know please if you know a better way to predict for new dataset on the loaded trained model.
Following the tutorial for sklearn, I attempted to save an object that was created via sklearn but was unsuccessful. It appears the problem is with the cross validation object, as I can save the actual (final) model.
Given:
rf_model = RandomForestRegressor(n_estimators=1000, n_jobs=4, compute_importances = False)
cvgridsrch = GridSearchCV(estimator=rf_model, param_grid=parameters,n_jobs=4)
cvgridsrch.fit(X,y)
This will succeed:
joblib.dump(cvgridsrch.best_estimator_, 'C:\\Users\\Desktop\\DMA\\cvgridsrch.pkl', compress=9)
and this will fail:
joblib.dump(cvgridsrch, 'C:\\Users\\Desktop\\DMA\\cvgridsrch.pkl', compress=9)
with error:
PicklingError: Can't pickle <type 'instancemethod'>: it's not found as __builtin__.instancemethod
How to save the full object?
If you are using Python 2,
try:
import dill
So that lambda functions can be pickled....
One possible cause could be multithreading issue, which you may refer to this stackoverflow answer.
Also, is it possible for you to dump your object not via joblib but a more fundamental method like pickle (and not even cPickle, which is more restrictive)?
I know this is an old question, but it might be useful for people coming here having the same, or similar, problem.
I'm not sure of the specific error message, but I managed to sucessfully save the entire GridSearchCV object in my own project by using pickle:
import pickle
gs = GridSearchCV(some parameters) #create the gridsearch object
gs.fit(X, y) # fit the model
with open('file_name', 'wb') as f:
pickle.dump(gs, f) # save the object to a file
Then you can use
with open('file_name', 'rb') as f:
gs = pickle.load(f)
to read the file and hence be able to use the object again.
How do I save a trained Naive Bayes classifier to disk and use it to predict data?
I have the following sample program from the scikit-learn website:
from sklearn import datasets
iris = datasets.load_iris()
from sklearn.naive_bayes import GaussianNB
gnb = GaussianNB()
y_pred = gnb.fit(iris.data, iris.target).predict(iris.data)
print "Number of mislabeled points : %d" % (iris.target != y_pred).sum()
Classifiers are just objects that can be pickled and dumped like any other. To continue your example:
import cPickle
# save the classifier
with open('my_dumped_classifier.pkl', 'wb') as fid:
cPickle.dump(gnb, fid)
# load it again
with open('my_dumped_classifier.pkl', 'rb') as fid:
gnb_loaded = cPickle.load(fid)
Edit: if you are using a sklearn Pipeline in which you have custom transformers that cannot be serialized by pickle (nor by joblib), then using Neuraxle's custom ML Pipeline saving is a solution where you can define your own custom step savers on a per-step basis. The savers are called for each step if defined upon saving, and otherwise joblib is used as default for steps without a saver.
You can also use joblib.dump and joblib.load which is much more efficient at handling numerical arrays than the default python pickler.
Joblib is included in scikit-learn:
>>> import joblib
>>> from sklearn.datasets import load_digits
>>> from sklearn.linear_model import SGDClassifier
>>> digits = load_digits()
>>> clf = SGDClassifier().fit(digits.data, digits.target)
>>> clf.score(digits.data, digits.target) # evaluate training error
0.9526989426822482
>>> filename = '/tmp/digits_classifier.joblib.pkl'
>>> _ = joblib.dump(clf, filename, compress=9)
>>> clf2 = joblib.load(filename)
>>> clf2
SGDClassifier(alpha=0.0001, class_weight=None, epsilon=0.1, eta0=0.0,
fit_intercept=True, learning_rate='optimal', loss='hinge', n_iter=5,
n_jobs=1, penalty='l2', power_t=0.5, rho=0.85, seed=0,
shuffle=False, verbose=0, warm_start=False)
>>> clf2.score(digits.data, digits.target)
0.9526989426822482
Edit: in Python 3.8+ it's now possible to use pickle for efficient pickling of object with large numerical arrays as attributes if you use pickle protocol 5 (which is not the default).
What you are looking for is called Model persistence in sklearn words and it is documented in introduction and in model persistence sections.
So you have initialized your classifier and trained it for a long time with
clf = some.classifier()
clf.fit(X, y)
After this you have two options:
1) Using Pickle
import pickle
# now you can save it to a file
with open('filename.pkl', 'wb') as f:
pickle.dump(clf, f)
# and later you can load it
with open('filename.pkl', 'rb') as f:
clf = pickle.load(f)
2) Using Joblib
from sklearn.externals import joblib
# now you can save it to a file
joblib.dump(clf, 'filename.pkl')
# and later you can load it
clf = joblib.load('filename.pkl')
One more time it is helpful to read the above-mentioned links
In many cases, particularly with text classification it is not enough just to store the classifier but you'll need to store the vectorizer as well so that you can vectorize your input in future.
import pickle
with open('model.pkl', 'wb') as fout:
pickle.dump((vectorizer, clf), fout)
future use case:
with open('model.pkl', 'rb') as fin:
vectorizer, clf = pickle.load(fin)
X_new = vectorizer.transform(new_samples)
X_new_preds = clf.predict(X_new)
Before dumping the vectorizer, one can delete the stop_words_ property of vectorizer by:
vectorizer.stop_words_ = None
to make dumping more efficient.
Also if your classifier parameters is sparse (as in most text classification examples) you can convert the parameters from dense to sparse which will make a huge difference in terms of memory consumption, loading and dumping. Sparsify the model by:
clf.sparsify()
Which will automatically work for SGDClassifier but in case you know your model is sparse (lots of zeros in clf.coef_) then you can manually convert clf.coef_ into a csr scipy sparse matrix by:
clf.coef_ = scipy.sparse.csr_matrix(clf.coef_)
and then you can store it more efficiently.
sklearn estimators implement methods to make it easy for you to save relevant trained properties of an estimator. Some estimators implement __getstate__ methods themselves, but others, like the GMM just use the base implementation which simply saves the objects inner dictionary:
def __getstate__(self):
try:
state = super(BaseEstimator, self).__getstate__()
except AttributeError:
state = self.__dict__.copy()
if type(self).__module__.startswith('sklearn.'):
return dict(state.items(), _sklearn_version=__version__)
else:
return state
The recommended method to save your model to disc is to use the pickle module:
from sklearn import datasets
from sklearn.svm import SVC
iris = datasets.load_iris()
X = iris.data[:100, :2]
y = iris.target[:100]
model = SVC()
model.fit(X,y)
import pickle
with open('mymodel','wb') as f:
pickle.dump(model,f)
However, you should save additional data so you can retrain your model in the future, or suffer dire consequences (such as being locked into an old version of sklearn).
From the documentation:
In order to rebuild a similar model with future versions of
scikit-learn, additional metadata should be saved along the pickled
model:
The training data, e.g. a reference to a immutable snapshot
The python source code used to generate the model
The versions of scikit-learn and its dependencies
The cross validation score obtained on the training data
This is especially true for Ensemble estimators that rely on the tree.pyx module written in Cython(such as IsolationForest), since it creates a coupling to the implementation, which is not guaranteed to be stable between versions of sklearn. It has seen backwards incompatible changes in the past.
If your models become very large and loading becomes a nuisance, you can also use the more efficient joblib. From the documentation:
In the specific case of the scikit, it may be more interesting to use
joblib’s replacement of pickle (joblib.dump & joblib.load), which is
more efficient on objects that carry large numpy arrays internally as
is often the case for fitted scikit-learn estimators, but can only
pickle to the disk and not to a string:
sklearn.externals.joblib has been deprecated since 0.21 and will be removed in v0.23:
/usr/local/lib/python3.7/site-packages/sklearn/externals/joblib/init.py:15:
FutureWarning: sklearn.externals.joblib is deprecated in 0.21 and will
be removed in 0.23. Please import this functionality directly from
joblib, which can be installed with: pip install joblib. If this
warning is raised when loading pickled models, you may need to
re-serialize those models with scikit-learn 0.21+.
warnings.warn(msg, category=FutureWarning)
Therefore, you need to install joblib:
pip install joblib
and finally write the model to disk:
import joblib
from sklearn.datasets import load_digits
from sklearn.linear_model import SGDClassifier
digits = load_digits()
clf = SGDClassifier().fit(digits.data, digits.target)
with open('myClassifier.joblib.pkl', 'wb') as f:
joblib.dump(clf, f, compress=9)
Now in order to read the dumped file all you need to run is:
with open('myClassifier.joblib.pkl', 'rb') as f:
my_clf = joblib.load(f)