I want to compare a trained model in stable-baseline 3 (SB3) with another one ( a base algorithm) and see how it performs on the same episode. However I am having issues with the evaluate_policy function on the BasePolicy.
Here is a small reproducible example
import gym
from stable_baselines3 import SAC
from stable_baselines3.common.policies import BasePolicy
from stable_baselines3.common.evaluation import evaluate_policy
env = gym.make("Pendulum-v1")
env.seed(123456)
model = SAC("MlpPolicy", env, verbose=1)
model.learn(total_timesteps=1000, log_interval=4)
reward_list, episode_list = evaluate_policy(model, model.get_env(), n_eval_episodes=10, return_episode_rewards=True)
reward_list_base, episode_list_base = evaluate_policy(BasePolicy, model.get_env(), n_eval_episodes=10, return_episode_rewards=True)
I am getting an error on the last line TypeError: BasePolicy.predict() missing 1 required positional argument: 'self'. I would like to compare the trained policy with a 'Do Nothing' policy starting from the same initial state.
Additionally one issue with this case is every time I run the evaluate_policy on the trained model, the reward_list changes even though I have fixed the environment seed.
Any help in clarifying my doubts would be appreciated.
first, you're trying to pass a class object instead of an instance of a class. But even though it won't work because BasePolicy is an abstract class and you can't instantiate an object of this class. You need to define a child that implements _predict method.
well, SAC is stochastic in its nature meaning that its output is sampled based on probability distributions and couldn't be always the same.
Related
I need to set the attribute activation_out = 'logistic' in a MLPRegressor of sklearn. It is supposed that this attribute can take the names of the relevant activation functions ('relu','logistic','tanh' etc). The problem is that I cannot find the way that you can control this attribute and set it to the preferred functions. Please, if someone has faced this problem before or knows something more, I want some help.
I have tried to set attribute to MLPRegressor(), error. I have tried with the method set_params(), error. I have tried manually to change it through Variable Explorer, error. Finally, I used MLPName.activation_out = 'logistic' but again when I used fit() method it changed to 'identity'.
CODE:
X_train2, X_test2, y_train2,y_test2 =
train_test_split(signals_final,masks,test_size=0.05,random_state =
17)
scaler2 = MinMaxScaler()
X_train2 = scaler.fit_transform(X_train2)
X_test2 = scaler.transform(X_test2)
MatchingNetwork = MLPRegressor(alpha = 1e-15,hidden_layer_sizes=
(300,)
,random_state=1,max_iter=20000,activation='logistic',batch_size=64)
MLPRegressor().out_activation_ = 'logistic'
You cannot. The output activation is determined by the problem type at fit time. For regression, the identity activation is used; see the User Guide.
Here is the relevant bit of source code. You might be able to hack it by fitting one iteration, changing the attribute, then using partial_fit, since then this _initialize method won't be called again; but it's likely to break when back-propogating.
Generally I think the sklearn neural networks aren't designed to be super flexible: there are other packages that play that role, are more efficient (use GPUs), etc.
I've been following Apple's coremltools docs for converting PyTorch segmentation models to CoreML.
While it works fine when we're loading a remote PyTorch model, I'm yet to figure out a working Python script to perform conversions with local/already-downloaded PyTorch models.
The following piece of code throws a TypeError: 'dict' object is not callable
#This works fine: model = torch.hub.load('pytorch/vision:v0.6.0', 'deeplabv3_resnet101',pretrained=True).eval()
model = torch.load('local_model_file.pth')
input_tensor = preprocess(input_image)
input_batch = input_tensor.unsqueeze(0)
with torch.no_grad():
output = model(input_batch)['out'][0] #error here
torch_predictions = output.argmax(0)
There is a SO answer that offers a solution by initialising the model class and loading the state_dict, but I wonder what's the concrete solution when we don't have access to the PyTorch model?
In your code, model is a state dict, which is a dictionary from parameter names to the parameter tensor values. As the linked answer stated, the right way to load a state dict is by (a) creating the model object that the state dict belongs to and then (b) use nn.Module.load_state_dict to load the state dict. To do (a), you need access to the model's class definition. If you don't have that access, then unfortunately I don't see any reliable way to load the state dict.
You might be able to guess what the class' __init__ look like by looking at the parameter names in the state dict (e.g., 'module.stage1.rebnconvin.conv_s1.weight' looks like a convolution). However, even if the guess is correct and the state dict can be loaded, you still need to define the forward method because the state dict only stores the parameters.
I have been following along with this really helpful XGBoost tutorial on Medium (code used towards bottom of article): https://medium.com/analytics-vidhya/random-forest-and-xgboost-on-amazon-sagemaker-and-aws-lambda-29abd9467795.
To-date, I've been able to get data appropriately formatted for ML purposes, a model created based on training data, and then test data fed through the model to give useful results.
Whenever I leave and come back to work more on the model or feed in new test data however, I find I need to re-run all model creation steps in order to make any further predictions. Instead I would like to just call my already created model endpoint based on the Image_URI and feed in new data.
Current steps performed:
Model Training
xgb = sagemaker.estimator.Estimator(containers[my_region],
role,
train_instance_count=1,
train_instance_type='ml.m4.xlarge',
output_path='s3://{}/{}/output'.format(bucket_name, prefix),
sagemaker_session=sess)
xgb.set_hyperparameters(eta=0.06,
alpha=0.8,
lambda_bias=0.8,
gamma=50,
min_child_weight=6,
subsample=0.5,
silent=0,
early_stopping_rounds=5,
objective='reg:linear',
num_round=1000)
xgb.fit({'train': s3_input_train})
xgb_predictor = xgb.deploy(initial_instance_count=1,instance_type='ml.m4.xlarge')
Evaluation
test_data_array = test_data.drop([ 'price','id','sqft_above','date'], axis=1).values #load the data into an array
xgb_predictor.serializer = csv_serializer # set the serializer type
predictions = xgb_predictor.predict(test_data_array).decode('utf-8') # predict!
predictions_array = np.fromstring(predictions[1:], sep=',') # and turn the prediction into an array
print(predictions_array.shape)
from sklearn.metrics import r2_score
print("R2 score : %.2f" % r2_score(test_data['price'],predictions_array))
It seems that this particular line:
predictions = xgb_predictor.predict(test_data_array).decode('utf-8') # predict!
needs to be re-written in order to not reference xgb.predictor but instead reference the model location.
I have tried the following
trained_model = sagemaker.model.Model(
model_data='s3://{}/{}/output/xgboost-2020-11-10-00-00/output/model.tar.gz'.format(bucket_name, prefix),
image_uri='XXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/xgboost:latest',
role=role) # your role here; could be different name
trained_model.deploy(initial_instance_count=1, instance_type='ml.m4.xlarge')
and then replaced
xgb_predictor.serializer = csv_serializer # set the serializer type
predictions = xgb_predictor.predict(test_data_array).decode('utf-8') # predict!
with
trained_model.serializer = csv_serializer # set the serializer type
predictions = trained_model.predict(test_data_array).decode('utf-8') # predict!
but I get the following error:
AttributeError: 'Model' object has no attribute 'predict'
that's a good question :) I agree, many of the official tutorials tend to show the full train-to-invoke pipeline and don't emphasize enough that each step can be done separately. In your specific case, when you want to invoke an already-deployed endpoint, you can either: (A) use the invoke API call in one of the numerous SDKs (example in CLI, boto3) or (B) or instantiate a predictor with the high-level Python SDK, either the generic sagemaker.model.Model class or its XGBoost-specific child: sagemaker.xgboost.model.XGBoostPredictor as illustrated below:
from sagemaker.xgboost.model import XGBoostPredictor
predictor = XGBoostPredictor(endpoint_name='your-endpoint')
predictor.predict('<payload>')
similar question How to use a pretrained model from s3 to predict some data?
Note:
If you want the model.deploy() call to return a predictor, your model must be instantiated with a predictor_cls. This is optional, you can also first deploy a model, and then invoke it as a separate step with the above technique
Endpoints create charges even if you don't invoke them; they are charged per uptime. So if you don't need an always-on endpoint, don't hesitate to shut it down to minimize costs.
I train my classifier using DeepPavlov, and then when i call trained model for some sample function returns only one class label, but I want to get the probabilities of every class. I did not find function parameters that would allow me to get probabilities.
Has anyone encountered such a problem? Thank!
from deeppavlov import configs, train_model
model = train_model(configs.classifiers.intents_snips)
model(['Some sentence'])
I want the output like np.array with number of classes length, but current output is one label like ['PlayMusic'].
You can change chainer.out parameter of your config to be ["y_pred_probas"] before inferring, but it will also most likely require you to update change train.metrics if you want to train your model on the same config.
Alternatively you can call your model like
model.compute(['Some sentence'], targets=["y_pred_probas"])
And to get classes indexes you can run
dict(model['classes_vocab'])
I am trying to introduce a mod/mixin for a problem. In particular I am focusing here on a SpeechRecognitionProblem. I intend to modify this problem and therefore I seek to do the following:
class SpeechRecognitionProblemMod(speech_recognition.SpeechRecognitionProblem):
def hparams(self, defaults, model_hparams):
SpeechRecognitionProblem.hparams(self, defaults, model_hparams)
vocab_size = self.feature_encoders(model_hparams.data_dir)['targets'].vocab_size
p = defaults
p.vocab_size['targets'] = vocab_size
def feature_encoders(self, data_dir):
# ...
So this one does not do much. It calls the hparams() function from the base class and then changes some values.
Now, there are already some ready-to-go problems e.g. Libri Speech:
#registry.register_problem()
class Librispeech(speech_recognition.SpeechRecognitionProblem):
# ..
However, in order to apply my modifications I am doing this:
#registry.register_problem()
class LibrispeechMod(SpeechRecognitionProblemMod, Librispeech):
# ..
This should, if I am not mistaken, overwrite everything (with identical signatures) in Librispeech and instead call functions of SpeechRecognitionProblemMod.
Since I was able to train a model with this code I am assuming that it's working as intended so far.
Now here comes the my problem:
After training I want to serialize the model. This usually works. However, it does not with my mod and I actually know why:
At a certain point hparams() gets called. Debugging to that point will show me the following:
self # {LibrispeechMod}
self.hparams # <bound method SpeechRecognitionProblem.hparams of ..>
self.feature_encoders # <bound method SpeechRecognitionProblemMod.feature_encoders of ..>
self.hparams should be <bound method SpeechRecognitionProblemMod.hparams of ..>! It would seem that for some reason hparams() of SpeechRecognitionProblem gets called directly instead of SpeechRecognitionProblemMod. But please note that it's the correct type for feature_encoders()!
The thing is that I know this is working during training. I can see that the hyper-paramaters (hparams) are applied accordingly simply because the model's graph node names change through my modifications.
There is one specialty I need to point out. tensor2tensor allows to dynamically load a t2t_usr_dir, which are additional python modules which get loaded by import_usr_dir. I make use of that function in my serialization script as well:
if usr_dir:
logging.info('Loading user dir %s' % usr_dir)
import_usr_dir(usr_dir)
This could be the only culprit I can see at the moment although I would not be able to tell why this may cause the problem.
If anybody sees something I do not I'd be glad to get a hint what I'm doing wrong here.
So what is the error you're getting?
For the sake of completeness, this is the result of the wrong hparams() method being called:
NotFoundError (see above for traceback): Restoring from checkpoint failed.
Key transformer/symbol_modality_256_256/softmax/weights_0 not found in checkpoint
symbol_modality_256_256 is wrong. It should be symbol_modality_<vocab-size>_256 where <vocab-size> is a vocabulary size which gets set in SpeechRecognitionProblemMod.hparams.
So, this weird behavior came from the fact that I was remote debugging and that the source files of the usr_dir were not correctly synchronized. Everything works as intended but the source files where not matching.
Case closed.