I've been playing with the Tensorflow library doing the tutorials.
I'm using this example. And I changed the parameters in the example from this: n_classes = 15
to this: n_classes = 2 as I have only two classes to classify.
I read data like:
train = pandas.read_csv('tensorflow_feed/test/train_with_abs.csv', header=None)
X_train, y_train = train[1], train[0]
test = pandas.read_csv('tensorflow_feed/test/test_with_abs.csv', header=None)
X_test, y_test = test[1], test[0]
But it gives following error:
Total words: 35
Traceback (most recent call last):
File "/home/sumit/PycharmProjects/experiments/text_classification_save_restore.py", line 94, in <module>
classifier.fit(X_train, y_train)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/base.py", line 160, in fit
monitors=monitors)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 449, in _train_model
train_op, loss_op = self._get_train_ops(features, targets)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 673, in _get_train_ops
_, loss, train_op = self._call_model_fn(features, targets, ModeKeys.TRAIN)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/estimator.py", line 656, in _call_model_fn
features, targets, mode=mode)
File "/usr/local/lib/python2.7/dist-packages/tensorflow/contrib/learn/python/learn/estimators/base.py", line 369, in _model_fn
predictions, loss = model_fn(features, targets)
File "/home/sumit/PycharmProjects/experiments/text_classification_save_restore.py", line 73, in rnn_model
word_list = tf.unpack(word_vectors, axis=1)
TypeError: unpack() got an unexpected keyword argument 'axis'
Process finished with exit code 1
The "axis" parameter was just added to tf.unpack on June 23, and the example you're looking at was changed to use it:
https://github.com/tensorflow/tensorflow/commit/eff93149a6dc8e6826898fd9f9c28c81e21c9836
So I suggest either:
use an older version of the example from before that commit, e.g.:
https://github.com/tensorflow/tensorflow/blob/892ca4ddc12852a7b4633fd08f163941356cb4e6/tensorflow/examples/skflow/text_classification_save_restore.py
build a newer Tensorflow from github HEAD.
I hope that helps!
Related
I'd need to re-fit a scikit-learn pipeline using a smaller dataset, without some features that are actually not used by the model.
(The actual situation is that I'm saving it through joblib and loading it in another file where I need to re-fit is since it contains some custom transformers I made, but adding all features would be a pain since it's a different kind of model. However this is not important since the same error happens also if I re-fit the model before saving it in the same file where I first trained it).
This is my custom transformer:
class TransformAdoptionFeatures(BaseEstimator, TransformerMixin):
def __init__(self):
pass
def fit(self, X, y=None):
return self
def transform(self, X):
adoption_features = X.columns
feats_munic = [feat for feat in adoption_features if '_munic' in feat]
feats_adj_neigh = [feat for feat in adoption_features
if '_adj' in feat]
feats_port = [feat for feat in adoption_features if '_port' in feat]
feats_to_keep_all = feats_munic + feats_adj_neigh + feats_port
feats_to_keep = [feat for feat in feats_to_keep_all
if 'tot_cumul' not in feat]
return X[feats_to_keep]
And this is my pipeline:
full_pipeline = Pipeline([
('transformer', TransformAdoptionFeatures()),
('scaler', StandardScaler())
])
model = Pipeline([
("preparation", full_pipeline),
("regressor", ml_model)
])
Where ml_model is whichever scikit-learn machine learning model. Both the full_pipeline and the ml_model are already fitted when saving the model. (In the actual model there is a ColumnTransformer intermediate step that represent the actual full_pipeline, since I need to have different transformers for different columns, but I copied only the important one for brevity).
Issue: I reduced the number of features of the dataset I already used to fit everything, removing some features that are not considered in TransformAdoptionFeatures() (they do not get into the features to keep). Then, I tried to re-fit the model to the new dataset with reduced features and I got this error:
Traceback (most recent call last):
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\pandas\core\indexes\base.py", line 2889, in get_loc
return self._engine.get_loc(casted_key)
File "pandas\_libs\index.pyx", line 70, in pandas._libs.index.IndexEngine.get_loc
File "pandas\_libs\index.pyx", line 97, in pandas._libs.index.IndexEngine.get_loc
File "pandas\_libs\hashtable_class_helper.pxi", line 1675, in pandas._libs.hashtable.PyObjectHashTable.get_item
File "pandas\_libs\hashtable_class_helper.pxi", line 1683, in pandas._libs.hashtable.PyObjectHashTable.get_item
KeyError: 'tot_cumul_adoption_pr_y_munic'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\sklearn\utils\__init__.py", line 447, in _get_column_indices
col_idx = all_columns.get_loc(col)
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\pandas\core\indexes\base.py", line 2891, in get_loc
raise KeyError(key) from err
KeyError: 'tot_cumul_adoption_pr_y_munic'
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "C:\Users\giaco\sbp-abm\municipalities_abm\test.py", line 15, in <module>
modelSBP = model.SBPAdoption(initial_year=start_year)
File "C:\Users\giaco\sbp-abm\municipalities_abm\municipalities_abm\model.py", line 103, in __init__
self._upload_ml_models(ml_clsf_folder, ml_regr_folder)
File "C:\Users\giaco\sbp-abm\municipalities_abm\municipalities_abm\model.py", line 183, in _upload_ml_models
self._ml_clsf.fit(clsf_dataset.drop('adoption_in_year', axis=1),
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\sklearn\pipeline.py", line 330, in fit
Xt = self._fit(X, y, **fit_params_steps)
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\sklearn\pipeline.py", line 292, in _fit
X, fitted_transformer = fit_transform_one_cached(
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\joblib\memory.py", line 352, in __call__
return self.func(*args, **kwargs)
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\sklearn\pipeline.py", line 740, in _fit_transform_one
res = transformer.fit_transform(X, y, **fit_params)
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\sklearn\compose\_column_transformer.py", line 529, in fit_transform
self._validate_remainder(X)
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\sklearn\compose\_column_transformer.py", line 327, in _validate_remainder
cols.extend(_get_column_indices(X, columns))
File "C:\Users\giaco\anaconda3\envs\mesa_geo_ml\lib\site-packages\sklearn\utils\__init__.py", line 454, in _get_column_indices
raise ValueError(
ValueError: A given column is not a column of the dataframe
I do not understand what does this error is due to, I thought scikit-learn was not storing the name of the columns that I pass.
I found my error and it was actually in the use of the ColumnsTransformer, that is also the only place where the column names enter.
My error was really simple, I just did not update the list of the columns to apply each transformation to removing the names of the features excluded.
When using custom estimators in Tensorflow 2, when the model contains BatchNorm or Dropout layers, tf fails while building the graph with the following error. It works just fine when I comment out the Dropout and BatchNorm layers.
The model I use is a simple CNN model with two conv blocks and dense layer at the end:
def build_conv_block(x: Model, filter_map_count: int, name: str):
x = Conv2D(filter_map_count, (3, 3), name=f'{name}_conv_2d')(x)
x = BatchNormalization(name=f'{name}_bn')(x) <------- Error when not commented out
x = ReLU(name=f'{name}_relu')(x)
x = MaxPool2D((2, 2), name=f'{name}_max_pool_2d')(x)
x = Dropout(0.25, name=f'{name}_dropout')(x) <------- Error when not commented out
return x
def get_model(params):
input_image = Input(shape=params.input_shape)
x = build_conv_block(input_image, filter_map_count=64, name='layer_1')
x = build_conv_block(x, filter_map_count=128, name='layer_2')
x = Flatten(name='flatten_conv')(x)
output_pred = Dense(10, activation='softmax', name='output')(x)
model = Model(inputs=input_image, outputs=output_pred)
model.optimizer = Adam(learning_rate=params.learning_rate)
return model
I have a standard train_op in the model_fn that takes mnist images and labels as input and the class as output:
# Calculate gradients
with tf.GradientTape() as tape:
y_pred = model(features, training=training)
loss = tf.losses.categorical_crossentropy(labels, y_pred)
if mode == tf.estimator.ModeKeys.TRAIN:
gradients = tape.gradient(loss, model.trainable_variables)
train_op = model.optimizer.apply_gradients(zip(gradients, model.trainable_variables))
return tf.estimator.EstimatorSpec(mode, loss=loss, train_op=train_op)
Here's the traceback of the error I get:
Traceback (most recent call last):
File "F:/Projects/python/my_project/train.py", line 38, in <module>
tf.estimator.train_and_evaluate(estimator, train_spec=train_spec, eval_spec=eval_spec)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 473, in train_and_evaluate
return executor.run()
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 613, in run
return self.run_local()
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_estimator\python\estimator\training.py", line 714, in run_local
saving_listeners=saving_listeners)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 370, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1160, in _train_model
return self._train_model_default(input_fn, hooks, saving_listeners)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1190, in _train_model_default
features, labels, ModeKeys.TRAIN, self.config)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_estimator\python\estimator\estimator.py", line 1148, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "F:\Projects\python\my_project\model.py", line 62, in model_fn
gradients = tape.gradient(loss, model.trainable_variables)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\eager\backprop.py", line 1014, in gradient
unconnected_gradients=unconnected_gradients)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\eager\imperative_grad.py", line 76, in imperative_grad
compat.as_str(unconnected_gradients.value))
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\eager\backprop.py", line 138, in _gradient_function
return grad_fn(mock_op, *out_grads)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\ops\cond_v2.py", line 120, in _IfGrad
true_graph, grads, util.unique_grad_fn_name(true_graph.name))
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\ops\cond_v2.py", line 395, in _create_grad_func
func_graph=_CondGradFuncGraph(name, func_graph))
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\framework\func_graph.py", line 915, in func_graph_from_py_func
func_outputs = python_func(*func_args, **func_kwargs)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\ops\cond_v2.py", line 394, in <lambda>
lambda: _grad_fn(func_graph, grads), [], {},
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\ops\cond_v2.py", line 373, in _grad_fn
src_graph=func_graph)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\ops\gradients_util.py", line 550, in _GradientsHelper
gradient_uid)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\ops\gradients_util.py", line 175, in _DefaultGradYs
constant_op.constant(1, dtype=y.dtype, name="grad_ys_%d" % i)))
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 227, in constant
allow_broadcast=True)
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\framework\constant_op.py", line 265, in _constant_impl
allow_broadcast=allow_broadcast))
File "F:\Python\envs\tf2\lib\site-packages\tensorflow_core\python\framework\tensor_util.py", line 484, in make_tensor_proto
(dtype, nparray.dtype, values))
TypeError: Incompatible types: <dtype: 'variant'> vs. int32. Value is 1
It looks similar to the error mentioned in TF Issue #31894, but it doesn't seem to solve this problem. The TypeError does not tell much about where and why the error is happening and directly googling it does not help.
Although it may not be too obvious from the TypeError variant vs int32, if we carefully check the logs, we can see that the error occurs when finding gradients:
File "F:\Projects\python\my_project\model.py", line 62, in model_fn
gradients = tape.gradient(loss, model.trainable_variables)
Also, it should be noted that we get the same error even if one of them is present. So, if we try and analyze the common attributes in BatchNormalization and Dropout layer, both may seem to not come under the core layers, but when we look carefully, only those two layers in the model have a different train/test phase i.e. dropout doesn't zero out the values in test phase and batch norm uses a moving mean and variance during test phase.
Now the problem is narrowed down to using any layer that has a different train/test phase. This happens because tensorflow identifies if training mode is on or not using training parameter passed to the model.
This problem can be solved by using
y_pred = model(features, training=True)
when finding the gradients i.e. for the training phase and by using
y_pred = model(features, training=False)
otherwise i.e. for predict and eval phases.
Linked: Errors where moving mean is not updating is also reported, which can be solved by adding the same attribute.
I am new to CNN and machine learning in general and have been trying to follow Image Classification tutorial of TensorFlow.
Now, the Google Colab can be found here. I've followed the this official tutorial of TensorFlow. And I've changed it slightly so it saves the model as h5 instead of tf format so I can use the Keras' model.predict_classes.
Now, I have the model trained and the model reloaded from the saved model alright. But I'm repeatedly getting list index out of range error whenever I am trying to predict the image which I am doing so:
def predict():
image = tf.io.read_file('target.jpeg')
image = tf.image.decode_jpeg(image, channels=3)
image = tf.image.resize(image, [224, 224])
print(model.predict_classes(image)[0])
target.jpeg is one of the images I took from the flowers_photos dataset on which the model is trained.
The Traceback is:
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "<stdin>", line 7, in predict
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/sequential.py", line 319, in predict_classes
proba = self.predict(x, batch_size=batch_size, verbose=verbose)
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 821, in predict
use_multiprocessing=use_multiprocessing)
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 712, in predict
callbacks=callbacks)
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 187, in model_iteration
f = _make_execution_function(model, mode)
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training_arrays.py", line 555, in _make_execution_function
return model._make_execution_function(mode)
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 2037, in _make_execution_function
self._make_predict_function()
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/engine/training.py", line 2027, in _make_predict_function
**kwargs)
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3544, in function
return EagerExecutionFunction(inputs, outputs, updates=updates, name=name)
File "/home/amitjoki/.local/lib/python3.6/site-packages/tensorflow/python/keras/backend.py", line 3468, in __init__
self.outputs[0] = array_ops.identity(self.outputs[0])
IndexError: list index out of range
I searched extensively but couldn't get any solution. It would be helpful if anyone can point me in the direction of getting this up and running.
All the predict functions in Keras expect batch of inputs. Therefore, since you are doing prediction on one single image, you need to add an axis at the beginning of image tensor to represent the batch axis:
image = tf.expand_dims(image, axis=0) # the shape would be (1, 224, 224, 3)
print(model.predict_classes(image)[0])
I'm new to Python and TensorFlow and I'm trying to build a simple working example with fake data in TensorFlow. My goal is to use the DNNRegressor estimator to predict a real value from a multidimensional input. This is the code I wrote:
import pandas as pd
import tensorflow as tf
import numpy as np
# Amount of train samples
m_train = 1000
# Amount of test samples
m_test = 100
# Dimensions for each sample
n = 10
def from_dataset(ds):
return lambda: ds.make_one_shot_iterator().get_next()
# Create random samples with numpy
train_data = (np.random.sample((m_train,n)), np.random.sample((m_train,1)))
test_data = (np.random.sample((m_test,n)), np.random.sample((m_test,1)))
# Create two datasets, one for trainning and the other for testing
train_dataset = tf.data.Dataset.from_tensor_slices(train_data)
test_dataset = tf.data.Dataset.from_tensor_slices(test_data)
feature_columns = [tf.contrib.layers.real_valued_column("", dimension=n)]
model = tf.estimator.DNNRegressor(hidden_units=[20, 20], feature_columns=feature_columns)
# Train the model
model.train(input_fn=from_dataset(train_dataset), steps=1000)
# Evaluate the unseen samples
eval_result = model.evaluate(input_fn=from_dataset(test_dataset))
And this is the error I get:
$ python fake.py
WARNING:tensorflow:Using temporary folder as model directory: /tmp/tmp1j5irF
Traceback (most recent call last):
File "fake.py", line 28, in <module>
model.train(input_fn=from_dataset(train_dataset), steps=1000)
File "/usr/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 314, in train
loss = self._train_model(input_fn, hooks, saving_listeners)
File "/usr/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 743, in _train_model
features, labels, model_fn_lib.ModeKeys.TRAIN, self.config)
File "/usr/lib/python2.7/site-packages/tensorflow/python/estimator/estimator.py", line 725, in _call_model_fn
model_fn_results = self._model_fn(features=features, **kwargs)
File "/usr/lib/python2.7/site-packages/tensorflow/python/estimator/canned/dnn.py", line 448, in _model_fn
config=config)
File "/usr/lib/python2.7/site-packages/tensorflow/python/estimator/canned/dnn.py", line 153, in _dnn_model_fn
'Given type: {}'.format(type(features)))
ValueError: features should be a dictionary of `Tensor`s. Given type: <class 'tensorflow.python.framework.ops.Tensor'>
I supose I have to use a dictionary of Tensors, but I'm just beginning in Python and I don't know how to do it.
You need to return the iterator returned by get_one(), rather than a lambda function that returns the iterator. Check out https://github.com/tensorflow/tensorflow/blob/r1.8/tensorflow/examples/get_started/regression/dnn_regression.py
I do multi-class classification on unbalanced classes. I'm using SGDClassifier(), GradientBoostingClassifier(), RandomForestClassifier(), and LogisticRegression()with class_weight='balanced'. To compare the results. it is required to compute the accuracy. I tried the following way to compute weighted accuracy:
n_samples = len(y_train)
weights_cof = float(n_samples)/(n_classes*np.bincount(data[target_label].as_matrix().astype(int))[1:])
sample_weights = np.ones((n_samples,n_classes)) * weights_cof
print accuracy_score(y_test, y_pred, sample_weight=sample_weights)
y_train is a binary array. So sample_weights has the same shape as y_train (n_samples, n_classes). When I run the script, I received the following error:
Update:
Traceback (most recent call last):
File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.3.2\helpers\pydev\pydevd.py", line 1596, in <module>
globals = debugger.run(setup['file'], None, None, is_module)
File "C:\Program Files (x86)\JetBrains\PyCharm Community Edition 2016.3.2\helpers\pydev\pydevd.py", line 974, in run
pydev_imports.execfile(file, globals, locals) # execute the script
File "D:/Destiny/DestinyScripts/MainLocationAware.py", line 424, in <module>
predict_country(featuresDF, score, featuresLabel, country_sample_size, 'gbc')
File "D:/Destiny/DestinyScripts/MainLocationAware.py", line 313, in predict_country
print accuracy_score(y_test, y_pred, sample_weight=sample_weights)
File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\metrics\classification.py", line 183, in accuracy_score
return _weighted_sum(score, sample_weight, normalize)
File "C:\ProgramData\Anaconda2\lib\site-packages\sklearn\metrics\classification.py", line 108, in _weighted_sum
return np.average(sample_score, weights=sample_weight)
File "C:\ProgramData\Anaconda2\lib\site-packages\numpy\lib\function_base.py", line 1124, in average
"Axis must be specified when shapes of a and weights "
TypeError: Axis must be specified when shapes of a and weights differ.
The error would seem to suggest that the shape of your sample_weights and your y_test/y_pred arrays differ. Basically the method creates a boolean array with y_test == y_pred and passes that along with sample_weights to np.average. One of the first checks in that method is to ensure that the entered array and the weights are the same shape, which apparently in this case they are not.
Update
Your comment "sample_weights, y_test, and y_pred have the same shape (n_samples, n_classes)" exposes the issue. According to the documentation for accuracy_score, y_pred and y_true (in your case y_test and y_pred) should be 1 dimensional. Are you perhaps using one hot encoded labels? If so you should convert them to single value labels and then try the accuracy score again.