Develop Reference

How to migrate from keras.engine.[...] to tf.keras.engine?

As discussed in keras vs. tensorflow.python.keras - which one to use? I generally do not want to include keras in my machine learning project. Yet in the following code example using keras works and tf.keras does not:
>>>import keras
>>>print(keras.engine.keras_tensor.KerasTensor)
<class 'keras.engine.keras_tensor.KerasTensor'>
and
>>>import tensorflow as tf
>>>print(tf.keras.engine.keras_tensor.KerasTensor)
Traceback (most recent call last):
File "c:\Users\reifv\root\Heidelberg Master\Netflix_AI_codes\working env - Copy\Netflix_AI\main copy.py", line 4, in <module>
print(tf.keras.engine.keras_tensor.KerasTensor)
File "C:\Users\reifv\root\Heidelberg Master\Netflix_AI_codes\working env - Copy\Netflix_AI\venvnf\lib\site-packages\tensorflow\python\util\lazy_loader.py", line 59, in __getattr__
return getattr(module, item)
AttributeError: module 'keras.api._v2.keras' has no attribute 'engine'
This is one example of tf.keras not working, but I encountered this issue several times along my project. Therefore, a general solution would be much appreciated, too.

Pipeline error: "AttributeError: 'ColumnTransformer' object has no attribute '_feature_names_in'"

I am trying to use scikitlearn to predict over new data using a pipeline object I had trained back in February. Since Friday, February 28th, the predict function no longer works for my pipeline object, citing the error:
>>> df = pd.read_csv('test_df_for_example.csv')
>>> mdl = joblib.load('split_0_model.pkl')
>>> mdl.predict(df)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/utils/metaestimators.py", line 116, in <lambda>
out = lambda *args, **kwargs: self.fn(obj, *args, **kwargs)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/pipeline.py", line 419, in predict
Xt = transform.transform(Xt)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/compose/_column_transformer.py", line 587, in transform
self._validate_features(X.shape[1], X_feature_names)
File "/Library/Frameworks/Python.framework/Versions/3.6/lib/python3.6/site-packages/sklearn/compose/_column_transformer.py", line 411, in _validate_features
if ((self._feature_names_in is None or feature_names is None)
AttributeError: 'ColumnTransformer' object has no attribute '_feature_names_in'
I am using Microsoft Azure's virtual machines to do this predicting (although the above code I ran on my local computer), so working with the versioning of the modules is difficult, and most of the time I am forced to use the latest versions of packages. I believe this error comes from scikitlearn's new version 0.22.2.post1, which I am using.
I have an example CSV with testing data here
The model file pickled with joblib here
And code to reproduce the error here
And yaml environment file here
Is there any way I can upgrade my model so that this error does not occur?
Thanks!
Kristine

I recommend pinning down versions in your YAML, especially with the speed of releases in the azureml space.
So downgrading sklearn to the last stable build for your use case may be the solution, or upgrading the rest of your code base to accommodate the new sklearn version.
Ex.
- pip:
- sklearn==0.20.0
- azureml-sdk==1.0.85
- etc...

Thanks to Nema, using the following specifications I'm able to use the previous scikit-learn version to load my model:
- scikit-learn<=0.21.3
- azureml-sdk<=1.0.83

Unable to load a pretrained model

After training my model for almost 2 days 3 files were generated:
best_model.ckpt.data-00000-of-00001
best_model.ckpt.index
best_model.ckpt.meta
where best_model is my model name.
When I try to import my model using the following command
with tf.Session() as sess:
saver = tf.train.import_meta_graph('best_model.ckpt.meta')
saver.restore(sess, "best_model.ckpt")
I get the following error
Traceback (most recent call last):
File "<stdin>", line 2, in <module>
File "/home/shreyash/.local/lib/python2.7/site-
packages/tensorflow/python/training/saver.py", line 1577, in
import_meta_graph
**kwargs)
File "/home/shreyash/.local/lib/python2.7/site-
packages/tensorflow/python/framework/meta_graph.py", line 498, in import_scoped_meta_graph
producer_op_list=producer_op_list)
File "/home/shreyash/.local/lib/python2.7/site-packages/tensorflow/python/framework/importer.py", line 259, in import_graph_def
raise ValueError('No op named %s in defined operations.' % node.op)
ValueError: No op named attn_add_fun_f32f32f32 in defined operations.
How to fix this?
I have referred this post: TensorFlow, why there are 3 files after saving the model?
Tensorflow version 1.0.0 installed using pip
Linux version 16.04
python 2.7

The importer can't find a very specific function in your graph, namely attn_add_fun_f32f32f32, which is likely to be one of attention functions.
Probably you've stepped into this issue. However, they say it's bundled in tensorflow 1.0. Double check that installed tensorflow version contains attention_decoder_fn.py (or, if you are using another library, check that it's there).
If it's there, here are your options:
Rename this operation, if possible. You might want to read this discussion for workarounds.
Duplicate your graph definition, so that you won't have to call import_meta_graph, but restore the model into the current graph.

Code for gensim Word2vec as an HTTP service 'KeyedVectors' Attribute error

I am using the w2v_server_googlenews code from the word2vec HTTP server running at https://rare-technologies.com/word2vec-tutorial/#bonus_app. I changed the loaded file to a file of vectors trained with the original C version of word2vec. I load the file with
gensim.models.KeyedVectors.load_word2vec_format(fname, binary=True)
and it seems to load without problems. But when I test the HTTP service with, let's say
curl 'http://127.0.0.1/most_similar?positive%5B%5D=woman&positive%5B%5D=king&negative%5B%5D=man'
I got an empty result with only the execution time.
{"taken": 0.0003361701965332031, "similars": [], "success": 1}
I put a traceback.print_exc() on the except part of the related method, which is in this case def most_similar(self, *args, **kwargs): and I got:
Traceback (most recent call last):
File "./w2v_server.py", line 114, in most_similar
topn=5)
File "/usr/local/lib/python2.7/dist-packages/gensim/models/keyedvectors.py", line 304, in most_similar
self.init_sims()
File "/usr/local/lib/python2.7/dist-packages/gensim/models/keyedvectors.py", line 817, in init_sims
self.syn0norm = (self.syn0 / sqrt((self.syn0 ** 2).sum(-1))[..., newaxis]).astype(REAL)
AttributeError: 'KeyedVectors' object has no attribute 'syn0'
Any idea on why this might happens?
Note: I use python 2.7 and I installed gensim using pip, which gave me gensim 2.1.0.

FYI that demo code was baed on gensim 0.12.3 (from 2015, as listed in its requirements.txt), and would need updating to work with the latest gensim.
It might be sufficient to add a line to w2v_server.py at line 70 (just after the load_word2vec_format()), to force the creation of the needed syn0norm property (which in older gensims was auto-created on load), before deleting the raw syn0 values. Specifically:
self.model.init_sims(replace=True)
(You would leave out the replace=True if you were going to be doing operations other than most_similar(), that might require raw vectors.)
If this works to fix the problem for you, a pull-request to the w2v_server_googlenews repo would be favorably received!

Anaconda Acclerate / NumbaPro CUDA Linking Error OSX

Overall goal is to use NumbaPro to run some functions on the GPU (on OSX 10.8.3).
Before starting, I just wanted to get everything set up. According to this page I installed CUDA, registered as a CUDA developer, downloaded the Compiler SDK and set up the NUMBAPRO_NVVM=/path/to/libnvvm.dylib environment variable.
However, running this basic test function:
from numbapro import autojit
#autojit(target='gpu')
def my_function(x):
if x == 0.0:
return 1.0
else:
return x*x*x
print my_function(4.4)
exit()
Brings up this error:
File ".../anaconda/lib/python2.7/site-packages/numba/decorators.py", line 207, in compile_function
compiled_function = dec(f)
File "...lib/python2.7/site-packages/numbapro/cudapipeline/decorators.py", line 35, in _jit_decorator
File "...lib/python2.7/site-packages/numbapro/cudapipeline/decorators.py", line 128, in __init__
File "...lib/python2.7/site-packages/numbapro/cudapipeline/environment.py", line 31, in generate_ptx
File "...lib/python2.7/site-packages/numbapro/cudapipeline/environment.py", line 186, in _link_llvm_math_intrinsics
KeyError: 1
I've tried #vectorize'ing instead of autojit, same error.
#autojit by itself with no target works fine.
Any ideas?

For posterity's sake, I asked Continuum Support. They responded:
It seems that you are running a CUDA GPU with compute capability 1.x. NVVM only supports CC2.0 and above. We definitely should have a better error reporting and make it clear in the NumbaPro documentation for the supported compute capability.