I have a fully trained and saved Tensorflow model that I would like to load and use as a plug-in to a third party application (UCSF Chimera). This third party application runs on Python 2.7 which does not support Tensorflow. If even possible, is there a way for me to use this model at all in python 2.7?
I was originally looking at this previous post but it was for Java/C++.
First, save your Tensorflow model using pickle
with open("xxx.pkl", "wb") as outfile:
pickle.dump(checkpointfile, outfile)
Second, install anaconda and create a python2.7 environment
Third, install tensorflow again in the python2.7 environment
conda install tensorflow
Fourth, read the model using pickle
pkl_file = open("xxx.pkl", "rb")
data = pickle.load(pkl_file, encoding="latin1")
Related
I'm trying to work on an ASR model using transfer learning on wav2vec 2 model.
Anyway when I ever I wan't to show or modifiy an audio file I get this problem
def prepare_dataset(batch):
audio = batch["audio"]
# batched output is "un-batched"
batch["input_values"] = processor(audio["array"], sampling_rate=audio["sampling_rate"]).input_values[0]
batch["input_length"] = len(batch["input_values"])
with processor.as_target_processor():
batch["labels"] = processor(batch["sentence"]).input_ids
return batch
common_voice_train = common_voice_train.map(prepare_dataset, remove_columns=common_voice_train.column_names)
common_voice_test = common_voice_test.map(prepare_dataset, remove_columns=common_voice_test.column_names)
The erorrs:
RuntimeError: Backend "sox_io" is not one of available backends: ['soundfile'].
ImportError: To support decoding 'mp3' audio files, please install 'sox'.
This is my pytorch and torchaudio versions:
import torch
import torchaudio
print(torch.__version__)
print(torchaudio.__version__)
1.13.1+cu117
0.13.1+cu117
I really need help fixing this problem, this is part of my junior project! )':
I've trying to installing pytorch and installing deffrent versions but nothing worked the code is working. fine in colab but it's impossible for me to train it there so I have to use visual code...
First, note that the second error message is not from torchaudio and it's not accurate. TorchAudio does not depend on an external sox package.
TorchAudio provides limited IO features on Windows, as libsox does not
compile on Windows with VS2019. This situation is being worked on, but as of v0.13, Windows users need a workaround.
A simple way is to use other libraries like soundfile and convert the decoded NumPy NdArray object into PyTorch Tensor.
Another way is to install FFmpeg, and use torchaudio.io.StreamReader. You can write your own load function, following the tutorial like this.
https://pytorch.org/audio/0.13.1/tutorials/streamreader_basic_tutorial.html#sphx-glr-tutorials-streamreader-basic-tutorial-py
I have a question about scikit models and (retro-)compatibility.
I have a model (saved using joblib) created in Python 3.5 from scikit-learn 0.21.2, which I then analyze with the package shap version 0.30. Since I upgraded to Ubuntu 20.04 I have Python 3.8 (and newer versions of both scikit-learn and shap).
Because of the new packages version I cannot load them with Python 3.8, so I make a virtual environment with Py3.5 and the original package versions.
Now my question is: is there a way to re-dump with joblib the models so I can also open them with Python 3.8? I'd like to re-analyze the model with the newest version of the package shap (but of course it has a scikit version requirement that would break the joblib loading).
Alternatively, what other options do I have? (The only thing I do not want is to re-train the model).
There are no standard solutions within scikit-learn. If your model is supported, you can try sklearn-json.
Although this does not solve your current issue, you can in the future save your models in formats with fewer compatibility issues – see the Interoperable formats section in scikit-learn's Model persistence page.
I am trying to convert .onnx model to .pb format by using onnx-tf package, however, after invoking onnx_tf.backend.prepare function, the python kernel crashes. I am using the code below:
import onnx
from onnx_tf.backend import prepare
import tensorflow
onnx_model = onnx.load(<path-to-model>) # load onnx model
tf_rep = prepare(onnx_model) # <------ That's where python crashes
tf_rep.export_graph(<output-path>) # export the model
I have double-checked package version dependencies, as wrong dependencies caused different errors while loading the .onnx model, and those are as follows:
onnx==1.7.0
onnx-tf==1.6.0
tensorflow==2.2.0
tensorflow-addons==0.10.0
torch==1.6.0+cu101
However, the dependencies seem to be correct, according to Github pages.
There were large changes between ONNX 1.6 and 1.7 that caused many dependency issues. Try with Onnx 1.6, which may be the source of your issue. Where did you find 1.7 mentioned as the required version?
Additionally, you mention the Python kernel crashing? Are you running in a Jupyter notebook?
I've found this package to be a lot more unstable in Juypter notebook, and not print out all of the relevant errors. If you could test your system in a standalone Python script you may get more information.
Importing onnx and onnx-tf related libraries before tf and pytorch should help. The problem is probably caused by a version clash since both onnx and onnx-tf use tf.
Can you install onnx-tf of this commit 7d8fa7d88fab469253d75e5e11cf9cdcb90104c4
Between order of imports and this commit the issue was solved for me
I have been getting multiple errors which are due to conflicts in the TensorFlow version installed in my system and the version used to write the code in Tensorflow API.
I am using python 3.6.7 and Tensorflow 2.0 to get started with the code https://github.com/tensorflow/models/blob/master/research/deeplab/g3doc/installation.md
But I am getting several errors :
flags = tf.app.flags
AttributeError: module 'tensorflow' has no attribute 'app.
As I am using 2.0 , I replaced tf.app.flags with tf.compat.v1.flags.
from tensorflow.contrib import slim as contrib_slim
ModuleNotFoundError: No module named 'tensorflow.contrib'
I am not able to solve the second one.
Can I get help to know which python and tensorflow version should be used to run DeepLab v3+?
You should use the Tensorflow version 1.x to run the DeepLabV3+ model because it uses a session to run and also the slim library which is based on the TensorFlow 1.x. And so your two problems can be solved as:
Do not need to replace tf.app.flags with tf.compat.v1.flags.
To run DeepLabV3+ model, you need to put deeplab and slim folder in a folder (deeplab_slim),
and export them by running following export commands from this parent folder (deeplab_slim):
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/slim
export PYTHONPATH=$PYTHONPATH:`pwd`:`pwd`/deeplab
Hi i try to launch the object_detection_tutorial on my PC. When i run the following code to load a (frozen) Tensorflow model into memory.
detection_graph = tf.Graph()
with detection_graph.as_default():
od_graph_def = tf.GraphDef()
with tf.gfile.GFile(PATH_TO_FROZEN_GRAPH, 'rb') as fid:
serialized_graph = fid.read()
od_graph_def.ParseFromString(serialized_graph)
tf.import_graph_def(od_graph_def, name='')
ValueError: No op named NonMaxSuppressionV2 in defined operations.
I googled the error and upgrading the tensorflow version to 1.4 should fix the bug. In my code i used tensorflow 1.13 and it worked in google cloud. But even after uninstalling and installing eg. tensorflow 1.4, python uses 1.2.1
Picture of my code: https://ibb.co/VYkq2rF
Looks like the module is not installed correctly. Try creating a new environment using conda and set up Object Detection in it. It should resolve your issue.
Also, as best practices, it is always better to work on a conda environment than working on the base environment.
Please use below command to create an empty new environment and then install packages on it.
conda create --no-default-packages --name <env_name> python=<version>