I am working on a project that requires me to use a pre-trained Tensorflow model (MLP model) and make predictions. But while creating the executable file using PyInstaller, the size of the executable file becomes more that 2GB. I inspected and found that Tensorflow alone takes up approximately 1.3GB of that.
So my main question is whether I can reduce the size of this executable, by using an API for serving. However every solution that I see requires tensorflow as a dependency, which would require tensorflow hooks and make my executable larger.
The closest I found was this question in stackoverflow, Serve Tensorflow model without installing Tensorflow. However this requires that I already know what my model is, and since it uses numpy I can't use CUDA acceleration in the future. Also if there are other non-GPU optimizations in tensorflow (besides basic multithreading in numpy), I would be missing out on them if I use just numpy.
So is there a tensorflow API that can just serve the model and keep my executable small in size. Some additional information, this is how my hook-tensorflow.py looks like,
from PyInstaller.utils.hooks import collect_all
def hook(hook_api):
packages = [
'tensorflow',
'tensorflow_core',
'keras',
'astor'
]
for package in packages:
datas, binaries, hiddenimports = collect_all(package)
hook_api.add_datas(datas)
hook_api.add_binaries(binaries)
hook_api.add_imports(*hiddenimports)
Is Tensorflow Lite what you are looking for?
'TensorFlow Lite is a mobile library for deploying models on mobile, microcontrollers and other edge devices'
https://www.tensorflow.org/lite
Related
I have a Deep Learning Code for Object Detection. What I did is that I ran the code on Google Colab and then Exported the model to use it locally. Now to run the model I have to again install whole Tensorflow package which is quite heavy for my system.
I want to ask if there is a way to download and run only specific parts of Tensorflow Library?
I am using Tensorflow at only 2 places in my code and I have to install whole Tensorflow library for it.
This is where I am loading the model.
detect_fn = tf.saved_model.load(PATH_TO_SAVED_MODEL)
This is where I am using Tensorflow 2nd time.
input_tensor = tf.convert_to_tensor(image_rgb)
These are the only 2 functions required to me from the Tensorflow Library and not the whole library... Thanks in anticipation.
Though I'm not entirely sure on the library as a whole, there is a Lite version of Tensorflow (I guess they realised 430MB is a bit much too).
Information regarding this can be found here:
https://www.tensorflow.org/lite/
A guide here seems to detail how to pick and choose parts of the Lite library and although not used myself, I should expect some degree of compatibility between the two...
https://www.tensorflow.org/lite/guide/reduce_binary_size
I have this wierd requirement where I have 2 modules
chatbot(using rasa)
image classifier
I want to create a single Flask Web service for both of them
But issue is RASA uses tensorflow 1.x while my image classifier module is built using tensorflow 2.x.
I know I can not have both versions of Tensorflow in my environment(if there's a way, i am not aware of it)
As RASA doesn't support Tensorflow 2.x yet, one thing I'm left with is to downgrade Tensorflow for image classifier.
Is there a way to tackle this issue without changing my existing code or with minimum changes?
System information
What is the top-level directory of the model you are using: http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz
Have I written custom code (as opposed to using a stock example script provided in TensorFlow): yes
OS Platform and Distribution (e.g., Linux Ubuntu 16.04): Windows 10
TensorFlow installed from (source or binary): binary
TensorFlow version (use command below): 2.1.0
Bazel version (if compiling from source):
CUDA/cuDNN version: 10.0 / 7.6.5
GPU model and memory: GTX 1060
Exact command to reproduce:
I am trying to do transfer learning using faster_rcnn.
Currently this model is not available through tensorflow-hub and thus I have to load a legacy module
from the address given above.
I am able to load the model, retrieving a tensorflow-hub.KerasLayer object and to pass a data trough it.
But now I would like to tune this network to my own dataset that only contains 2 classes, so I wonder how can I modify the KerasLayer object so the classification layer does not output 90 classes, but just 2 ?
If I am not using the right approach, what do you advise to solve my problem ?
I would like to avoid using tensorflow object detection API.
Also as it is mentionned in the documentation, legacy models are not trainable, but I do not think it would be a problem if I add my own layer, am I right ?
Here is the code I am using
import tensorflow_hub as hub
import numpy as np
import cv2
# Run this once, then provide the local path where the model was downloaded,
# instead of the below http address
model = hub.KerasLayer(
"http://download.tensorflow.org/models/object_detection/faster_rcnn_inception_v2_coco_2018_01_28.tar.gz",
signature="serving_default",
signature_outputs_as_dict=True,
)
image = np.expand_dims(cv2.imread("/your/image.jpg"), 0).astype(np.uint8)
# I would like this to output only 2 different classes in 'outputs["detection_classes"]'
outputs = model(image)
I have a problem with Keras and multiprocessing. I have already searched a lot and I found a lot of questions with the same subjects:
Importing Keras breaks multiprocessing
Keras + Tensorflow and Multiprocessing in Python
(and lot more)
I tried these solutions, so basically importing Keras after the multiprocessing has been instantiated. In actual fact, I see this message:
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Using TensorFlow backend.
Before this message was only printed one time, so I assume that the backend is different, however, my processes are running on the same core. If run again the main process, it creates more processes that run in the same processor as well. It seems that something blocks the execution on different processors.
Any idea on how to fix it?
PS: I am using the second solution I have linked, in particular the following :
DO NOT LOAD KERAS TO YOUR MAIN ENVIRONMENT
The problem was in the installation of tensorflow and keras. The methods for achieving parallelization are correct.
The tensorflow documentation clearly states that is highly suggested to install the package using pip as the conda package is maintained only by the community (https://www.tensorflow.org/install/pip).
I fixed the problem uninstalling keras and tensorflow and reinstalling them with:
pip install tensorflow
pip install keras
In development, I have been using the gpu-accelerated tensorflow
https://storage.googleapis.com/tensorflow/linux/gpu/tensorflow_gpu-1.2.1-cp35-cp35m-linux_x86_64.whl
I am attempting to deploy my trained model along with an application binary for my users. I compile using PyInstaller (3.3.dev0+f0df2d2bb) on python 3.5.2 to create my application into a binary for my users.
For deployment, I install the cpu version, https://storage.googleapis.com/tensorflow/linux/cpu/tensorflow-1.2.1-cp35-cp35m-linux_x86_64.whl
However, upon successful compilation, I run my program and receive the infamous tensorflow cuda error:
tensorflow.python.framework.errors_impl.NotFoundError:
tensorflow/contrib/util/tensorflow/contrib/cudnn_rnn/python/ops/_cudnn_rnn_ops.so:
cannot open shared object file: No such file or directory
why is it looking for cuda when I've only got the cpu version installed? (Let alone the fact that I'm still on my development machine with cuda, so it should find it anyway. I can use tensorflow-gpu/cuda fine in uncompiled scripts. But this is irrelevant because deployment machines won't have cuda)
My first thought was that somehow I'm importing the wrong tensorflow, but I've not only used pip uninstall tensorflow-gpu but then I also went to delete the tensorflow-gpu in /usr/local/lib/python3.5/dist-packages/
Any ideas what could be happening? Maybe I need to start using a virtual-env..