I am trying to convert a tensor to numpy in the tesnorflow2.0 version. Since tf2.0 have eager execution enabled then it should work by default and working too in normal runtime. While I execute code in tf.data.Dataset API then it gives an error
"AttributeError: 'Tensor' object has no attribute 'numpy'"
I have tried ".numpy()" after tensorflow variable and for ".eval()" I am unable to get default session.
from __future__ import absolute_import, division, print_function, unicode_literals
import tensorflow as tf
# tf.executing_eagerly()
import os
import time
import matplotlib.pyplot as plt
from IPython.display import clear_output
from model.utils import get_noise
import cv2
def random_noise(input_image):
img_out = get_noise(input_image)
return img_out
def load_denoising(image_file):
image = tf.io.read_file(image_file)
image = tf.image.decode_png(image)
real_image = image
input_image = random_noise(image.numpy())
input_image = tf.cast(input_image, tf.float32)
real_image = tf.cast(real_image, tf.float32)
return input_image, real_image
def load_image_train(image_file):
input_image, real_image = load_denoising(image_file)
return input_image, real_image
This works fine
inp, re = load_denoising('/data/images/train/18.png')
# Check for correct run
plt.figure()
plt.imshow(inp)
print(re.shape," ", inp.shape)
And this produces mentioned error
train_dataset = tf.data.Dataset.list_files('/data/images/train/*.png')
train_dataset = train_dataset.map(load_image_train,num_parallel_calls=tf.data.experimental.AUTOTUNE)
Note: random_noise have cv2 and sklearn functions
You can't use the .numpy method on a tensor, if this tensor is going to be used in a tf.data.Dataset.map call.
The tf.data.Dataset object under the hood works by creating a static graph: this means that you can't use .numpy() because the tf.Tensor object when in a static-graph context do not have this attribute.
Therefore, the line input_image = random_noise(image.numpy()) should be input_image = random_noise(image).
But the code is likely to fail again since random_noise calls get_noise from the model.utils package.
If the get_noise function is written using Tensorflow, then everything will work. Otherwise, it won't work.
The solution? Write the code using only the Tensorflow primitives.
For instance, if your function get_noise just creates random noise with the shee of your input image, you can define it like:
def get_noise(image):
return tf.random.normal(shape=tf.shape(image))
using only the Tensorflow primitives, and it will work.
Hope this overview helps!
P.S: you could be interested in having a look at the articles "Analyzing tf.function to discover AutoGraph strengths and subtleties" - they cover this aspect (perhaps part 3 is the one related to your scenario): part 1 part 2 part 3
In TF2.x version, use tf.config.run_functions_eagerly(True).
Related
I am currently experimenting with performing some pre-processing in CuPy within a larger TensorFlow pipeline. As I understand, one can convert both a CuPy array and a Tensorflow into a DLpack capsule, allowing for seamless conversion between tensors and cupy arrays. However, in the following:
import os
import numpy as np
import tensorflow as tf
import cupy as cp
def cp2dl(cp_array):
return cp_array.toDlpack()
def tf2dl(tf_tensor):
return tf.experimental.dlpack.to_dlpack(tf_tensor)
def cp2tf(cp_array):
return tf.experimental.dlpack.from_dlpack(cp2dl(cp_array))
def tf2cp(tf_tensor):
return cp.fromDlpack(tf2dl(tf_tensor))
def preprocess(tf_tensor):
tensor_2_cupy = tf2cp(tf_tensor)
tensor_2_cupy = cp.divide(1000)
return cp2tf(tensor_2_cupy)
X = np.random.random_sample((100, 128, 128, 128))
Y = np.random.random_sample((100, 128, 128, 128))
dataset = tf.data.Dataset.from_tensor_slices((X, Y))
dataset.map(preprocess)
I get the error:
InvalidArgumentError: The argument to `to_dlpack` must be a TF tensor, not Python object
The error seems to be that "CuPy, like NumPy, is dependent on a Python runtime to run. When executing a tf.function, there is no such Python runtime - all ops are compiled into a graph that is executed inside the TF executor, which isolated from Python." (https://github.com/tensorflow/tensorflow/issues/51642).
My question is, are there any other ways around this to perform CuPy pre-processing in a larger Tensorflow setting?
I was trying to follow along Jeremy Howard Fast Ai course.
I am on Part 2 of of the course Lecture 9A: https://www.youtube.com/watch?v=0_BBRNYInx8&t=0s
The code I am trying to run can be found below (its the code in the second cell):
https://github.com/fastai/diffusion-nbs/blob/master/Stable%20Diffusion%20Deep%20Dive.ipynb
The imports:
from base64 import b64encode
import numpy
import torch
from diffusers import AutoencoderKL, LMSDiscreteScheduler, UNet2DConditionModel
from huggingface_hub import notebook_login
# For video display:
from IPython.display import HTML
from matplotlib import pyplot as plt
from PIL import Image
from torch import autocast
from torchvision import transforms as tfms
from tqdm.auto import tqdm
from transformers import CLIPTextModel, CLIPTokenizer, logging
torch.manual_seed(1)
# if not (Path.home()/'.huggingface'/'token').exists(): notebook_login()
# Supress some unnecessary warnings when loading the CLIPTextModel
logging.set_verbosity_error()
# Set device
print(torch.cuda.is_available())
torch_device = "cuda" if torch.cuda.is_available() else "cpu"
The actual code:
# Load the autoencoder model which will be used to decode the latents into image space.
vae = AutoencoderKL.from_pretrained("CompVis/stable-diffusion-v1-4", subfolder="vae")
# Load the tokenizer and text encoder to tokenize and encode the text.
tokenizer = CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
text_encoder = CLIPTextModel.from_pretrained("openai/clip-vit-large-patch14")
# The UNet model for generating the latents.
unet = UNet2DConditionModel.from_pretrained("CompVis/stable-diffusion-v1-4", subfolder="unet")
# The noise scheduler
scheduler = LMSDiscreteScheduler(beta_start=0.00085, beta_end=0.012, beta_schedule="scaled_linear", num_train_timesteps=1000)
# To the GPU we go!
vae = vae.to(torch_device)
text_encoder = text_encoder.to(torch_device)
unet = unet.to(torch_device);
The Problem: Cuda out of memory error
in the last line I unet.to(torch_device)
I encounter the Cuda out of memory error
I have a 2 GB Nvidia Graphics card (I know this memory is too small, so I tried a few things to fix this)
The solution I tried but it did not work
In the below code
I used set_attention_slice
I set vae and text_encoder to "cpu"
## To Prevent CUDA out of memory. Change 1024 to any numbeer
slice_size = unet.config.attention_head_dim // 1024
unet.set_attention_slice(slice_size)
# To the GPU we go!
vae = vae.to("cpu") #vae.to(torch_device)
text_encoder = text_encoder.to("cpu")#text_encoder.to(torch_device)
unet = unet.to(torch_device).to_fp16 # to_fp16 is my code comment
This did not work,
Is there anything else I can try?
I run the example code found at: https://huggingface.co/microsoft/beit-large-patch16-512 of the BEiT (large-sized model, fine-tuned on ImageNet-1k), with no changes:
from transformers import BeitFeatureExtractor, BeitForImageClassification
from PIL import Image
import requests
url = 'http://images.cocodataset.org/val2017/000000039769.jpg'
image = Image.open(requests.get(url, stream=True).raw)
feature_extractor = BeitFeatureExtractor.from_pretrained('microsoft/beit-large-patch16-512')
model = BeitForImageClassification.from_pretrained('microsoft/beit-large-patch16-512')
inputs = feature_extractor(images=image, return_tensors="pt")
outputs = model(**inputs)
logits = outputs.logits
# model predicts one of the 1000 ImageNet classes
predicted_class_idx = logits.argmax(-1).item()
print("Predicted class:", model.config.id2label[predicted_class_idx])
The code runs correctly but it raises the following warnings:
/home/name/anaconda3/lib/python3.8/site-packages/torch/functional.py:568: UserWarning: torch.meshgrid: in an upcoming release, it will be required to pass the indexing argument. (Triggered internally at /opt/conda/conda-bld/pytorch_1646755853042/work/aten/src/ATen/native/TensorShape.cpp:2228.)
return _VF.meshgrid(tensors, **kwargs) # type: ignore[attr-defined]
/home/name/anaconda3/lib/python3.8/site-packages/transformers/feature_extraction_utils.py:158: UserWarning: Creating a tensor from a list of numpy.ndarrays is extremely slow.
Please consider converting the list to a single numpy.ndarray with numpy.array() before converting to a tensor.
(Triggered internally at /opt/conda/conda-bld/pytorch_1646755853042/work/torch/csrc/utils/tensor_new.cpp:210.)
Are these warnings safe to ignore? If not how to fix them? Should I just signal this as a bug on the Huggingface page?
I have trained a CNN in Matlab 2019b that classifies images between three classes. When this CNN was tested in Matlab it was functioning fine and only took 10-15 seconds to classify an image. I used the exportONNXNetwork function in Maltab so that I can implement my CNN in Tensorflow. This is the code I am using to use the ONNX file in python:
import onnx
from onnx_tf.backend import prepare
import numpy as np
from PIL import Image
onnx_model = onnx.load('trainednet.onnx')
tf_rep = prepare(onnx_model)
filepath = 'filepath.png'
img = Image.open(filepath).resize((224,224)).convert("RGB")
img = array(img).transpose((2,0,1))
img = np.expand_dims(img, 0)
img = img.astype(np.uint8)
probabilities = tf_rep.run(img)
print(probabilities)
When trying to use this code to classify the same test set, it seems to be classifying the images correctly but it is very slow and freezes my computer as it reaches high memory usages of up to 95+% at some points.
I also noticed in the command prompt while classifying it prints this:
2020-04-18 18:26:39.214286: W tensorflow/core/grappler/optimizers/meta_optimizer.cc:530] constant_folding failed: Deadline exceeded: constant_folding exceeded deadline., time = 486776.938ms.
Is there any way I can make this python code classify faster?
Maybe you could try to understand what part of the code takes a long time this way:
import onnx
from onnx_tf.backend import prepare
import numpy as np
from PIL import Image
import datetime
now = datetime.datetime.now()
onnx_model = onnx.load('trainednet.onnx')
tf_rep = prepare(onnx_model)
filepath = 'filepath.png'
later = datetime.datetime.now()
difference = later - now
print("Loading time : %f ms" % (difference.microseconds / 1000))
img = Image.open(filepath).resize((224,224)).convert("RGB")
img = array(img).transpose((2,0,1))
img = np.expand_dims(img, 0)
img = img.astype(np.uint8)
now = datetime.datetime.now()
probabilities = tf_rep.run(img)
later = datetime.datetime.now()
difference = later - now
print("Prediction time : %f ms" % (difference.microseconds / 1000))
print(probabilities)
Let me know what the output looks like :)
In this case, it appears that the Grapper optimization suite has encountered some kind of infinite loop or memory leak. I would recommend filing an issue against the Github repo.
It's challenging to debug why constant folding is taking so long, but you may have better performance using the ONNX TensorRT backend as compared to the TensorFlow backend. It achieves better performance as compared to the TensorFlow backend on Nvidia GPUs while compiling typical graphs more quickly. Constant folding usually doesn't provide large speedups for well optimized models.
import onnx
import onnx_tensorrt.backend as backend
import numpy as np
model = onnx.load("trainednet.onnx'")
engine = backend.prepare(model, device='CUDA:1')
filepath = 'filepath.png'
img = Image.open(filepath).resize((224,224)).convert("RGB")
img = array(img).transpose((2,0,1))
img = np.expand_dims(img, 0)
img = img.astype(np.uint8)
output_data = engine.run(img)[0]
print(output_data)
You should consider some points while working on TensorFlow with Python. A GPU will be better for work as it fastens the whole processing. For that, you have to install CUDA support. Apart from this, the compiler also sometimes matters. I can tell VSCode is better than Spyder from my experience.
I hope it helps.
Since the command prompt states that your program takes a long time to perform constant folding, it might be worthwhile to turn this off. Based on this documentation, you could try running:
import numpy as np
import timeit
import traceback
import contextlib
import onnx
from onnx_tf.backend import prepare
from PIL import Image
import tensorflow as tf
#contextlib.contextmanager
def options(options):
old_opts = tf.config.optimizer.get_experimental_options()
tf.config.optimizer.set_experimental_options(options)
try:
yield
finally:
tf.config.optimizer.set_experimental_options(old_opts)
with options({'constant_folding': False}):
onnx_model = onnx.load('trainednet.onnx')
tf_rep - prepare(onnx_model)
filepath = 'filepath.png'
img = Image.open(filepath).resize((224,224)).convert("RGB")
img = array(img).transpose((2,0,1))
img = np.expand_dims(img, 0)
img = img.astype(np.uint8)
probabilities = tf_rep.run(img)
print(probabilities)
This disables the constant folding performed in the TensorFlow Graph optimization. This can work both ways: on the one hand it will not reach the constant folding deadline, but on the other hand disabling constant folding can result in significant runtime increases. Anyway it is worth trying, good luck!
I face problems when trying to get a numpy array from a tensorflow tensor. I use a tensorflow hub module but I don't want to use tensorflow in downstream tasks but rather need a numpy array.
I know that I have to call the 'eval()' method on the tensor from within a tensorflow session. But unfortuantely I cannot get it to work... :( It tells me that the "tables are not initialized". I tried to add 'sess.run(tf.tables_initializer())' but then I get the error: 'NotFoundError: Resource localhost/module_1/embeddings_morph_specialized/class tensorflow::Var does not exist'. I am not sure what to try next. I have also tried 'sess.run()' but have also been unsuccessful.
import numpy as np
import tensorflow as tf
import tensorflow_hub as hub
embed = hub.Module("https://public.ukp.informatik.tu-darmstadt.de/arxiv2018-xling-sentence-embeddings/tf-hub/monolingual/1")
X = embed(["This is a test."])
# I tried:
#with tf.Session() as sess:
# sess.run(tf.tables_initializer())
# X.eval()
'X' is the tensor which I would like to convert to a numpy array.
Any help is appreciated. :)
Thank you.
Unfortunately, tf_hub modules are not yet supported in eager mode except in tf 2 (which is still in beta and I think needs slightly different hub modules anyway).
Therefore you'll need to run this in a session.
Something like:
embed = hub.Module("https://public.ukp.informatik.tu-darmstadt.de/arxiv2018-xling-sentence-embeddings/tf-hub/monolingual/1")
X = embed(["This is a test."])
with tf.Session() as session:
session.run([tf.global_variables_initializer(), tf.tables_initializer()])
numpy_arr = session.run(X)