I am following the tutorial on neural style transfer. The style transfer is done by minimizing a loss function with respect to an image (initialized with the content image). What confuses me is the following piece of code:
preprocessed_input = tf.keras.applications.vgg19.preprocess_input(inputs)
which is part of the call method in the StyleContentModel class. How does TensorFlow know the gradient of this operation? I have checked if this operation has a gradient function using get_gradient_function in the module tensorflow.python.framework.ops, and as far as I can tell it does not.
It is very simple, the function internally uses symbolic tensor operations that are differentiable. TensorFlow can compute gradients through functions that internally use TensorFlow operations, there is no need to manually define a gradient for each function.
You can confirm by looking at the code of that function here, specially if you look at the _preprocess_symbolic_function here which is using normal scalar operations and Keras backend functions (which are just TensorFlow functions in tf.keras).
This has nothing to do with the model or gradients. What this function does is scale the input images so the pixels are in the range from -1 to +1. This is a common requirement for many models used in transfer learning like VGG and MobileNet. If you use the ImageDataGenerator it has a parameter preprocessing_function which the generator calls to preprocess the images. Make sure if you preprocess the training images you do the same for the test and validation images.
Related
I want to make a custom layer which takes image as an input, transforms it to string of a certain kind, than performs some complicated calculations with this string and transforms it back to image. These functions represent images as numpy arrays.
As far as I know, Keras custom layers should utilize Tensorflow tensors and operations over them. But in this case it seems to be impossible to write my functions in such format.
So, can I somehow just put my function into the model code?
I am using Tensorflow v1.14 for creating networks and training them. Everything works fine and I don't have any problem with code. I use the function tf.reduce_min() in my loss function. For the gradients to flow, it is essential that the loss function is differentiable. But a min operator is not differentiable as such. This link, gives the necessary explanation for the tf.reduce_min() function but without references.
In general there are functions in Tensorflow (tf.cond, tf.where, among many more) that are inherently not differentiable by their definition. I want to know how these are made differentiable by defining "pseudo gradients" and the proper references to documentation. Thanks.
I am a freshman in Keras and deep learning, I am not quite sure the right way to add the regularization, I wrote a CNN autoencoder using the API model class, right now I add the regularizer in each of the "Conv2D" Keras function,I am not sure if this is the right place to add regularization, could anyone please give me some suggestions?
(I tried to run the training and check the reconstructed test images, it is OK, but not very good, I use MNIST to test, the line of the reconstructed MNIST number is thicker than the original one.)
In my problem, the input image is an impaired one, and the original good image is used as a training label, by comparing the output image of the CNN with the training label image, I use the "mean absolute error" to define the loss , and also use it as the metric.
I defined three functions first, one downsampling function (the one below), one upsampling function, and one function to squeeze the third dimension of the matrix to get a two-dimensional matrix as the output.
My code is too long, just to help illustrate the problem, part of my code is as follow:
After having three defined functions, I defined the model as follow (not in detail, just part of it to help explain my problem)
load all necessary parameters to the model,then define the optimizer parameters, and compile the model
I am trying to define a custom loss function in Keras where I have an additional term that is an integral over the domain of the neural network output. So this would look like:
The key point is that the integral runs over an entire domain that I've specified, not just training data. I don't mind using any form of quadrature to evaluate the integral, I just need to be able to evaluate it. Currently, as far as the documentation indicates, this is not possible to do with a custom loss as it only provides access to y_pred and y_true.
Is there any way of achieving this in Keras?
If the idea is just defining extra variables, you can do this either inside (locally) or outside (globally) the loss function, using keras backend functions:
import keras.backend as K
myDomain = K.variable(range(100)) / 10 #for instance
def custom_loss(y_true,y_pred):
localVar = K.variable([[1,2],[3,1]])
return calculationsWith(y_true,y_pred,localVar,myDomain)
It's important that you use functions coming from the backend to do the calculations. (Either from K or directly from tensoflow, theano or CNTK).
I am working with the DCGAN code. I need to modify the reward that is given to one of the neural nets by adding a function that would take the output of this neural net, analyse it, and issue a penalty on it. So my loss function would look like:
self.g_loss = self.g_loss + self.penalty
Problem is
this penalty function only takes the numpy arrays as an input (I have no way of modifying this),
neural network output is a tf.tensor,
and as the values haven't been assigned to the neural net yet (technically it hasn't been built yet) I can't run neither .eval() nor sess.run().
So how would I convert a tensorflow tensor into numpy array in this case?
Tensorflow has tf.py_func for wrapping Python functions and passing tensors to them. However, you can't then use this loss function to train the network, because Tensorflow doesn't automatically differentiate numpy code.
Luckily for you, autograd does automatically differentiate numpy code. If you use that, in another tf.pyfunc call, you can get gradients, which you can then put back into the tensorflow graph on the backward pass.
Here's an example of how you can do it all in this gist.