Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm experimenting with recent ideas coming from adversarial training and I'm specifically interested in a loss function which includes the input. This means I would like to derive the loss function with respect to the input (not only the model parameters).
One solution I can see is the function tf.conv2d_backprop_input(...). This can work as a solution for conv layers, however I also require a solution for fully connected layers as well. Another way to approach this problem is using the Cleverhans library written by Ian Goodfellow and Nicolas Papernot. This can be a more "complete" solution however its usage is not exactly clear (I need a simple example and not a complete API).
I would love to hear your thoughts and methodology on creating a custom deep learning simulation with adverserial training.
The dependence of an output node on the input can be calculated by backpropagation and is called saliency. It can be used to understand which parts of an input are most strongly contributing to a neuron's output for any differentiable neural network. This repository contains a collection of methods for calculating saliency and links to papers.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have an undirected graph with edges of equal distance with 7 features per node. I want to train a Neural Network with this graph as an input and output a scalar. What network architecture do I need for my network to analyse the graph locally (for example, a node and it's neighbours) and to generalise, much like Convolutional Neural Networks operate on grid data. I have heard of Graph Neural Networks however I don't know if this is what i'm looking for. Will it be able to analyse my graph much like a CNN does with an image, sharing the generalisation benefits that convolution kernels bring?
I want to implement the solution in TensorFlow, ideally with Keras.
Thank you
The performance will most likely depend on the exact output that you're hoping to get. From your description a 2D-CNN should be good enough and easier to implement with Keras than a GNN.
However, there are some advantages to retaining the graph structure from your data. I think this is too much to present here, but you can find a proper explanation on "Spatio-Temporal Analysis and Prediction of Cellular Traffic in Metropolis" by Wang et al.
This paper also has the benefit of describing data processing to input into the network.
If you don't want to use basic Keras models to assemble your own GNN you may also want to take a look at Spektral, which is a python library for graph deep learning.
Without any other constraints I would firstly use a CNN, because it will be faster to implement with almost ready to use models from Keras.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 3 years ago.
Improve this question
I am wondering if there is a way to link the deep neural network built in python and use it in MATLAB code? for example, suppose I built a deep neural network as a function in python, so I need to call it in MATLAB code as a function in order to used it with MATLAB. Is that possible? if so, anyone can provide me a guidance or the steps to do that.
thank you
You can import pretrained networks in MATLAB if you have access to the Deep Learning Toolbox.
You can use the importKerasNetwork function for Tensorflow-Keras networks, importCaffeNetwork for Caffe networks or importONNXNetwork for ONNX networks.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
While reading the great book by F. Chollet, I'm experimenting with Keras / Tensorflow, on a simple Sequential model that I train on simulated images, which come from a physical analytical model.
Having full control of the simulations, I wrote a generator which produces an infinite stream of data and label batches, which I use with fit_generator in Keras. The data so generated are never identical, plus I can add some random noise to each image.
Now I'm wondering: is it a problem if the model never sees the same input data from one epoch to the next?
Can I assume my problems in getting the loss down are not due to the fact that the data are "infinite" (so I only have to concentrate on hyper parameters tuning)?
Please feel free if you have any advice for dealing with DL on simulated data.
A well trained network will pick up on patterns in the data, prioritizing new data over old. If your data comes from a constant distribution this doesn't matter, but if that distribution is changing over time it should adapt (slowly) to the more recent distribution.
The fact that the data is never identical does not matter. Most trained networks use some form of data augmentation (e.g. for image processsing, it is common for images to be randomly cropped, rotated, resized, and have color manipulations applied etc, so each example is never identical even if it comes from the same base image).
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have started learning Tensorflow recently and I am wondering if it is worth using in simple optimization problems (least squares, maximum likelihood estimation, ...) instead of more traditional libraries (scikit-learn, statsmodel)?
I have implemented a basic AR model estimator using Tensorflow with MLE and the AdamOptimizer and the results are not convincing either performance or computation speed wise.
What do you think?
This is somewhat opinion based, but Tensorflow and similar frameworks such as PyTorch are useful when you want to optimize an arbitrary, parameter-rich non-linear function (e.g., a deep neural network). For a 'standard' statistical model, I would use code that was already tailored to it instead of reinventing the wheel. This is true especially when there are closed-form solutions (as in linear least squares) - why go into to the murky water of local optimization when you don't have to? Another advantage of using existing statistical libraries is that they usually provide you with measures of uncertainty about your point estimates.
I see one potential case in which you might want to use Tensorflow for a simple linear model: when the number of variables is so big the model can't be estimated using closed-form approaches. Then gradient descent based optimization makes sense, and tensorflow is a viable tool for that.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 7 years ago.
Improve this question
I've been trying to train a neural network to recognise the three types of tags I have images of (circle, rectangle and blank). I used the example set up for recognising the digits dataset provided here and found that I got a 75% correct prediction rate with barely any tweaking (provided my images had a certain level of preprocessing with filters etc).
What I'm interested in understanding more about is the classifier section (code below). I'm not sure what the different convolution and layer options do and what options I have for tweaking them. Does anyone have any advice for other convolution or layers that I could use to try improve my prediction accuracy and what they mean? Apologies for being vague, this is the first time I've touched a NN and am struggling to get my head around it.
nn = Classifier(
layers=[
Convolution('Rectifier', channels=12, kernel_shape=(3, 3), border_mode='full'),
Convolution('Rectifier', channels=8, kernel_shape=(3, 3), border_mode='valid'),
Layer('Rectifier', units=64),
Layer('Softmax')],
learning_rate=0.002,
valid_size=0.2,
n_stable=10,
verbose=True)
I would recommend the excellent video course by Hugo Larochelle on Youtube. The 9th chapter is about convolutional networks and explains all the parameters. You might start from the first two chapters, they explain how the neural networks work in general, and you will get used to the terms like softmax and rectifier.
Another good resource: Andrej Karpathy's lecture notes