Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have started learning Tensorflow recently and I am wondering if it is worth using in simple optimization problems (least squares, maximum likelihood estimation, ...) instead of more traditional libraries (scikit-learn, statsmodel)?
I have implemented a basic AR model estimator using Tensorflow with MLE and the AdamOptimizer and the results are not convincing either performance or computation speed wise.
What do you think?
This is somewhat opinion based, but Tensorflow and similar frameworks such as PyTorch are useful when you want to optimize an arbitrary, parameter-rich non-linear function (e.g., a deep neural network). For a 'standard' statistical model, I would use code that was already tailored to it instead of reinventing the wheel. This is true especially when there are closed-form solutions (as in linear least squares) - why go into to the murky water of local optimization when you don't have to? Another advantage of using existing statistical libraries is that they usually provide you with measures of uncertainty about your point estimates.
I see one potential case in which you might want to use Tensorflow for a simple linear model: when the number of variables is so big the model can't be estimated using closed-form approaches. Then gradient descent based optimization makes sense, and tensorflow is a viable tool for that.
Related
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
This question does not appear to be about programming within the scope defined in the help center.
Closed 1 year ago.
Improve this question
I'm using sklearn's decision tree for a binary class problem. However it turns out that after optimizing everything (optimizing hyper parameters and using the optimal number of selected features), the best I can do is get an accuracy and f1-score that's as good as baseline (no feature selection and use all features).
Sure now it's less messy (less features), and the code runs faster. But is this expected? Or is the point of feature selection to improve performance metrics of the classifier?
That's right. Feature selection will mostly give you performance benefits and might help a little against overfitting if relevant. It's not really supposed to improve the training metrics as you are essentially trying to solve the same problem with less information in your hands.
It doesn't mean you shouldn't do it though. If you can achieve the same performance with fewer features - use fewer features :)
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have an undirected graph with edges of equal distance with 7 features per node. I want to train a Neural Network with this graph as an input and output a scalar. What network architecture do I need for my network to analyse the graph locally (for example, a node and it's neighbours) and to generalise, much like Convolutional Neural Networks operate on grid data. I have heard of Graph Neural Networks however I don't know if this is what i'm looking for. Will it be able to analyse my graph much like a CNN does with an image, sharing the generalisation benefits that convolution kernels bring?
I want to implement the solution in TensorFlow, ideally with Keras.
Thank you
The performance will most likely depend on the exact output that you're hoping to get. From your description a 2D-CNN should be good enough and easier to implement with Keras than a GNN.
However, there are some advantages to retaining the graph structure from your data. I think this is too much to present here, but you can find a proper explanation on "Spatio-Temporal Analysis and Prediction of Cellular Traffic in Metropolis" by Wang et al.
This paper also has the benefit of describing data processing to input into the network.
If you don't want to use basic Keras models to assemble your own GNN you may also want to take a look at Spektral, which is a python library for graph deep learning.
Without any other constraints I would firstly use a CNN, because it will be faster to implement with almost ready to use models from Keras.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Are these libraries fairly interchangeable?
Looking here, https://stackshare.io/stackups/keras-vs-pytorch-vs-scikit-learn, it seems the major difference is the underlying framework (at least for PyTorch).
Yes, there is a major difference.
SciKit Learn is a general machine learning library, built on top of NumPy. It features a lot of machine learning algorithms such as support vector machines, random forests, as well as a lot of utilities for general pre- and postprocessing of data. It is not a neural network framework.
PyTorch is a deep learning framework, consisting of
A vectorized math library similar to NumPy, but with GPU support and a lot of neural network related operations (such as softmax or various kinds of activations)
Autograd - an algorithm which can automatically calculate gradients of your functions, defined in terms of the basic operations
Gradient-based optimization routines for large scale optimization, dedicated to neural network optimization
Neural-network related utility functions
Keras is a higher-level deep learning framework, which abstracts many details away, making code simpler and more concise than in PyTorch or TensorFlow, at the cost of limited hackability. It abstracts away the computation backend, which can be TensorFlow, Theano or CNTK. It does not support a PyTorch backend, but that's not something unfathomable - you can consider it a simplified and streamlined subset of the above.
In short, if you are going with "classic", non-neural algorithms, neither PyTorch nor Keras will be useful for you. If you're doing deep learning, scikit-learn may still be useful for its utility part; aside from it you will need the actual deep learning framework, where you can choose between Keras and PyTorch but you're unlikely to use both at the same time. This is very subjective, but in my view, if you're working on a novel algorithm, you're more likely to go with PyTorch (or TensorFlow or some other lower-level framework) for flexibility. If you're adapting a known and tested algorithm to a new problem setting, you may want to go with Keras for its greater simplicity and lower entry level.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 4 years ago.
Improve this question
I'm experimenting with recent ideas coming from adversarial training and I'm specifically interested in a loss function which includes the input. This means I would like to derive the loss function with respect to the input (not only the model parameters).
One solution I can see is the function tf.conv2d_backprop_input(...). This can work as a solution for conv layers, however I also require a solution for fully connected layers as well. Another way to approach this problem is using the Cleverhans library written by Ian Goodfellow and Nicolas Papernot. This can be a more "complete" solution however its usage is not exactly clear (I need a simple example and not a complete API).
I would love to hear your thoughts and methodology on creating a custom deep learning simulation with adverserial training.
The dependence of an output node on the input can be calculated by backpropagation and is called saliency. It can be used to understand which parts of an input are most strongly contributing to a neuron's output for any differentiable neural network. This repository contains a collection of methods for calculating saliency and links to papers.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I was wondering whether and how it is possible to use a python generator as data input to scikit-learn classifier's .fit() functions? Due to huge amounts of data, this seems to make sense to me.
In particular I am about to implement a random forest approach.
Regards
K
The answer is "no". To do out of core learning with random forests, you should
Split your data into reasonably-sized batches (restricted by the amount of RAM you have; bigger is better);
train separate random forests;
append all the underlying trees together in the estimators_ member of one of the trees (untested):
for i in xrange(1, len(forests)):
forests[0].estimators_.extend(forests[i].estimators_)`
(Yes, this is hacky, but no solution to this problem has been found yet. Note that with very large datasets, it might pay to just sample a number training examples that fits in the RAM of a big machine instead of training on all of it. Another option is to switch to linear models with SGD, those implement a partial_fit method, but obviously they're limited in the kind of functions they can learn.)
The short answer is "No, you can't". Classical Random Forest classifier is not an incremental or online classifier, so you can't discard training data while learning, and have to provide all the dataset at once.
Due to popularity of RF in machine learning (not least because of the good prediction results for some interesting cases), there are some attempts to implement online variation of Random Forest, but to my knowledge those are not yet implemented in any python ML package.
See Amir Saffari's page for such an approach (not Python).