Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
Are these libraries fairly interchangeable?
Looking here, https://stackshare.io/stackups/keras-vs-pytorch-vs-scikit-learn, it seems the major difference is the underlying framework (at least for PyTorch).
Yes, there is a major difference.
SciKit Learn is a general machine learning library, built on top of NumPy. It features a lot of machine learning algorithms such as support vector machines, random forests, as well as a lot of utilities for general pre- and postprocessing of data. It is not a neural network framework.
PyTorch is a deep learning framework, consisting of
A vectorized math library similar to NumPy, but with GPU support and a lot of neural network related operations (such as softmax or various kinds of activations)
Autograd - an algorithm which can automatically calculate gradients of your functions, defined in terms of the basic operations
Gradient-based optimization routines for large scale optimization, dedicated to neural network optimization
Neural-network related utility functions
Keras is a higher-level deep learning framework, which abstracts many details away, making code simpler and more concise than in PyTorch or TensorFlow, at the cost of limited hackability. It abstracts away the computation backend, which can be TensorFlow, Theano or CNTK. It does not support a PyTorch backend, but that's not something unfathomable - you can consider it a simplified and streamlined subset of the above.
In short, if you are going with "classic", non-neural algorithms, neither PyTorch nor Keras will be useful for you. If you're doing deep learning, scikit-learn may still be useful for its utility part; aside from it you will need the actual deep learning framework, where you can choose between Keras and PyTorch but you're unlikely to use both at the same time. This is very subjective, but in my view, if you're working on a novel algorithm, you're more likely to go with PyTorch (or TensorFlow or some other lower-level framework) for flexibility. If you're adapting a known and tested algorithm to a new problem setting, you may want to go with Keras for its greater simplicity and lower entry level.
Related
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 2 years ago.
Improve this question
I have an undirected graph with edges of equal distance with 7 features per node. I want to train a Neural Network with this graph as an input and output a scalar. What network architecture do I need for my network to analyse the graph locally (for example, a node and it's neighbours) and to generalise, much like Convolutional Neural Networks operate on grid data. I have heard of Graph Neural Networks however I don't know if this is what i'm looking for. Will it be able to analyse my graph much like a CNN does with an image, sharing the generalisation benefits that convolution kernels bring?
I want to implement the solution in TensorFlow, ideally with Keras.
Thank you
The performance will most likely depend on the exact output that you're hoping to get. From your description a 2D-CNN should be good enough and easier to implement with Keras than a GNN.
However, there are some advantages to retaining the graph structure from your data. I think this is too much to present here, but you can find a proper explanation on "Spatio-Temporal Analysis and Prediction of Cellular Traffic in Metropolis" by Wang et al.
This paper also has the benefit of describing data processing to input into the network.
If you don't want to use basic Keras models to assemble your own GNN you may also want to take a look at Spektral, which is a python library for graph deep learning.
Without any other constraints I would firstly use a CNN, because it will be faster to implement with almost ready to use models from Keras.
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I have started learning Tensorflow recently and I am wondering if it is worth using in simple optimization problems (least squares, maximum likelihood estimation, ...) instead of more traditional libraries (scikit-learn, statsmodel)?
I have implemented a basic AR model estimator using Tensorflow with MLE and the AdamOptimizer and the results are not convincing either performance or computation speed wise.
What do you think?
This is somewhat opinion based, but Tensorflow and similar frameworks such as PyTorch are useful when you want to optimize an arbitrary, parameter-rich non-linear function (e.g., a deep neural network). For a 'standard' statistical model, I would use code that was already tailored to it instead of reinventing the wheel. This is true especially when there are closed-form solutions (as in linear least squares) - why go into to the murky water of local optimization when you don't have to? Another advantage of using existing statistical libraries is that they usually provide you with measures of uncertainty about your point estimates.
I see one potential case in which you might want to use Tensorflow for a simple linear model: when the number of variables is so big the model can't be estimated using closed-form approaches. Then gradient descent based optimization makes sense, and tensorflow is a viable tool for that.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
We don’t allow questions seeking recommendations for books, tools, software libraries, and more. You can edit the question so it can be answered with facts and citations.
Closed 5 years ago.
Improve this question
I have recently been looking into using TensorFlow for creating a custom CNN, and have been attempting to use the tutorials for insight on the most straightforward way to design, train, and deploy an image classification network.
The two approaches that have stood out to me are:
TF Layers API: This API seems to provide the most straightforward and intuitive way of defining a network, layer by layer. That said, the way that they train and evaluate the model uses the tf.learn.Estimator class, which seems a bit limiting in that the network is strictly trained using the Estimator's fit() method and validated using the evaluate() method. This tutorial does not even use a tf.Session.
Low-level API: Defining a network seems a bit more tedious. Also, training and deploying is done in a very manual fashion, but it appears to offer more control.
For a TensorFlow novice looking to implement and train relatively basic CNN's who is looking for the ability to tinker with network architecture and basic hyperparameter tuning, what would be the best API to get familiar with?
Also, if there are any useful tutorials or examples using your preferred interface, links would be greatly appreciated.
Keras is a nice frontend for Tensorflow. It sounds like it should fit your needs. Here is an example of someone training a CNN with Keras.
I like what is described on this page: write your stuff "by hand", but give the model a transparent class interface to the outside. TF graph stuff is handled as properties and set up at construction, and then the model can be used without having to know TF. A bit like Keras, but giving you full control (and forcing you to learn the low level). It lacks Keras' composability, though.
Basically, it recommends you do something along the lines of this:
class Model:
#lazy_property
def prediction(self):
...
#lazy_property
def optimize(self):
# actual TF stuff here, e.g.:
cross_entropy = -tf.reduce_sum(self.target, tf.log(self.prediction))
optimizer = tf.train.RMSPropOptimizer(0.03)
return optimizer.minimize(cross_entropy)
#lazy_property
def error(self):
...
If it interests you, I tried to package up that approach with a common base class and decorators here. I tried to stick to the Keras API, and sessions are explicitly handled with withs. The code, however, has not actually been used in training -- I just wrote it after getting fed up with repeating everything in serveral university projects, and wanting to produce something cleaner.
Closed. This question does not meet Stack Overflow guidelines. It is not currently accepting answers.
Questions asking us to recommend or find a tool, library or favorite off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it.
Closed 8 years ago.
Improve this question
In my project, we are supposed to use an SVM based algorithm. So to get a basic idea about implementation of SVM, we are trying to implement an algorithm, which when fed with an array of 1000 integers where first 95 integers are of values ranging from 0-5, then the next 5 around 10,000 and then again 95 integers of values ranging from 0-5 and next 5 around 10,000 and so on, will be able to predict the next 100 integers (1001st - 1100th) with first 95 integers around 0-5 and the last 5 around 10,000 ...
How to implement it? Preferred programming language is python. So are there any svm modules like libsvm which will facilitate this?
I know this might be a stupid question, but any help would be appreciated a lot !!
Please reply
Here are some resources on AI (SVM specifically) from the Python wiki:
Milk - Milk is a machine learning toolkit in Python. Its focus is on supervised classification with several classifiers available: SVMs (based on libsvm), k-NN, random forests, decision trees. It also performs feature selection. These classifiers can be combined in many ways to form different classification systems.
LibSVM - LIBSVM is an integrated software for support vector classification, (C-SVC, nu-SVC), regression (epsilon-SVR, nu-SVR) and distribution estimation (one-class SVM). It supports multi-class classification. A Python interface is available by by default.
Shogun - The machine learning toolbox's focus is on large scale kernel methods and especially on Support Vector Machines (SVM) . It provides a generic SVM object interfacing to several different SVM implementations, among them the state of the art OCAS, Liblinear, LibSVM, SVMLight, SVMLin and GPDT. Each of the SVMs can be combined with a variety of kernels. The toolbox not only provides efficient implementations of the most common kernels, like the Linear, Polynomial, Gaussian and Sigmoid Kernel but also comes with a number of recent string kernels. SHOGUN is implemented in C++ and interfaces to Matlab(tm), R, Octave and Python and is proudly released as Machine Learning Open Source Software
I'd go with SVM from SciKit. Other options include svmlight and LaSVM.
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
I was wondering whether and how it is possible to use a python generator as data input to scikit-learn classifier's .fit() functions? Due to huge amounts of data, this seems to make sense to me.
In particular I am about to implement a random forest approach.
Regards
K
The answer is "no". To do out of core learning with random forests, you should
Split your data into reasonably-sized batches (restricted by the amount of RAM you have; bigger is better);
train separate random forests;
append all the underlying trees together in the estimators_ member of one of the trees (untested):
for i in xrange(1, len(forests)):
forests[0].estimators_.extend(forests[i].estimators_)`
(Yes, this is hacky, but no solution to this problem has been found yet. Note that with very large datasets, it might pay to just sample a number training examples that fits in the RAM of a big machine instead of training on all of it. Another option is to switch to linear models with SGD, those implement a partial_fit method, but obviously they're limited in the kind of functions they can learn.)
The short answer is "No, you can't". Classical Random Forest classifier is not an incremental or online classifier, so you can't discard training data while learning, and have to provide all the dataset at once.
Due to popularity of RF in machine learning (not least because of the good prediction results for some interesting cases), there are some attempts to implement online variation of Random Forest, but to my knowledge those are not yet implemented in any python ML package.
See Amir Saffari's page for such an approach (not Python).