41
Python comes with a huge amount of inbuilt libraries. Many of the libraries are for Artificial Intelligence and Machine Learning. Some of the libraries are Tensorflow (which is high-level neural network library), scikit-learn (for data mining, data analysis and machine learning), pylearn2 (more flexible than scikit-learn), etc. The list keeps going and never ...
32
Keras is a simple and high-level neural networks library, written in Python, that works as a wrapper for Tensorflow and Theano. It's easy to learn and use. Using Keras is like working with Lego blocks. It was built so that people can do quick experiments and proofs-of-concept before launching into a full-scale build process.
With that in mind, it was made ...
32
The concept you are looking for is called epistemic uncertainty, also known as model uncertainty. You want the model to produce meaningful calibrated probabilities that quantify the real confidence of the model.
This is generally not possible with simple neural networks as they simply do not have this property, for this you need a Bayesian Neural Network (...
28
Practically all of the most popular and widely used deep-learning frameworks are implemented in Python on the surface and C/C++ under the hood.
I think the main reason is that Python is widely used in scientific and research communities, because it's easy to experiment with new ideas and code prototypes quickly in a language with minimal syntax like Python.
...
21
The bottleneck in a neural network is just a layer with less neurons then the layer below or above it. Having such a layer encourages the network to compress feature representations to best fit in the available space, in order to get the best loss during training.
In a CNN (such as Google's Inception network), bottleneck layers are added to reduce the ...
14
Your classifier is specifically learning the ways in which 0s are different from other digits,
not what it really means for a digit to be a zero.
Philosophically, you could say the model appears to have some powerful understanding when restricted
to a tightly controlled domain, but that facade is lifted as soon as you throw any sort of wrench in
the works....
11
There are many approaches to this kind of problem. The most obvious one is to create new features. The best features I can come up with is to transform the coordinates to spherical coordinates.
I have not found a way to do it in playground, so I just created a few features that should help with this (sin features). After 500 iterations it will saturate and ...
10
Great question! NN is very promising for this type of problem: Giraffe Chess. Lai's accomplishment was considered to be a pretty big deal, but unfortunately came just a few months before AlphaGo took the spotlight. (It all turned out well, in that Lai was subsequently hired by DeepMind, although not so well for the Giraffe engine;)
I've found Lai's ...
8
What attracts me to Python for my analysis work is the "full-stack" of tools that are available by virtue of being designed as a general purpose language vs. R as a domain specific language. The actual data analysis is only part of the story, and Python has rich tools and a clean full-featured language to get from the beginning to the end in a single ...
8
Imagine, you want to re-compute the last layer of a pre-trained model :
Input->[Freezed-Layers]->[Last-Layer-To-Re-Compute]->Output
To train [Last-Layer-To-Re-Compute], you need to evaluate outputs of [Freezed-Layers] multiple times for a given input data. In order to save time, you can compute these ouputs only once.
Input#1->[Freezed-Layers]...
8
From nvidia www (https://developer.nvidia.com/discover/lstm):
Accelerating Long Short-Term Memory using GPUs
The parallel processing capabilities of GPUs can accelerate the LSTM training and inference processes. GPUs are the de-facto standard for LSTM usage and deliver a 6x speedup during training and 140x higher throughput during inference when compared ...
8
I wanted to know how the performance of my net would be compared to the same in Tensor Flow. Not to specific but just a rough aproximation.
This is very hard to answer in specific terms because benchmarking is very hard and is often wrong.
The main point of TensorFlow as I see it is to make it easier for you to use a GPU and further allows you to use a ...
7
Python has a standard library in development, and a few for AI. It has an intuitive syntax, basic control flow, and data structures. It also supports interpretive run-time, without standard compiler languages. This makes Python especially useful for prototyping algorithms for AI.
7
I'm a chess player and my answer will be only on chess.
Training a neutral network with reinforcement learning isn't new, it has been done many times in the literature.
I'll briefly explain the common strategies.
The purpose of a network is to learn position evaluation. We all know a queen is stronger than a bishop, but can we make the network know about ...
7
Broken assumptions
Generalization relies on making strong assumptions (no free lunch, etc). If you break your assumptions, then you're not going to have a good time. A key assumption of a standard digit-recognition classifier like MNIST is that you're classifying pictures that actually contain a single digit. If your real data contains pictures that have ...
6
This should make a difference, but how big is the difference heavily depends on your task. However generally speaking, a smaller batch size will have a lower speed if counted in sample/minutes, but have a higher speed in batch/minutes. If the batch size is too small the batch/minute will be very low and therefore decreasing training speed severely. However a ...
5
Tensorflow bottleneck is the last pre prosessing phase before the actual training with data recognitions start. It is a phase where a data structure is formed from each training image that the final phase of training can take place and distinguish the image from every other image used in training material. Somewhat like a fingerprint of the image.
It is ...
5
All of these could be problem specific (except maybe accuracy). Most of it is documented here:
accuracy: Percentage of correct number of classifications
accuracy_baseline: Accuracy baseline based on labels mean. This is the best the model could do by always predicting one class. (source)
AUC or Area Under the (ROC) Curve is quite complicated, but tells you ...
5
RNN's stand for Recurrent Neural Networks which is, in fact, Deep Learning.
There has to be a loss since you're dealing with supervised learning and the typical loss metrics used are the same as you would see in feedforward networks (usually binary cross-entropy), the main difference being loss would be calculated between the true label at a particular time ...
4
I think you should get familiar with reinforcement learning. In this field of machine learning the agent interacts whit its environment and after that the agent gets some reward. Now, the agent is the neural network the environment is the game and the agent can get a reward +1 if it wins or -1 if loses. You can use this state, action, reward experienc tuple ...
4
Ideally neural networks should be able to find out the function out on it's own without us providing the spherical features. After some experimentation I was able to reach a configuration where we do not need anything except $X_1$ and $X_2$. This net converged after about 1500 epochs which is quite long. So the best way might still be to add additional ...
4
Python has rich library, it is also object oriented, easy to program. It can be also used as frontend language. That's why it is used in artificial intelligence. Rather than AI it is also used in machine learning, soft computing, NLP programming and also used as web scripting or in Ethical hacking.
4
OpenCV does include 2D filter convolution functions for custom separable and non-separable filters. The latter uses DFT for large filters, which may or may not be faster than the conventional method. It also includes (partial?) support for deep nets with various types of layers. Theoretically, you should be able to stitch everything together into a complete ...
4
I found that there are cuDNN accelerated cells in Keras for example: https://keras.io/layers/recurrent/#cudnnlstm
They very fast. The normal LSTM cells are faster on CPU then on GPU.
Also see here for a comparisem: https://wiki.eniak.de/ml/geschwindigkeitsvergleich_keras_lstm_und_cudnnlstm
4
You can use CNN on any data, but it's recommended to use CNN only on data that have spatial features (It might still work on data that doesn't have spatial feature, see DuttaA's comment below).
For example, in image, connection between pixels in some area give you another feature (e.g. edge) instead of feature from one pixel (e.g. color). So, as long as you ...
4
So I am assuming that you are trying to detect a lego brick from the image. One idea is that you can use transfer learning. Leveraging a pre-trained machine learning model is called transfer learning. The underlying idea behind transfer learning is that one takes a well-trained model from one dataset or domain, and applies it to a new one. François Chollet ...
4
It actually depends on a couple of things here -
How many output classes do you have? If you have only 2 or 3 classes, it is a very easy task for the classifier that you have built. So, it is highly possible that convergence has occurred.
As @Djib2011 mentioned already, if your input training set is not balanced and is heavier with one of the output classes ...
4
NOTE: All the observations and results are from the paper The Lottery Ticket Hypothesis: Finding Sparse, Trainable Neural Networks.
To answer your questions one by one:
Yes there are ways to determine which filters have more impact on the output. Its a very naive way but works very good in practice. Filters with small weights impact output less (according ...
4
A neural network is not good at selecting a function based on those 3 input parameters, because of the way a neuron is setup.
What you should do is either make a neural network for each operation, or use different input neurons for each operation. E.g. 2 input neurons for the addition operation, 2 for the multiplication, and 2 for the minus. 6 inputs in ...
4
Trying to address all the questions asked in the end in the same order
Most definitely possible.
I would say its best you approach this with segmentation to start with.
Just use a free GPU runtime notebook service such as Google Colab or Kaggle Kernels. But you would not directly be able to integrate with the device, you'd have to keep moving input and ...
Only top voted, non community-wiki answers of a minimum length are eligible
