New answers tagged

2

Smoothness here is the mathematical definition, so as you implied smoothness is ruled out by output data with sharp spikes or discontinuous jumps (and possibly the data of the gradient, the gradient's gradient, ad infinitum, depending on who defines smoothness). By any definition a lot of activation functions are not smooth, for example RELU. This means ...


0

This is actually a highly technical term which has been kind of misused and overgenralized in many places. What does 'convergence' mean in a literal sense? It simply means that a series of terms indexed by $\mathbb{N}$ ($X_1,X_2,X_3,..$) tends to a certain fixed value say $X$ as $\mathbb{N} \rightarrow \infty$, but may not achieve the fixed value. (there are ...


3

Is there a term for the humans who do [machine] learning? Typically you will see "AI researchers" for people studying machine intelligence in general, or "data scientists" for people working with statistics or studying specific solutions in machine learning. Both those terms are used quite flexibly, and generally understood to be ...


1

In the least-squares SVM (LS-SVM) the non-zero Lagrange multipliers ($\alpha$) are the support values. The corresponding data points are the support vectors. Johan Suykens explains this in Least Squares Support Vector Machines.


3

You're right! The generative model $f$ is not the same as the probability density (p.d.f.) function $p_{data}$. The kind of phrases you've referred to are to be interpreted informally. You learn $f$ with the hope that sampling a latent vector $z$ from some known distribution (from which it is easy to sample), results in $f(z)$ that has the probability ...


1

Embedding is the process of representing data (from a source domain) in a new (or target) domain. Usually, the source domain is discrete, and the target domain is continuous. For example, embedding words into the continuous vector space can be done by the word2vec method. The main reason behind using the embedding is doing meaningful mathematical ...


0

I'll give you my initial $0.02 for symmetric relaxation or relaxation in general in working with neural networks. The book covers 'Weight perturbation' and this is a basic outline of that. Say you want to host a wedding and every person gives you a 'must-have' list of requirements for them to attend. You can abide by all the requirements of each wedding ...


1

Grammatical evolution To understand what a codon is, we need to understand what GE is, so let me first provide a brief description of this approach. Grammatical evolution (GE) is an approach to genetic programming where the genotypes are binary (or integer) arrays, which are mapped to the phenotypes (i.e. the actual solutions, which can be represented as ...


2

A unified neural network model consists of one neural network as opposed to other models that rely on two or more neural networks. For example, from page two of the YOLO paper: 2. Unified Detection We unify the separate components of object detection into a single neural network. Our network uses features from the entire image to predict each bounding box. ...


Top 50 recent answers are included