New answers tagged

1

Turns out the reason is because, for places where a dot is shown in the image above, they're actually element-wise multiplications, not dot products. A lot of sources use an X or . to denote multiplication, but don't clearly indicate they mean element-wise multiplication.


0

Several ways to add new class to trained model which require just training for new classes. Incremental training Transfer Learning Twice Continual learning approaches (Regularization, Expansion, Rehearsal)


1

SVM complexity is $O(\max(n,d)\min(n,d)^2)$ according to Chapelle, Olivier. "Training a support vector machine in the primal." Neural Computation 19.5 (2007): 1155-1178. $n$ is the number of instances and $d$ is the number of dimensions. I'm assuming that you have more instances than dimensions giving a complexity of $O(nd^2)$. Hopefully this ...


3

In some sense, you're right that a neural net is just another tool to fit data. However, it's quite the tool! There's this universal approximation theorem saying that, under decent conditions, a neural network can get as close as you want to a wide class of functions. This means that you can get the network to give you complicated shapes with squiggles all ...


2

is it common to deal with weights and biases in everyday tasks or in most of the cases existing algorithms do it well? No; and it is no coincidence that you will not be able to find any reference to such a practice in any course or tutorial about neural networks. Such a practice would require a whole additional level of (business/SME) know-how in order to ...


1

Is there an AI technology out there or being developed that can predict human behaviour ? If it can predict (all) human behavior, it can act as an human, thus, it will be the first real (strong) AI. This has not happened yet. I must remark that the question contains a lot of weakly defined terms. Fix these terms can help to work in the question subject: &...


1

I have not implement the backprop of a bi-directional RNN from scratch so I can't be sure my answer is correct but I hope it helps. You can see how bi-directional RNN works from this video from Andrew NG. I got the image below from that video: For more clarity: So if you know how to backprop through a simple RNN, you should be able to do so for bi-...


0

I only have one good news... There is nothing wrong with your code. Neural networks tend to do that. Especially with a really complex function. Increasing the amount of neurons will not decrease how the error is distributed. There are better loss functions for each case but is not a really effective solution. Neural networks are really good managing noise. ...


1

In a simple linear model of the form $y = \beta_0 + \beta_1 x $ we can see that increasing $x$ by a unit will increase the prediction on $y$ by $\beta_1$. Here we can completely determine what the effect on the models prediction will be by increasing $x$. With more complex models such as neural networks it is much more difficult to tell due to all the ...


1

Formula Theory The softmax regression function is a higher dimensional generalisation of the logistic function (commonly known as sigmoid ). It's commonly used in multiclass regression problems where we need to get a normalized output over a probability distribution. It takes the exponent of each output of a layer and sums it over all the exponents. This ...


0

You will need to convert that to something that a neural network can understand. Movie Name Is useless. At least you want to judge a movie by its name. Description You will need to perform a Tokenization. Grab the x's more common words and convert them on an array. I recommend you to see these videos from TensorFlow. https://www.youtube.com/watch?v=...


0

After reading your question I can relate it to the Representation Learning papers such as SimCLR and SwAV. These models use a "Big Task agnostic CNN" to obtain smaller representations of the images and then they train another CNN for classification. I suggest you read Big Self-Supervised Models are Strong Semi-Supervised Learners by Ting Chen, ...


1

Use of Transposed Convolution can lead to checkerboard artifacts. So we prefer to up-sample and then apply convolution. You can check this article for more information https://distill.pub/2016/deconv-checkerboard/.


3

SOM (Self-Organinizing Map) is a type of artificial neural network (ANN), introduced by the Finnish professor Teuvo Kohonen in the 1980s, that is trained using unsupervised learning to produce a low-dimensional, discretized representation of the input space of the training samples, called a map, and is therefore a method to do dimensionality reduction. SOM ...


2

Section 5.2 Error Decomposition of the book Understanding Machine Learning: From Theory to Algorithms (2014) gives a description of the approximation error and estimation error in the context of empirical risk minimization (ERM), so in the context of learning theory. I will just summarise their definition. If you want to know more about these topics, I ...


1

The problem you state is a well known problem, and it is called "keyword spotting" os KWS. If you add a wake up word before it (like "hey google/siri"), you can also use "voice command" system to alleviate the problem. There are two kind of KWS systems: those which develop to detect a hard coded set of keywords, and those who ...


0

It is hard to tell what exactly is better because these are hyperparameters. However, the sigmoid activation function is closer to biological neurons. In the paper below, Bengio demonstrates why ReLU activation functions are better for hidden layers. In summary, they increase the sparsity of calculations (matrix in each layer shod multiply to its relative ...


1

If you have an erratic loss landscape, it can lead to an unstable learning curve. Thus, it's always better to choose a simpler function which creates a simple landscape. Sometimes even due to uneven dataset distribution, we can observe those jumps/irregularities in the training curve. And yes, those jumps do mean it might've found something significant in ...


2

There is an approach to machine learning, called Simulated Annealing, which varies the rate: starting from a large rate, it is slowly reduced over time. The general idea is that the initial larger rate will cover a broader range, while the increasingly lower rate then produces a less 'erratic' climb towards a maximum. If you only use a low rate, you risk ...


5

It really depends on what words you put after "ground truth". Sometimes people will talk about "ground truth labels", for example in the context of classification or regression problems. The "ground truth labels" in such a case would refer to the true labels of instances; the labels that we use as target labels for instances ...


7

In the context of ML, ground truth refers to information provided by direct observation (empirical evidence). If you're training an algorithm to classify your data, then the ground truth will be the actual, true labels which could for example be manually annotated by an domain expert. Please note that the models prediction or the inferred labels, are not ...


0

Kintu, TF - have built value for fingerprint: the fingerprint method is farmhash64 FarmHash64 provides a portable 64-bit hash function for strings (byte array). The function mix the input bits thoroughly but is not suitable for cryptography. FarmHash64 by default, which provides a consistent hashed output example: # Importing the library import tensorflow ...


1

In Boosting, we improve the overall metrics of the model by sequentially building weak models and then building upon the weak metrics of previous models. We start out by applying basic non-specific algorithms to the problem, which returns some weak prediction functions by taking arbitrary solutions (like sparse weights or assigning equal weights/attention). ...


6

As @desertnaut mentioned in the comment No weak learner becomes strong; it is the ensemble of the weak learners that turns out to be strong Boosting is an ensemble method that integrates multiple models(called as weak learners) to produce a supermodel (Strong learner). Basically boosting is to train weak learners sequentially, each trying to correct its ...


3

Parametric Methods A parametric approach (Regression, Linear Support Vector Machines) has a fixed number of parameters and it makes a lot of assumptions about the data. This is because they are used for known data distributions. i.e, it makes a lot of presumptions about the data Non-Parametric Methods A non-parametric approach (k-Nearest Neighbours, Decision ...


1

The standard tool to work with XML files is XSLT. You may not need AI to solve this problem. But.. you have to learn how to program with XSLT ;) On Windows you can use MSXML, if you work from C++ - msxsl.exe, C# has internal supoort for XSLT. That is what I know about. There are also non-MS tools.


4

In machine learning, the term bias can refer to at least 2 related concepts A (learnable) parameter of a model, such as a linear regression model, which allows you to learn a shifted function. For example, in the case of a linear regression model $y = f(x) = mx + b$, the bias $b$ allows you to shift the straight-line up an down: without the bias, you would ...


1

You take a bunch of weak learners, each of them trained on a subset of the data. You then just get all of them to make a prediction, and you learn how much you can trust each one, resulting in a weighted vote or other type of combination of the individual predictions.


0

We have a parametric model and non-parametric models a learning model that summarize data with a set of parameters of fixed size(independent of the number of training exmample) is called a parametric modeland if it couldn't do that we say non parametric model. The non-parametric model is good when you have a lot of data and no prior knowledge and when you ...


-1

When we are talking about sparse vectors, the dimensions of vectors are high and if we go for a feature cross to get non-linearinty for the model then the number of dimensions goes further and most of them we don't need it. When the model features are high then our model will be easily going to have high variance if we trained too much. We need to do ...


2

If the learning rate is greater than or equal to $1$ the Robbins-Monro condition $$\sum _{{t=0}}^{{\infty }}a_{t}^{2}<\infty\label{1}\tag{1},$$ where $a_t$ is the learning rate at iteration $t$, does not hold (given that a number bigger than $1$ squared becomes a bigger number), so stochastic gradient descent is not generally guaranteed to converge to a ...


1

I think the best thing to use here is a form of "structured prediction". Our "target" is a sequence of operations. The framework of structured prediction allows us to chain together as many filters as we want. With a neural network of fixed architecture, you would have to make sure you have enough space for all the filters you might need.


2

T. Mitchell defines machine learning in "Machine Learning" book as a computer program is said to learn from experience 𝐸 concerning some class of tasks 𝑇 and performance measure 𝑃, if its performance at tasks in 𝑇, as measured by 𝑃, improves with experience 𝐸 Hence, based on the above definition, we can't say a machine learning method to ...


2

Is it trying to make sure there is no symmetry in the gradients? The aim of weight initialization is to make sure that we don't converge to a trivial solution. That's why we have different kinds of initialization depending on the dataset type. So, Yes it is trying to avoid symmetry. Is it trying to allow the gradients to be large so it can quickly converge?...


0

We usually divide the dataset into multiple subsets namely (training, validation and test sets). During training, we validate the model against the validation set. And during testing, we use the test dataset to obtain metrics for the model. We should make sure the subsets are taken from the same sample. Once you've tested it against the test subset, there's ...


Top 50 recent answers are included