Skip to main content
Share Your Experience: Take the 2024 Developer Survey
5 votes
Accepted

What is the $\ell_{2, 1}$ norm?

$\ell_{2,1}$ is a matrix norm, as stated in this paper. For a certain matrix $A \in \mathbb{R}^{r\times c}$, we have $$\|A\|_{2,1} = \sum_{i=1}^r \sqrt{\sum_{j=1}^c A_{ij}^2}$$ You first apply $\ell_2$...
bluesisnotrock's user avatar
3 votes
Accepted

How does the decision tree implicitly do feature selection?

Consider a dataset $S \in \mathbb{R}^{N \times (M + 1)}$ with $N$ observations (or examples), where each observation $S_i \in \mathbb{R}^{M + 1}$ is composed of $M$ elements, one value for each of the ...
nbro's user avatar
  • 40.8k
2 votes

What is a good descriptor for similar objects?

How to develop a program that can sort images by similarity is similar to the problem of how to develop a program that can sort words by how similar they look. For example: "theory" is more similar ...
Tone Škoda's user avatar
2 votes
Accepted

How to recognize non-circular radial symmetry in images?

The Hough Transform extended to orthogonal ellipses uses this model, accumulating on $\theta$ for all $\{x, y\}$ with parameter matrix \begin{Bmatrix} c_x & c_y \\ r_x & r_y \end{Bmatrix} ...
Douglas Daseeco's user avatar
2 votes

How come that the addition of features can decrease the performance of a neural network?

Additional features can also cause overfitting if they have low or misleading information. Consider the following problem: $X = [1, 3, 3, 4, 5]$, $Y = [1, 3, 4, 4, 5]$. Suppose that the real ...
John Doucette's user avatar
2 votes
Accepted

Should I use my redundant feature as an auxiliary output or as another input feature?

For extra input that does not matter, you should not input it to the network. Feature selection, the process of finding and selecting the most useful features in a dataset, is a crucial step of ...
Clement's user avatar
  • 1,745
2 votes

Can neural networks be used to find features importance?

This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to ...
nickw's user avatar
  • 327
2 votes

How to analyze data before going for machine learning training?

Though there is no universal method which can be blindly used for all datasets, but here is what i usually do; Fill missing values using interpolation or mean, if missing values are less than 10-15 ...
jazib jamil's user avatar
2 votes
Accepted

When should I use feature learning as opposed to feature engineering?

manual feature engineering started becoming obsolete That is wrong. Any suggestion on when to use manual feature engineering, feature learning or a combination of the two? Deep learning is ...
Martin Thoma's user avatar
  • 1,055
2 votes

the best choice to reduce a features vector

Feature selection -- the case in which the features are highly correlated is the prototypical case in which you want to select a subset of independent features that allows for an equal performance. ...
Peblo's user avatar
  • 31
2 votes

Is automated feature engineering a path to general AI?

Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are meaningful, as opposed to those that simply ...
Neil Slater's user avatar
  • 32.4k
2 votes
Accepted

When is adding a feature useless?

Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A. This will help inform you how much redundancy there is between ...
Neil Slater's user avatar
  • 32.4k
1 vote
Accepted

Methods of constructing input and ouput vectors in Reinforcement Learning with approximation function learning?

If you build a function like $Q(s,a)$ using DQN, you have the problem that given 100 actions, you'll need 100 forward pass of your network Now, since neural networks can handle multiple outputs, we ...
Alberto's user avatar
  • 2,163
1 vote
Accepted

Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?

SVM complexity is $O(\max(n,d)\min(n,d)^2)$ according to Chapelle, Olivier. "Training a support vector machine in the primal." Neural Computation 19.5 (2007): 1155-1178. $n$ is the number of ...
Cameron Chandler's user avatar
1 vote

Is there a way to see the feature importance in deep learning (neural networks)?

In the current state, Deep learning for Tabular is not very popular, so it is very hard to find libraries that support feature importance. However, TabNet also provides the ...
Minh-Long Luu's user avatar
1 vote

Which correlated feature should be eliminated from a model?

In practice multicollinearity could be very common if your features really act as correlated causes for your target. If multicollinearity is moderate or you're only interested in using your trained ML ...
cinch's user avatar
  • 2,277
1 vote

Which correlated feature should be eliminated from a model?

I appreciate you for asking the question. Well, speaking of statistics, the problem of multicollinearity is catered to using partial correlation. Also, The correlation matrix is analyzed to understand ...
oseekero's user avatar
1 vote

Which type of neural network to use to classify data by which equation most likely generated it?

Any neural network might be able to find some pattern (if there is one), provided adequate data. But you can always optimize with right assumptions. For instance, there might not be always a relation ...
Aether's user avatar
  • 265
1 vote

Features for a Content-Based recommendation system

Some features that have been found to work well for content-based recommender systems include: Item category (e.g. food, clothing, electronics, etc.) Item sub-category (e.g. type of food, type of ...
Faizy's user avatar
  • 1,104
1 vote

Feature Engineering on transactional dataset clustering

The average transaction is a central measure, while the minimum and maximum transactions together give an idea of dispersion. However, these can be very sensitive to individual purchases that might ...
Jaume Oliver Lafont's user avatar
1 vote

Why my classification results are correlated with the proportionality of my data?

Despite how software might work, neural networks do not return labels. Neural networks return probabilities of class membership (typically fairly poor ones, which is a topic for a separate question). ...
Dave's user avatar
  • 618
1 vote

Why my classification results are correlated with the proportionality of my data?

I see two main issues here: you have really few data you're using a generic MLP What you observe if just overfitting. You multi layer perceptron is just learning to predict the majority class cause ...
Edoardo Guerriero's user avatar
1 vote

How many singular vectors do we need to calculate for SVD?

The number of singular vectors we need to find during SVD is not unique. The possible values for k are from 1 to $r$. Here, $r$ is the rank of matrix $A$, on which we are performing decomposition. The ...
hanugm's user avatar
  • 3,890
1 vote

Selecting features for a neural network: is it redundant to have a feature that is an average (or max, or min) of some other features

Or is there no clear answer and would this be something I'd only be able to figure out by testing against data? That is the general rule you should always consider when looking at feature engineering ...
Neil Slater's user avatar
  • 32.4k
1 vote

How to predict the best from a set of messages - best practice

One way you can definitely approach the problem is by using (Deep) Reinforcement Learning (DRL). YouTube is actually using DRL as well to suggest videos to users in order to maximize users' engagement ...
Daniel B.'s user avatar
  • 825
1 vote

How much can the addition of new features improve the performance?

It depends on the used network as well as the feeding mechanism but let's give an example; When working with LSTM, giving the time data (as an integer sequence) in addition to the time-series data(...
laserwoman's user avatar
1 vote

How do I select the relevant features of the data?

Since you have all your data in a table, a relatively simple thing to do is to consider each column independently, and then seeing if the output variable (cost incurred) has a correlation to that. If ...

Only top scored, non community-wiki answers of a minimum length are eligible