5
votes
Accepted
What is the $\ell_{2, 1}$ norm?
$\ell_{2,1}$ is a matrix norm, as stated in this paper.
For a certain matrix $A \in \mathbb{R}^{r\times c}$,
we have
$$\|A\|_{2,1} = \sum_{i=1}^r \sqrt{\sum_{j=1}^c A_{ij}^2}$$
You first apply $\ell_2$...
3
votes
Accepted
How does the decision tree implicitly do feature selection?
Consider a dataset $S \in \mathbb{R}^{N \times (M + 1)}$ with $N$ observations (or examples), where each observation $S_i \in \mathbb{R}^{M + 1}$ is composed of $M$ elements, one value for each of the ...
- 39.6k
2
votes
What is a good descriptor for similar objects?
How to develop a program that can sort images by similarity is similar to the problem of how to develop a program that can sort words by how similar they look.
For example:
"theory" is more similar ...
- 219
2
votes
Accepted
How to recognize non-circular radial symmetry in images?
The Hough Transform extended to orthogonal ellipses uses this model, accumulating on $\theta$ for all $\{x, y\}$ with parameter matrix
\begin{Bmatrix}
c_x & c_y \\
r_x & r_y
\end{Bmatrix}
...
- 7,443
2
votes
How come that the addition of features can decrease the performance of a neural network?
Additional features can also cause overfitting if they have low or misleading information.
Consider the following problem:
$X = [1, 3, 3, 4, 5]$, $Y = [1, 3, 4, 4, 5]$.
Suppose that the real ...
- 9,207
2
votes
Accepted
Should I use my redundant feature as an auxiliary output or as another input feature?
For extra input that does not matter, you should not input it to the network.
Feature selection, the process of finding and selecting the most
useful features in a dataset, is a crucial step of ...
- 1,735
2
votes
Can neural networks be used to find features importance?
This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to ...
- 327
2
votes
How to analyze data before going for machine learning training?
Though there is no universal method which can be blindly used for all datasets, but here is what i usually do;
Fill missing values using interpolation or mean, if missing values
are less than 10-15 ...
- 166
2
votes
Accepted
When should I use feature learning as opposed to feature engineering?
manual feature engineering started becoming obsolete
That is wrong.
Any suggestion on when to use manual feature engineering, feature learning or a combination of the two?
Deep learning is ...
- 1,055
2
votes
the best choice to reduce a features vector
Feature selection -- the case in which the features are highly correlated is the prototypical case in which you want to select a subset of independent features that allows for an equal performance. ...
- 31
2
votes
Is automated feature engineering a path to general AI?
Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are meaningful, as opposed to those that simply ...
- 30.4k
2
votes
Accepted
When is adding a feature useless?
Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A.
This will help inform you how much redundancy there is between ...
- 30.4k
1
vote
Accepted
Methods of constructing input and ouput vectors in Reinforcement Learning with approximation function learning?
If you build a function like $Q(s,a)$ using DQN, you have the problem that given 100 actions, you'll need 100 forward pass of your network
Now, since neural networks can handle multiple outputs, we ...
- 1,358
1
vote
Accepted
Why does the training time of SVMs dramatically decrease after applying dimensionality reduction to the features?
SVM complexity is $O(\max(n,d)\min(n,d)^2)$ according to Chapelle, Olivier. "Training a support vector machine in the primal." Neural Computation 19.5 (2007): 1155-1178.
$n$ is the number of ...
- 249
1
vote
Is there a way to see the feature importance in deep learning (neural networks)?
In the current state, Deep learning for Tabular is not very popular, so it is very hard to find libraries that support feature importance. However, TabNet also provides the ...
- 1,140
1
vote
Which correlated feature should be eliminated from a model?
In practice multicollinearity could be very common if your features really act as correlated causes for your target. If multicollinearity is moderate or you're only interested in using your trained ML ...
- 950
1
vote
Which correlated feature should be eliminated from a model?
I appreciate you for asking the question. Well, speaking of statistics, the problem of multicollinearity is catered to using partial correlation.
Also, The correlation matrix is analyzed to understand ...
- 31
1
vote
Which type of neural network to use to classify data by which equation most likely generated it?
Any neural network might be able to find some pattern (if there is one), provided adequate data. But you can always optimize with right assumptions.
For instance, there might not be always a relation ...
- 265
1
vote
Features for a Content-Based recommendation system
Some features that have been found to work well for content-based recommender systems include:
Item category (e.g. food, clothing, electronics, etc.)
Item sub-category (e.g. type of food, type of ...
- 1,094
1
vote
Feature Engineering on transactional dataset clustering
The average transaction is a central measure, while the minimum and maximum transactions together give an idea of dispersion. However, these can be very sensitive to individual purchases that might ...
1
vote
Why my classification results are correlated with the proportionality of my data?
Despite how software might work, neural networks do not return labels. Neural networks return probabilities of class membership (typically fairly poor ones, which is a topic for a separate question). ...
- 548
1
vote
Why my classification results are correlated with the proportionality of my data?
I see two main issues here:
you have really few data
you're using a generic MLP
What you observe if just overfitting. You multi layer perceptron is just learning to predict the majority class cause ...
- 5,218
1
vote
Is there a way to select the subset of most important features using PCA?
There is a way to select the subset of the most important features using PCA. The basic idea is to choose variables according to the magnitude (from largest to smallest in absolute values) of their ...
- 11
1
vote
How many singular vectors do we need to calculate for SVD?
The number of singular vectors we need to find during SVD is not unique. The possible values for k are from 1 to $r$. Here, $r$ is the rank of matrix $A$, on which we are performing decomposition.
The ...
- 3,612
1
vote
Selecting features for a neural network: is it redundant to have a feature that is an average (or max, or min) of some other features
Or is there no clear answer and would this be something I'd only be able to figure out by testing against data?
That is the general rule you should always consider when looking at feature engineering ...
- 30.4k
1
vote
How to predict the best from a set of messages - best practice
One way you can definitely approach the problem is by using (Deep) Reinforcement Learning (DRL).
YouTube is actually using DRL as well to suggest videos to users in order to maximize users' engagement ...
- 815
1
vote
How much can the addition of new features improve the performance?
It depends on the used network as well as the feeding mechanism but let's give an example;
When working with LSTM, giving the time data (as an integer sequence) in addition to the time-series data(...
- 46
1
vote
How do I select the relevant features of the data?
Since you have all your data in a table, a relatively simple thing to do is to consider each column independently, and then seeing if the output variable (cost incurred) has a correlation to that.
If ...
Only top scored, non community-wiki answers of a minimum length are eligible