4
votes
Accepted
Why are decision trees and random forests scale invariant?
Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number.
Imagine: For a feature that goes from 0 to 100 a cut at 50 may ...
- 321
3
votes
How to perform prediction when some features have missing values?
You should look into "missing values". This is an entire research field in itself.
First, you need to identify the type of missing values:
They can be missing purely at random.
Whether they ...
3
votes
Accepted
When is it necessary to manually extract features to feed into the neural network rather than providing raw data?
Yes, neural networks learn features themselves freeing you from the need to manually engineer them. I will illustrate it here with a toy problem.
Let's assume that we want to learn the areas of ...
2
votes
When is it necessary to manually extract features to feed into the neural network rather than providing raw data?
Feature engineering may be necessary when one cannot achieve acceptable error rate — within a budget or in principle.
NN may be stalling due to information bottleneck: too many pigeons, not enough ...
- 121
2
votes
Accepted
Does feature scaling have any benefits if all features are on the same scale?
If all you features are binary, then, you don't need to apply normalization on them. Since their values are on the same scale already.
- 868
2
votes
Is automated feature engineering a path to general AI?
Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are meaningful, as opposed to those that simply ...
- 32.9k
2
votes
Why are decision trees and random forests scale invariant?
Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector ...
- 21
2
votes
Features for a Content-Based recommendation system
Some features that have been found to work well for content-based recommender systems include:
Item category (e.g. food, clothing, electronics, etc.)
Item sub-category (e.g. type of food, type of ...
- 1,114
2
votes
Can neural networks be used to find features importance?
This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to ...
- 327
2
votes
Accepted
permutation importance shuffling
Distributional mismatch, the value that you pick might be out of distribution, and if inside the distribution, might be that it correspond to the correct one for some samples
Again, distributional ...
- 2,473
1
vote
Accepted
Is a linear regression model able to figure out the relation of division among two features?
What you mentioned is called feature engineering, i.e., the process of using domain knowledge to create new features or modify existing ones in a way that makes machine learning algorithms work more ...
- 166
1
vote
Non constant Feature Importance
I think you can derive new features that should be more stable over time using PCA. A more simple approaches is to calculate the technical indicators of these numerical features such as RSI, moving ...
- 11
1
vote
Can we generate labels for an unlabelled dataset by doing some feature engineering?
Whether you can do this is going to depend on what "recurring donor" is supposed to mean. Defining it based on age and number of donations seems pretty sensible to me, but, in the end, it's ...
- 1,369
1
vote
Feature Extraction for timeseries temperature signal
Cross correlation
A high cross-correlation value indicates a strong correlation between the two signals, which can be useful for identifying the specific event that one of the signals is sensitive to.
...
- 111
1
vote
Feature Engineering on transactional dataset clustering
The average transaction is a central measure, while the minimum and maximum transactions together give an idea of dispersion. However, these can be very sensitive to individual purchases that might ...
1
vote
How to handle list features in clustering?
You essentially want to have a numerical value to represent the similarity of the lists of two distinct objects. There are a number of metrics to deal with that, eg the Jaccard Index or Dice's ...
- 5,397
1
vote
Sensible integer embedding/encoding for distinguishing elements of a set?
As you have asserted that the id is not meaningful of itself, it probably doesn't matter how you encode it.
I would recommend the following, in order:
Don't encode the id. It is not clear that you ...
- 32.9k
1
vote
Are derived or computed inputs bad for CNNs?
It seems to me that, you're basically asking whether feature engineering is bad or not. It's not necessarily bad, but the main advantage of deep neural networks stem from the fact that they do feature ...
- 1,020
1
vote
Accepted
Generating a dataset from data with "assumed" lables
I think you need to look into semi-supervised learning, which combines supervised and unsupervised learning for problems where large labelled datasets are not available. To use this family of ...
- 506
1
vote
What can be an example for the prior knowledge used in Deep Learning systems?
I would distinguish at least 2 cases when it comes to a generic expression like prior knowledge:
generic extra information provide to a model, really close if not the same as feature engineering.
...
- 5,378
1
vote
Feeding CNN FFT of an image, a dumb idea?
FFT is in essence linear transformation of the input image and can be represented by application of convolutional filter of the same size as image on the input.
Provided, the convolutinoal neural ...
- 2,724
1
vote
Accepted
Is it a good practice to pad signal before feature extraction?
Padding is a common practice both in image-processing (typically via CNNs) and in sequence-processing tasks (RNNs, Transformers).
For CNNs all the standard convolutional layers - Conv1D, Conv2D and ...
- 2,554
1
vote
Is feature engineer an important step for a deep learning approach?
From what I believe, feature engineering is important, it's a part of the job of ML network designer.
Network designing involves
Feature engineering: What should be in the input to the network, as ...
- 1,293
1
vote
Can feature engineering change the selection of the model according to the minimum description length?
I think the wrong assumption here is that you've forgotten the cost of encoding the new features!
MDL should be considered relative to the original or raw dataset. The idea is that you want to find ...
- 9,327
Only top scored, non community-wiki answers of a minimum length are eligible