Skip to main content
Share Your Experience: Take the 2024 Developer Survey
4 votes
Accepted

Why are decision trees and random forests scale invariant?

Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number. Imagine: For a feature that goes from 0 to 100 a cut at 50 may ...
N. Kiefer's user avatar
  • 321
3 votes

How to perform prediction when some features have missing values?

You should look into "missing values". This is an entire research field in itself. First, you need to identify the type of missing values: They can be missing purely at random. Whether they ...
Robby Goetschalckx's user avatar
2 votes
Accepted

When is it necessary to manually extract features to feed into the neural network rather than providing raw data?

Yes, neural networks learn features themselves freeing you from the need to manually engineer them. I will illustrate it here with a toy problem. Let's assume that we want to learn the areas of ...
Vladislav Gladkikh's user avatar
2 votes

When is it necessary to manually extract features to feed into the neural network rather than providing raw data?

Feature engineering may be necessary when one cannot achieve acceptable error rate — within a budget or in principle. NN may be stalling due to information bottleneck: too many pigeons, not enough ...
ShadowsInRain's user avatar
2 votes
Accepted

Does feature scaling have any benefits if all features are on the same scale?

If all you features are binary, then, you don't need to apply normalization on them. Since their values are on the same scale already.
Abhishek Verma's user avatar
2 votes

Is automated feature engineering a path to general AI?

Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are meaningful, as opposed to those that simply ...
Neil Slater's user avatar
  • 32.4k
2 votes

Why are decision trees and random forests scale invariant?

Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector ...
Ghostpunk's user avatar
2 votes
Accepted

permutation importance shuffling

Distributional mismatch, the value that you pick might be out of distribution, and if inside the distribution, might be that it correspond to the correct one for some samples Again, distributional ...
Alberto's user avatar
  • 2,153
2 votes

Can neural networks be used to find features importance?

This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to ...
nickw's user avatar
  • 327
1 vote

Non constant Feature Importance

I think you can derive new features that should be more stable over time using PCA. A more simple approaches is to calculate the technical indicators of these numerical features such as RSI, moving ...
Gene's user avatar
  • 11
1 vote

Can we generate labels for an unlabelled dataset by doing some feature engineering?

Whether you can do this is going to depend on what "recurring donor" is supposed to mean. Defining it based on age and number of donations seems pretty sensible to me, but, in the end, it's ...
Alexander Wan's user avatar
1 vote

Feature Extraction for timeseries temperature signal

Cross correlation A high cross-correlation value indicates a strong correlation between the two signals, which can be useful for identifying the specific event that one of the signals is sensitive to. ...
Joypal's user avatar
  • 111
1 vote

Features for a Content-Based recommendation system

Some features that have been found to work well for content-based recommender systems include: Item category (e.g. food, clothing, electronics, etc.) Item sub-category (e.g. type of food, type of ...
Faizy's user avatar
  • 1,104
1 vote

Feature Engineering on transactional dataset clustering

The average transaction is a central measure, while the minimum and maximum transactions together give an idea of dispersion. However, these can be very sensitive to individual purchases that might ...
Jaume Oliver Lafont's user avatar
1 vote

How to handle list features in clustering?

You essentially want to have a numerical value to represent the similarity of the lists of two distinct objects. There are a number of metrics to deal with that, eg the Jaccard Index or Dice's ...
Oliver Mason's user avatar
  • 5,387
1 vote

Sensible integer embedding/encoding for distinguishing elements of a set?

As you have asserted that the id is not meaningful of itself, it probably doesn't matter how you encode it. I would recommend the following, in order: Don't encode the id. It is not clear that you ...
Neil Slater's user avatar
  • 32.4k
1 vote

Are derived or computed inputs bad for CNNs?

It seems to me that, you're basically asking whether feature engineering is bad or not. It's not necessarily bad, but the main advantage of deep neural networks stem from the fact that they do feature ...
SpiderRico's user avatar
  • 1,000
1 vote
Accepted

Generating a dataset from data with "assumed" lables

I think you need to look into semi-supervised learning, which combines supervised and unsupervised learning for problems where large labelled datasets are not available. To use this family of ...
Avatrin's user avatar
  • 496
1 vote

What can be an example for the prior knowledge used in Deep Learning systems?

I would distinguish at least 2 cases when it comes to a generic expression like prior knowledge: generic extra information provide to a model, really close if not the same as feature engineering. ...
Edoardo Guerriero's user avatar
1 vote

Feeding CNN FFT of an image, a dumb idea?

FFT is in essence linear transformation of the input image and can be represented by application of convolutional filter of the same size as image on the input. Provided, the convolutinoal neural ...
spiridon_the_sun_rotator's user avatar
1 vote
Accepted

Is it a good practice to pad signal before feature extraction?

Padding is a common practice both in image-processing (typically via CNNs) and in sequence-processing tasks (RNNs, Transformers). For CNNs all the standard convolutional layers - Conv1D, Conv2D and ...
Kostya's user avatar
  • 2,534
1 vote

Is feature engineer an important step for a deep learning approach?

From what I believe, feature engineering is important, it's a part of the job of ML network designer. Network designing involves Feature engineering: What should be in the input to the network, as ...
Dan D.'s user avatar
  • 1,283
1 vote

Can feature engineering change the selection of the model according to the minimum description length?

I think the wrong assumption here is that you've forgotten the cost of encoding the new features! MDL should be considered relative to the original or raw dataset. The idea is that you want to find ...
John Doucette's user avatar

Only top scored, non community-wiki answers of a minimum length are eligible