3

You should look into "missing values". This is an entire research field in itself. First, you need to identify the type of missing values: They can be missing purely at random. Whether they are missing or not is itself a useful feature, and should be treated as a class of its own. (Those two are the best case scenarios.) Whether they are missing ...


2

Yes, neural networks learn features themselves freeing you from the need to manually engineer them. I will illustrate it here with a toy problem. Let's assume that we want to learn the areas of parallelograms built on pairs of vectors: The input data are six coordinates: $(x_1, y_1, x_2, y_2, x_3, y_3)$. import numpy as np n_tr = 1000 # training data x_tr = ...


2

Feature engineering may be necessary when one cannot achieve acceptable error rate — within a budget or in principle. NN may be stalling due to information bottleneck: too many pigeons, not enough holes. In that case, custom features may provide slightly better information compression. (Alas, this is not a panacea: some layer(s) may still be too narrow. That'...


2

If all you features are binary, then, you don't need to apply normalization on them. Since their values are on the same scale already.


2

Automated feature engineering, if it is part of any aproach towards general intelligence, cannot be the whole solution. The search for features that are meaningful, as opposed to those that simply exist with no utility, needs some guidance. In machine learning, feature engineering is typically a search for features that improve performance at a specific task,...


2

Feature scaling happens to be a problem when a model is characterized by having a distance metric (or another kind of numerical evaluation for that matter). Therefore models such as support vector machines, neural networks, distance based clustering methods (e.g. k means) and linear/logistic regression are prone to changes by feature scaling. Those which are ...


2

Scaling only makes sense when there is something that reacts to that scale. Decision Trees though, just make a cut at a certain number. Imagine: For a feature that goes from 0 to 100 a cut at 50 may be improving performance. Scaling this down to 0 to 1 making the cut a 0.5 doesn't change a thing. Now on the other hand NN have some kind of activation function ...


2

This should be possible, considering universal approximation theorem you should be able to build a ann that approximates features that gives the most likely best feature set for a different net to train on. I would us a rnn for with a softmax output layer that ranks features by performance. You can find a good explanation of softmax here: https://developers....


1

I think you need to look into semi-supervised learning, which combines supervised and unsupervised learning for problems where large labelled datasets are not available. To use this family of techniques, you need a small labelled dataset and a large unlabelled one. Create a dataset over good athletes, lets say the ones who are professional, and the traits ...


1

I would distinguish at least 2 cases when it comes to a generic expression like prior knowledge: generic extra information provide to a model, really close if not the same as feature engineering. literal prior probability distributions used to initialize or guide a model during training. For the first case there's plenty of examples that we can provide. ...


1

Padding is a common practice both in image-processing (typically via CNNs) and in sequence-processing tasks (RNNs, Transformers). For CNNs all the standard convolutional layers - Conv1D, Conv2D and Conv3D,- have the padding argument. The padding values can be valid or same for 2d and 3d convolutions. And extra causal type of padding is possible for 1d ...


1

From what I believe, feature engineering is important, it's a part of the job of ML network designer. Network designing involves Feature engineering: What should be in the input to the network, as processed from similar or totally different data Deciding network shape, layer shapes, types of neurons in layers, etc. Feature engineering again (but labels), in ...


1

I think the wrong assumption here is that you've forgotten the cost of encoding the new features! MDL should be considered relative to the original or raw dataset. The idea is that you want to find an expression you could send to someone else that encodes the structure of the dataset in terms of the original variables. If you define new features, you need ...


Only top voted, non community-wiki answers of a minimum length are eligible