0
$\begingroup$

Features in machine learning are the attributes of the elements of a data set. They are considered as random variables in probability.

Consider the following excerpt from 1.1: The deep learning revolution of the textbook named Deep learning with PyTorch by Eli Stevens, Luca Antiga, Thomas Viehmann,

On the right, with deep learning, the raw data is fed to an algorithm that extracts hierarchical features automatically, guided by the optimization of its own performance on the task; the results will be as good as the ability of the practitioner to drive the algorithm toward its goal.

When can we call a feature hierarchical? Does it refer to a random variable that is a (function on) derived from some other random variables?

Improve this question
$\endgroup$
6
  • 1
    $\begingroup$ It may be a good idea to quote/cite something that uses the term/expression "hierarchical feature". I suppose that you're talking about the hierarchical nature of neural networks, which are interpreted as (hierarchical) feature extractors. However, I am not sure if "hierarchical feature" is a common term. $\endgroup$
    – nbro
    Commented Oct 8, 2021 at 12:35
  • $\begingroup$ Yeah, @nbro. It is true. So, we can call a feature hierarchial only if it is obtained from hidden or output layers of a neural network? $\endgroup$
    – hanugm
    Commented Oct 8, 2021 at 22:07
  • 1
    $\begingroup$ I'm not sure if this term is well-defined. It's common to say that neural networks learn "hierarchical representations of the data" (because they are essentially hierarchies of layers), and representations can be viewed as "features" of the data, but I've never seen a mathematical definition of "hierarchical feature". Intuitively, hierarchical features are associated with the hierarchy that the neural network represents, but I don't know the exact answer to your question because, as I said, I am not aware of a mathematical definition of "hierarchical feature". $\endgroup$
    – nbro
    Commented Jan 15, 2022 at 0:44
  • 1
    $\begingroup$ I guess they are just referring to the hierarchical nature of the neural network, which "stores" the "representation" of the data. For example, here, after a quick search, they use the term "Hierarchical Feature Extraction". This makes more sense to me, because they are just saying that the "feature extraction" is hierarchical, i.e. we extract the features as a hierarchy, they are not saying that features are hierarchical. $\endgroup$
    – nbro
    Commented Jan 15, 2022 at 0:46
  • $\begingroup$ But, saying that features are hierarchical would also be ok, if you're just referring to the neural network as the representation/features of the data. Note: this is just my interpretation of that term. So, take these words with a grain of salt. $\endgroup$
    – nbro
    Commented Jan 15, 2022 at 0:46

1 Answer 1

Reset to default
1
$\begingroup$

You can find a brief explanation of hierarchical feature selection in the following from "An Empirical Evaluation of Hierarchical Feature Selection Methods for Classification in Bioinformatics Datasets with Gene Ontology-based Features" paper:

Hierarchical feature selection is a new research area in machine learning/data mining, which consists of performing feature selection by exploiting dependency relationships among hierarchically structured features.

Therefore, hierarchical features correspond to the dependency structure between features.

Improve this answer
$\endgroup$

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .