I have seen many places that features and inputs have been used interchangeably when talking about machine learning especially deep neural networks. I want to know if they are indeed the same thing or there is a difference between between the two.
$\begingroup$
$\endgroup$
Add a comment
|
2 Answers
$\begingroup$
$\endgroup$
5
An input usually refers to an example (sometimes also known as sample, observation or data point) $x$ from a dataset that you pass to the model. For example, in supervised learning, you have a labelled dataset $D = \{(x_i, y_i)\}_{i=1}^N$, where $x_i$ is the $i$th input and $y_i$ the corresponding label (aka target or output).
This is similar to the terminology used for functions. For example, if you have the function $f: \mathcal{X} \rightarrow \mathcal{Y}$, then $x \in \mathcal{X}$ is the input and $f(x) = y \in \mathcal{Y}$ is the output of the function for that input $x$. In fact, models (like neural networks or linear regression models) are functions.
Examples
- In image classification, an image can be an input
- In machine translation, an input can be a sentence or a word (depending on the model)
- In reinforcement learning, an input can be a state
A feature is an attribute associated with an input or sample. For example, a feature of an image could be a pixel. The feature of a state could be the Euclidean distance to the goal state. An input can be composed of multiple features.
It's possible that people also refer to features as inputs (in fact, if you pass e.g. an image to a model, you're also passing the pixels, the features, which are thus also inputs to the model). There are also other terms used to refer to these. For example, in statistics, people may refer to features as independent variables (or regressors), and maybe a sample refers to a dataset rather than a single observation. So, you should always take into account the context when reading these terms.
For more info, you could read this and this Wikipedia articles.
-
$\begingroup$ I read this article and it says
feature vector as a subset of data that is used to tackle a problemSo is feature a subset of input the ML model finds more useful to deal with? $\endgroup$– user0193Commented May 15, 2022 at 9:35 -
1$\begingroup$ @user0193 I think it's more accurate to think of a feature as an attribute of the input rather than a subset because the input might not be a set. A feature vector is a vector of features. Sometimes, in ML, you convert your original input to a transformed input, which contains more "useful" features. For example, if you're doing image classification, you could pass an image of pixels ("raw input") to the model. But what if you first transformed this image into another input (a vector - the feature vector), where, e.g., element $i$ represents whether there's a human face or not in the image? $\endgroup$– nbroCommented May 15, 2022 at 9:43
-
1$\begingroup$ With the transformed input (the feature vector), the model now might perform better because the new features (presence of a human face or not) are more informative than the original ones (pixels). You could also think of a raw image (with pixels) as a feature vector, but I don't know if this is common. In summary, I think it's better to think of features as attributes of the inputs. For example, my laptop is gray. Being gray is an attribute (aka feature). The laptop is the input and the color is a feature (attribute) of the input. $\endgroup$– nbroCommented May 15, 2022 at 9:45
-
$\begingroup$ So essentially in convolution neural network from the second layer till the second-last layer subsequently extracts features across several layers, i.e. as per your moments "transforming input" and final layer looks into this transformed layer and making final decision/output in the final layer. Right? $\endgroup$– user0193Commented May 15, 2022 at 10:23
-
$\begingroup$ @user0193 Yes, you think of a CNN, but also other neural networks, as transforming the input and as (automatic) feature extractors (based on the data, training objective and learning algorithm). $\endgroup$– nbroCommented May 16, 2022 at 8:25
$\begingroup$
$\endgroup$
An input involves everything as a data. For example in image classification, all images are input to model while features are the kind of specific properties of that input images based on model will decide the results.e.g in dog and cat dataset all images of dogs and cats is a input to model and properties such as ears, nose, other facial differences are the features of that input data.
