Return to Answer

To conclude, BoF is a method to represent features of an image, which could then be used to train classifiers or generative models to solve different computer vision tasks (such as CBIR). The first two steps above are concerned with the creation of a visual vocabulary (or codebook), which is then used to create the feature vector of a new test (or query) image.

To conclude, BoF is a method to represent features of an image, which could then be used to train classifiers or generative models to solve different computer vision tasks (such as CBIR). More precisely, if you want to perform CBIR, you could compare your query's feature vector with the feature vector of every image in the database, e.g. using the cosine similarity.

The first two steps above are concerned with the creation of a visual vocabulary (or codebook), which is then used to create the feature vector of a new test (or query) image.

In this last step, given a new (test) image $u \not\in D$ (often called the query image, in this context), then we will represent $u$ as a $k$-dimensional vector (where $k$, if you remember, is the number of codewords) that will represent its feature vector. To do that, we need to follow the following steps.

In this last step, given a new (test) image $u \not\in D$ (often called the query image in this context of CBIR), then we will represent $u$ as a $k$-dimensional vector (where $k$, if you remember, is the number of codewords) that will represent its feature vector. To do that, we need to follow the following steps.

So, after this step, we will have $k$ clusters, each of them associated with a centroid $C = \{ c_1, \dots, c_k\}$, where $C$ is the set of centroids (and $c_i \in \mathbb{R}^{128}$ in the case that SIFT descriptors have been used). These centroids represent the main features that are present in the whole training dataset $D$. In this context, they are often known as the codewords (which derives from the vector quantization literature) or visual words (hence the name bag-of-visual-words). The set of codewords is often called codebook or, equivalently, the visual vocabulary.

At the end of this process, we will have a vector $I \in \mathbb{R}^k$ that represents the frequency of the codewords in the query image $u$ (akin to the term frequency in the context of the bag-of-words model). Equivalently, $I$ can also be viewed as a histogram of features of the query image $u$, i.e. the feature vector of $u$. Here's an illustrative example of such a histogram.

From this diagram, we can see that there are $11$ codewords (of course, this is an unrealistic scenario!), and, on the y-axis, we have the frequency of each of the codewords in a given image.

So, after this step, we will have $k$ clusters, each of them associated with a centroid $C = \{ c_1, \dots, c_k\}$, where $C$ is the set of centroids (and $c_i \in \mathbb{R}^{128}$ in the case that SIFT descriptors have been used). These centroids represent the main features that are present in the whole training dataset $D$. In this context, they are often known as the codewords (which derives from the vector quantization literature) or visual words (hence the name bag-of-visual-words). The set of codewords $C$ is often called codebook or, equivalently, the visual vocabulary.

At the end of this process, we will have a vector $I \in \mathbb{R}^k$ that represents the frequency of the codewords in the query image $u$ (akin to the term frequency in the context of the bag-of-words model), i.e. $u$'s feature vector. Equivalently, $I$ can also be viewed as a histogram of features of the query image $u$. Here's an illustrative example of such a histogram.

From this diagram, we can see that there are $11$ codewords (of course, this is an unrealistic scenario!). On the y-axis, we have the frequency of each of the codewords in a given image. We can see that the $7$th codeword is the most frequent in this particular query image.