The Problem
We can see from the question that existing information on detection and
classification in the small automotive vehicle domain has been
located (in the form of two independent sets of vectors usable for
machine training), and there is no already existing mapping or
other correspondence between the elements of one set and the
elements of the other. They were obtained independently, remain
independent, and are linked only by the conventions of the domain
(today's aesthetically acceptable and thermodynamically workable
forms of small vehicles).
The goal stated in the question is to create a computer vision system
that both detects cars and classifies them leveraging the
information contained in the two distinct sets.
In the vision systems of mammals, there are also two distinct equivalences
of sets; one arising from a genetic algorithm, the DNA that is
expressed during the formation of the neural net geometry and
bio-electro-chemistry of the visual system in early development;
and the cognitive and coordinative pathways in the cerebrum and
cerebellum.
If a robot, wheelchair, or other vehicle is to avoid traffic, we must
produce a system that in some way matches or exceeds the
collision avoidance performance of mammals.
In crime prevention, toll collection, sales lot inventory,
county traffic analysis, and other like applications,
performance will again be expected to match or exceed the
performance of biological systems.
If a person can record the make, model, year, color, and
license plate strings, so should the machine we employ in
these capacities.
Consequently, this question is pertinent beyond academic curiosity, as
it is applicable in current research and development of products.
That this question author notices the lack of a unified data
set that can be used to train it to detect and characterize
in a single network objects of interest is apropos and
key to the challenge of finding a solution.
Approach
The simplest approach would be to compose the system of two functions.
- $\quad\mathcal{D}: \mathbb{I}^4 \to {(\mathbb{I}^2, \mathbb{I}^2)}_1, \; {(\mathbb{I}^2, \mathbb{I}^2)}_2, \; ... $
- $\quad\mathcal{C}: {(\mathbb{I}^2, \mathbb{I}^2)}_i \to {(\mathbb{I})}_i$
The four dimensions of input for $\mathcal{D}$, the detector, are horizontal position, vertical position,
rgb index, and brightness to decribe the pixelized image; and the output are bounding boxes as two "corner" coordinates corresponding to each identified vehicle, the second coordinate being either relative to the first or to a specific corner of the entire frame.
The categorizer, $\mathcal{C}$, receives as input bounding boxes and produces as output the index
or code that maps to the categories corresponding to the labels of the training set available for
categorization.
The system can then be described as follows.
$\quad\quad\mathcal{S}: \mathcal{C} \circ \mathcal{D}$
If the system is not color, subtract one from the above dimensionality of the input. If the system processes video, add one to the dimensionality of the input and consider using LSTM or GRU cell types.
The above substitution represented by "$\circ$" appears to be what is meant by, "I use the images from the detection dataset as input
and get classification predictions on top of detected bounding boxes."
The interrogative, "How do I verify whether the classification model trained on the
classification dataset is working on images from detection dataset?
(In terms of classification accuracy),"
appears to refer to the fact that labels do not exist for the second set that correspond to
input elements of the first set, so an accuracy metric cannot be directly obtained.
Since there is no obvious automatic way of generating labels for the vehicles in the pre-detected images
containing potentially multiple vehicles,
there is no way to check actual results against expected results.
Composing multiple vehicle images from the categorization set to use as test input to
the entire system $\mathcal{S}$ will only be useful in evaluating an aspect
of the performance of $\mathcal{D}$, not $\mathcal{C}$.
Solution
The only way to evaluate the accuracy and reliability of $\mathcal{C}$ is with portions
of the set used to train it that were excluded from the training and trust
that the vehicles depicted in those images were sufficiently representative
of the concept "car" to provide consistency of accuracy and reliability across
the range of those detected by $\mathcal{D}$ in the application of $\mathcal{S}$.
This means that the leveraging of the information, even if optimized to the degree
possible by any arbitrary algorithm or parallelism in the set of all possible
algorithms or parallelisms, is limited by the categorization training set.
The number of set elements and the comprehensiveness and distribution of categories
within that set must be sufficient to achieve an approximate equality between
these two accuracy metrics.
- Categorizing a test sample from the labeled set for $\mathcal{C}$ excluded from the training
- Categorizing the vehicles isolated by $\mathcal{D}$ from its training input
With Additional Resources
Of course this discussion is in a particular environment, that of the system
defined as the two artificial networks, one involving convolution based
recognition and the other involving feature extraction, and the two training
sets.
What is needed is a wider environment where known vehicles are in view so that
performance data of $\mathcal{S}$ is evaluated and a tap on the transfer
of information between $\mathcal{D}$ and $\mathcal{C}$ can be used
to differentiate between mistakes made on either side of the tap point.
Unsupervised Approach
Another course of action could be to not use the training set for categorization
on the training of $\mathcal{C}$ at all, but rather use feature extraction and
auto-correlation in an "unsupervised" approach, and then evaluate the results of
on the basis of the final convergence metrics at the point when stability in
categorization is detected. In this case, the images in the bounding boxes
output by $\mathcal{D}$ would be used as training data.
The auto-trained network realizing $\mathcal{C}$ can then be further evaluated
using the entire categorization training set.
Further Research
Hybrids of these two approaches are possible. Also, the independent training only in the rarest of cases leads to optimal performance. Understanding feedback as originally treated with rigor by MacColl in chapter 8 of his Fundamental Theory of Servomechanisms, later applied to the problem of linearity and stability of analog circuitry, and then to training, first in the case of GANs, may lead to effective methods to bi-train the two networks.
That evolved biological networks are trained in situ is an indicator that the most optimal performance can be gained by finding training architectures and information flow strategies that create optimality in both components simultaneously. No biological niche has ever been filled by a neural component that is first optimized and then inserted or copied in some way to a larger brain system. That is no proof that such component-ware can be optimal, but there is also no proof that the DNA driven systems that have emerged are not nearly optimized for the majority of terrestrial conditions.
