There has been previous research with promising results cited at length in the following recent article, and although they have limited training data, here is some impressive research for an undergraduate thesis at the University of Arkansas which extends that research using an artificial neural network on enhancing a classifying algorithm's capacity to facilitate unspoken, or imagined, speech recognition by collecting and analyzing a large dataset of simultaneous EEG signal and video data streams.
Imagined speech (unspoken speech, silent speech, or covert speech) is
the process by which one thinks about a word, or “hears” the word in
one’s head, in the absence of any vocalization or physical movement
indicating the word. Though there exists evidence that it is possible
for imagined speech information to be captured and interpreted. To
facilitate imagined speech, a Brain-to-Computer Interface (BCI) must
be implemented to provide silent communication abilities directly
between the two entities. One of the most popular methods for
interfacing directly between a human brain and a computer is through
electroencephalographic signals.
Researchers have created models capable of achieving 70 - 90%
predictive accuracy in recognizing patterns in EEG data;
however, the accuracy of current methods for unspoken speech
recognition is not yet sufficient to enable fluid communication
between humans and machines.
High Level Experiment Design
the subjects were asked to imagine a specific word or feeling (label).
The subjects responded to a set of uniform verbal cues describing the
set of labels as well as the desired individual label to imagine. The
data was then processed in order to minimize the effects of irrelevant
signal activity, or noise. Additionally the data was processed to
minimize its volume while still maintaining the core “information” in
the data. The condensed dataset was created by dropping irrelevant
information from the EEG device and applying principal component
analysis (PCA) to the video stream data. Once the data was processed
and assembled into the correct format, cross-validation using a random
forest algorithm was performed on the control group of EEG signals
alone and on the hypothesis group consisting of both EEG and video
data. The predictive accuracy measurements obtained from the
cross-validation experiments were used as metrics to evaluate the
success of the hypothesis.
The results show a notable improvement classifying thoughts when in conjunction with the video streams.