The hot topics of today might be the cold, wet ashes of tomorrow. For instance, the convergence speed of CNN and LSTM approaches, especially in combination, have diverted considerable attention away from basic RNN designs.
Similarly, the cold topics of today might be the burning embers of tomorrow. Of course, some of the cold topics will stay cold. The sweet spot may be to identify those that are getting warmer and are likely to be sustainable building blocks future technology.
Residual Attention Networks
Residual attention networks, like LSTM networks, are an improvement over RNNs using a different approach. Because attention networks are designed to conserve resources, they either converge faster or with less demand on hardware and network to support parallel execution.
Automated Development of Non-Cartesian Models
Research into the automation of modelling is key to many AI applications. Some of the algorithms under development do not simply extract tensors of features (arrays, matrices, cubes, and hyper-cubes), but develop graph models, directed or associative, with or without cycles permitted.
- Hierarchical Self-organizing Maps System for Action Classification
Z Gharaee, P Gärdenfors, M Johnsson, ICAART, 2017
- Runtime Modelling for User-Centric Smart Cyber-Physical-Human Applications, Lorena Castañeda Bueno, 2017
- Probing the Topological Properties of Complex Networks Modeling Short Written Texts, Diego R. Amancio, 2015
- Schema-summarization in Linked-Data-based feature selection for recommender systems, Azzurra Ragone et. al., 2017
Signal Topologies That Support Equilibria
Many ignore the importance of GANs, not because they can do interesting things with images but because of how they deviate from the simple topology of signal path where convergence on a trained set of parameters is achieved over a one-dimensional array of layers and blocks of layers.
The discriminative and generative components in GAN design are described in some detail in another AI Stack Exchange question on *Understanding the GAN Loss FUnction. Although the generation of images from the GAN approach and its conceptual children demonstrate a new capability in the artificial network space, the breadth of this multi-network significance may not be immediately obvious. It is not a stack in depth of layers, but a stack of two deep networks in a figure-eight topology, conceptually much like a Möbius strip.
This topology creates a balance between two networks, the generative (G) and the discriminative (D). Its designer termed it an adversarial relationship because G and D play opposing roles. However, their action in the system is actually collaborative, creating a balance that is much like a chemical equilibrium or symbiosis in biology, so that a specific objective is achieved. This may reveal the most promising direction in AI today.
Designing signal topologies that support additional forms of collaboration and symbiosis between networks, where each network is a component that learns its roll in conjunction with other component networks, so that the aggregate system learns its function can synthesize forms of artificial intelligence that DNNs cannot.
Rules based systems and deep networks are one dimensional in terms of signal flow. By themselves may never approximate the most notable features of the human brain.
Parallel Processing Using GPUs as DSPs
VLSI implementations of spiking networks is important, and there are now implementations such as https://github.com/Hananel-Hazan/bindsnet that leverage GPU hardware acceleration to investigate them without access to the VLSI chips being developed by large corporations.
Speech Recognition and Synthesis for End-to-End TTS
The recent emergence of excellence in synthesis using systems such as Google's WaveNet have opened the door to more accurate TTS (text to sound) applications, such that it is probably a good time to become an expert in voice recording for use in training example sets but a bad time to start a custom speech production house using live speakers.
Automated vehicles of various types need experts in vehicle physics, automotive manufacture, aeronautics, and consumer products for a wide range of vehicle types with strong economic and safety incentives driving semi-automation and full automation.
- Mars landers
- Consumer drones
- Industrial drones
- Military drones
- Passenger aircraft
- Passenger automobiles
- Wheel chairs
- Delivery vehicles
- Automated food distribution
- Nuclear plant repair robots
- Electrical distribution repair robots
It may be difficult to discover in advance what of hot technologies in AI will remain dominant in five years or which of the warming technologies will be blazing hot then, but the above are solid technologies showing significant early promise and for which there are high business, industrial, and consumer demands.