I did my Master's thesis on Deep Generative Models and I'm currently looking for a new subject.

Q: What are the "hottest" research topics that are taking a lot of attention of the deep learning community lately?

A few clarifications:

  • I did look through similar questions and none of them answered my question.
  • I come from a pure mathematical background, I only transitioned into deep learning a year ago, and my research on generative models was mostly theoretical. Which means, most of my work revolved around structured probabilistic models, and approximate inference. That said, I have yet to explore real world applications of deep learning.
  • I did my homework before posing the question. My goal was to get ai SE's input on the matter and see what people are working on.
  • 2
    $\begingroup$ NAS Net is really cool. They used a neural network to optimize the structure of a neutral network $\endgroup$
    – keiv.fly
    Mar 21 '18 at 18:17
  • $\begingroup$ I'm leaving open since the OP did review similar questions and didn't find an answer. That said (without having recently reviewed the potential duplicates) it would be good to try to distinguish this question as much as possible from the previous questions. $\endgroup$
    – DukeZhou
    Mar 23 '18 at 20:25

The hot topics of today might be the cold, wet ashes of tomorrow. For instance, the convergence speed of CNN and LSTM approaches, especially in combination, have diverted considerable attention away from basic RNN designs.

Similarly, the cold topics of today might be the burning embers of tomorrow. Of course, some of the cold topics will stay cold. The sweet spot may be to identify those that are getting warmer and are likely to be sustainable building blocks future technology.

Residual Attention Networks

Residual attention networks, like LSTM networks, are an improvement over RNNs using a different approach. Because attention networks are designed to conserve resources, they either converge faster or with less demand on hardware and network to support parallel execution.

Automated Development of Non-Cartesian Models

Research into the automation of modelling is key to many AI applications. Some of the algorithms under development do not simply extract tensors of features (arrays, matrices, cubes, and hyper-cubes), but develop graph models, directed or associative, with or without cycles permitted.

Signal Topologies That Support Equilibria

Many ignore the importance of GANs, not because they can do interesting things with images but because of how they deviate from the simple topology of signal path where convergence on a trained set of parameters is achieved over a one-dimensional array of layers and blocks of layers.

The discriminative and generative components in GAN design are described in some detail in another AI Stack Exchange question on *Understanding the GAN Loss FUnction. Although the generation of images from the GAN approach and its conceptual children demonstrate a new capability in the artificial network space, the breadth of this multi-network significance may not be immediately obvious. It is not a stack in depth of layers, but a stack of two deep networks in a figure-eight topology, conceptually much like a Möbius strip.

This topology creates a balance between two networks, the generative (G) and the discriminative (D). Its designer termed it an adversarial relationship because G and D play opposing roles. However, their action in the system is actually collaborative, creating a balance that is much like a chemical equilibrium or symbiosis in biology, so that a specific objective is achieved. This may reveal the most promising direction in AI today.

Designing signal topologies that support additional forms of collaboration and symbiosis between networks, where each network is a component that learns its roll in conjunction with other component networks, so that the aggregate system learns its function can synthesize forms of artificial intelligence that DNNs cannot.

Rules based systems and deep networks are one dimensional in terms of signal flow. By themselves may never approximate the most notable features of the human brain.

Parallel Processing Using GPUs as DSPs

VLSI implementations of spiking networks is important, and there are now implementations such as https://github.com/Hananel-Hazan/bindsnet that leverage GPU hardware acceleration to investigate them without access to the VLSI chips being developed by large corporations.

Speech Recognition and Synthesis for End-to-End TTS

The recent emergence of excellence in synthesis using systems such as Google's WaveNet have opened the door to more accurate TTS (text to sound) applications, such that it is probably a good time to become an expert in voice recording for use in training example sets but a bad time to start a custom speech production house using live speakers.

Automated Vehicles

Automated vehicles of various types need experts in vehicle physics, automotive manufacture, aeronautics, and consumer products for a wide range of vehicle types with strong economic and safety incentives driving semi-automation and full automation.

  • Mars landers
  • Consumer drones
  • Industrial drones
  • Military drones
  • Passenger aircraft
  • Passenger automobiles
  • Limos
  • Trains
  • Wheel chairs
  • Delivery vehicles
  • Automated food distribution
  • Nuclear plant repair robots
  • Electrical distribution repair robots


It may be difficult to discover in advance what of hot technologies in AI will remain dominant in five years or which of the warming technologies will be blazing hot then, but the above are solid technologies showing significant early promise and for which there are high business, industrial, and consumer demands.


Well, there're certainly a lot of areas where you can contribute in research. Since you're saying you did a Master Thesis in deep Generative models, I assume you're comfortable in Machine and Deep Learning.

Digital Epidemiology is one of the areas where you can certainly apply deep learning. It's still a relatively new field compared to other branches of computational biology. An example would be to see the impact of online digital record on the prediction and further prevalence of diseases.

Such online record can be received from different search engines, social media sites, and sometimes Government agencies. For Example, you can see here an example of search term "Skin Cancer" and the corresponding record shows the interest of this term across the Globe, this data can be used to find new Hypotheses. For example, if the data shows that we have more interest from a specific region of the world/country, that may show that the specific disease is more common in that region/part/country of the world. Similar hypotheses can be built, drawn and tested. And For sure,deep learning can improve the accuracy of traditional models used in validation of such Hypotheses.

Another interesting area of research may be the comparison of Long Short Term Neural Networks against the traditional time series models. I don't believe there exists a mature research on this area. Maybe you can start from this good blog here.

Signal Processing maybe another very interesting, and also very practical area to build and validate theories on top of Deep Learning models. However, Mathematics in Signal Processing can be pretty hard to get. All of these options, however will require you to work in a team with people from the specific domains. That is if you want to produce high quality research.

Other areas may be NLP , especially the case of language translation from Hindi to Urdu or Persian, online digital marketing, behavioral sciences, manufacturing and investment. Specific areas of research maybe improved further if you know experts from these fields.

  • $\begingroup$ Thank you for your answer. Great suggestions! As a matter of fact, I have briefly worked with LSTMs. They can be used to generate images with long time dependencies in PixelRNN. As for signal processing, I do come from a math background so that's actually my cup of tea. $\endgroup$ Mar 22 '18 at 17:34
  • 2
    $\begingroup$ Welcome to AI and thanks for contributing. We've had some previous questions about using current AI methods in the medical field. (Too numerous to list here, but if interested in field some of them, just search for "medical" on this stack.) $\endgroup$
    – DukeZhou
    Mar 22 '18 at 19:48

I am finding value using lstm multivariable input for time series predictions. I also found some value in the cnn for timeseries for business using 1d convolution neural networks. I want to use a cnn for 3d data.

The real question should be "what customer pain can be reduced by deep learning and ai technology?" Since ai is only as good as the customer willingness to pay.

One of the largest and most important areas of ai that needs improved is natural language processing. GPT3 is just the beginning. New attention models for summarization and context graphing need to be created. LSTM are the heart of nlp generative text and summarization technology. However, the nlp can not perform accurate mechanical reasoning.


Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.