6
$\begingroup$

In this article here, the writer claims that a new type of neural net is required to deal with data that is both continuous, and also sparsely sampled.

It was my understanding that this was the entire purpose of techniques that use neural nets, to make assumptions about a system with a non-continuous data set.

So why do we need to switch to a non-layered design to deal with these data sets better?

Improve this question
$\endgroup$
2
  • $\begingroup$ If time (or intervals between measurements) matter then neural nets don't work as well because Neural nets excel at finding patterns. Neural nets tend to find patterns that have the same sequence, regardless of interval between inputs. If a sequence of inputs is exactly the same taken at fixed intervals or sparse intervals, the neural net will tend to find the patterns as the same; which may not be desirable. If a person's health is evaluated at 30 then 50, you'd want different results than evaluations between 30 and 31, assuming the 31 year old's health now matches the 50 year old's health. $\endgroup$
    – Dunk
    Dec 18 '18 at 20:41
  • $\begingroup$ I only posted because you got no answers and I thought I could shed a little bit more light on what the article said but did a poor job of conveying in an easily understandable way. My comment isn't worthy of being posted as an answer to your question but thanks for the offer. $\endgroup$
    – Dunk
    Dec 19 '18 at 0:51
0
$\begingroup$

They struggle because if your network have an inductive bias towards modeling datasets which are described with ODEs well, you will learn faster, and with smaller dataset. I think, this what the authors of the original article meant.

In a similar way, CNNs recognize images much better, because their features are translation invariant whereas fully connected net needs to learn to recognize a cat in each different position from scratch.

Improve this answer
$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.