35
votes
How to select number of hidden layers and number of memory cells in an LSTM?
Your question is quite broad, but here are some tips.
Specifically for LSTMs, see this Reddit discussion Does the number of layers in an LSTM network affect its ability to remember long patterns?
...
- 1,197
31
votes
Accepted
What are some well-known problems where neural networks don't do very well?
Here's a snippet from an article by Gary Marcus
In particular, they showed that standard deep learning nets often fall
apart when confronted with common stimuli rotated in three dimensional
...
- 436
30
votes
What are the current theories on the development of a conscious AI?
To answer this question, first we need to know why developing conscious AI is hard.
The main reason is that there is no mathematically or otherwise rigorous definition of consciousness.
Sure you ...
- 3,787
30
votes
Accepted
Which explainable artificial intelligence techniques are there?
Explainable AI and model interpretability are hyper-active and hyper-hot areas of current research (think of holy grail, or something), which have been brought forward lately not least due to the (...
- 1,024
30
votes
Accepted
Where can I find the proof of the universal approximation theorem?
There are multiple papers on the topic because there have been multiple attempts to prove that neural networks are universal (i.e. they can approximate any continuous function) from slightly different ...
- 39.6k
28
votes
Accepted
Is there any research on the development of attacks against artificial intelligence systems?
Yes, there is some research on this topic, which can be called adversarial machine learning, which is more an experimental field.
An adversarial example is an input similar to the ones used to train ...
- 39.6k
21
votes
What are some well-known problems where neural networks don't do very well?
In theory, most neural networks can approximate any continuous function on compact subsets of $\mathbb{R}^n$, provided that the activation functions satisfy certain mild conditions. This is known as ...
- 39.6k
20
votes
How do neural networks play chess?
Minimax and related algorithms are used to play chess. That is how chess programs have worked for many years (with some additions such as standard opening playbooks). They do not need to process the ...
- 30.3k
18
votes
Are there other approaches to deal with variable action spaces?
Does anyone know any paper regarding this subject?
I'm not familiar with any off the top of my head. I do know that the vast majority of Reinforcement Learning literature focuses on settings with a ...
- 10.1k
18
votes
Accepted
When should I use 3D convolutions?
3D convolutions are used when you want to extract features in 3 dimensions or establish a relationship between 3 dimensions.
Essentially, it's the same as 2D convolutions, but the kernel movement is ...
- 1,409
17
votes
How can I encode angle data to train neural networks?
The main problem with simply using the values $\alpha \in [0, 2\pi]$ is that semantically $0 = 2\pi$, but numerically $0$ and $2\pi$ are maximally far apart. A common way to encode this is by a vector ...
- 1,536
16
votes
Accepted
What are the mathematical prerequisites for an AI researcher?
Good Mathematics Foundation
Begin by ensuring full competency with intermediate algebra and some other foundations of calculus and discrete math, including the terminology and basic concepts within ...
16
votes
What are examples of promising AI/ML techniques that are computationally intractable?
AIXI is a Bayesian, non-Markov, reinforcement learning and artificial general intelligence agent that is incomputable, given the involved incomputable Kolmogorov complexity. However, there are ...
- 39.6k
16
votes
What are some well-known problems where neural networks don't do very well?
In our deep learning lecture, we discussed the following example (from Unmasking Clever Hans predictors and assessing what machines really learn (2019) by Lapuschkin et al.).
Here the neural network ...
- 261
14
votes
How could we build a neural network that is invariant to permutations of the inputs?
Here is a few that might be what you are looking for:
Deep Sets, https://papers.nips.cc/paper/6931-deep-sets.pdf
BRUNO: A Deep Recurrent Model for Exchangeable Data, https://arxiv.org/pdf/1802.07535....
- 241
14
votes
What are examples of promising AI/ML techniques that are computationally intractable?
Exact Bayesian inference is (often) intractable (i.e. there is no closed-form solution, or numerical approximations are also computationally expensive) because it involves the computation of an ...
- 39.6k
13
votes
Accepted
Are the dialogs at Sophia's (the robot) appearings scripted?
Sophia uses ChatScript.
You can read about what ChatScript can do here.
ChatScript keeps track of conversations with each user; can record
where it is in a conversational flow and what facts it ...
- 146
13
votes
Accepted
How could we build a neural network that is invariant to permutations of the inputs?
Traditionally, due to the way the network is structured, each input has a set of weights, that are connected to more inputs. If the inputs switch, the output will too.
Approach 1
However, you can ...
- 906
12
votes
Where can I find the original paper that introduced RNNs?
The two tech reports below both call RNNs explicitly "recurrent net(work)s".
Rumelhart, David E; Hinton, Geoffrey E, and Williams, Ronald J (Sept. 1985). Learning internal representations ...
- 221
12
votes
Is there any research on the development of attacks against artificial intelligence systems?
Sometimes if the rules used by an AI to identify characters are discovered, and if the rules used by a human being to identify the same characters are different, it is possible to design characters ...
- 363
12
votes
How do neural networks play chess?
This is a good question. your understanding in general is correct. Indeed, data can be used to construct a proper evaluation of a move/board position and recommended moves based on its history (at ...
- 823
11
votes
Accepted
How can I automate the choice of the architecture of a neural network for an arbitrary problem?
I think in this case, you'll probably want to use a genetic algorithm to generate a topology rather than working on your own. I personally like NEAT (NeuroEvolution of Augmenting Topologies).
The ...
- 226
11
votes
Accepted
Can an AI make a constructed (natural) language?
could an AI make/construct its own natural language, with words, conjugations, and grammar rules?
Sure. This might be helpful: Simulated Evolution of Language: a Review of the Field
Basically, a ...
- 1,945
11
votes
Is there any research on the development of attacks against artificial intelligence systems?
Yes there are, for instance one pixel attacks described in
Su, J.; Vargas, D.V.; Kouichi, S. One pixel attack for fooling deep
neural networks. arXiv:1710.08864
One pixels attacks are attacks in ...
- 211
11
votes
What are examples of promising AI/ML techniques that are computationally intractable?
This question gets at a really interesting fact about AI research in general: AI is hard.
In fact, almost every AI problem is computationally hard (typically NP-Hard, or #P-Hard). This means that ...
- 9,207
10
votes
What are the mathematical prerequisites for an AI researcher?
I work as a professor, and recently designed the mathematics requirements for a new AI major, in consultation with many of my colleagues at other institutions.
The other answers, particularly this one ...
- 9,207
10
votes
What are examples of approaches to create an AI for a fighting robot in an MMO game?
I would set up a list of goals for your bot. These could be 'maintain a minimum level of health', 'knock out human player', 'block way to location X', etc. This obviously depends on the domain of your ...
- 5,362
10
votes
Accepted
What are some online courses on artificial general intelligence?
As far as I know, no AGI system has yet been created, so that's why there aren't yet many courses on AGI. However, there are a few courses that attempt to address AGI as the main topic but from ...
- 39.6k
9
votes
Can AI write good jokes yet?
I don't think the AI has gotten to that point yet. Here are some of the interesting papers on the subject:
A paper was recently written that attempted to generate jokes using unsupervised learning. ...
- 840
9
votes
Accepted
What are some resources on computational learning theory?
Although I have only partially read or not read at all some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are ...
- 39.6k
Only top scored, non community-wiki answers of a minimum length are eligible