Questions tagged [theory]
Use for questions on AI theory and other theoretical subjects related to AI.
92
questions
-1
votes
0answers
31 views
Does the direction of Backpropogation matter? [closed]
If I have a simple MLP model, would the order of Backpropagation on the network matter? I think that error is the only real factor, but can someone clarify?
Also, please redirect the question to ...
0
votes
1answer
39 views
What is “Pattern Theory”?
I came across Grenander's work "Probabilities on Algebraic Structures" recently and found that much of Grenander's work focused on what he called "Pattern Theory." He's written ...
2
votes
1answer
78 views
Is GPT-3 an early example of Strong AI in a Narrow setting?
In GPT-2, the large achievement was being able to generate coherent text over a long-form while maintaining context. This was very impressive but for GPT-2 to do new language tasks, it had to be ...
0
votes
0answers
18 views
The convolutional network architectures with enhanced invariance
It is well known, that CNN have advantage with respect to the Dense neural networks in the image classification and other pattern recognition tasks, because they have a translationall invariance built ...
2
votes
1answer
45 views
The are some fundamental learning theories for developing an AI that imitates human behavior
Most if not all AI systems are not to imitate human, but to finally out-perform human. Examples include using AI to play a game, classification problems, auto-driving, and goal-oriented chatbots. ...
0
votes
0answers
35 views
Is it a good idea to train a neural network to classify images without base-hypothesis?
I'm a relative beginner in deep-learning (understand by that, I'm doing my first kaggle competition right now, and I have loads to learn still) and I was just wondering something.
Let's say you have ...
1
vote
1answer
70 views
How artificial intelligence will change the future?
AI is the emerging field and biggest business opportunity of the next decade. It's already automating manual and repetitive tasks. And in some areas, it can learn faster than humans, if not yet as ...
3
votes
1answer
833 views
What is the relationship between gradient accumulation and batch size?
I am currently training some models using gradient accumulation since the model batches do not fit in GPU memory. Since I am using gradient accumulation, I had to tweak the training configuration a ...
2
votes
0answers
40 views
Is there any application of topology to deep learning?
Is there any application of topology (as in math discipline) to deep learning? If so, what are some examples?
0
votes
2answers
41 views
Is it theoretically possible (or impossible) that principal component analysis worsens the performance of the model?
In case I had a prediction model and decided to add a PCA step prior to the model, is it theoretically possible/impossible that the number of output dimensions that is better for all tests may perform ...
2
votes
0answers
34 views
Is there a theory that captures the following ideas?
A big class of problems that are relevant in today's society are full of uncertainty and are also sometimes computationally intractable. Along our lives we come to realize that we are solving the same ...
1
vote
0answers
19 views
Are there any good resources about techniques used for entity extraction
Given some natural language sentences like "I would like to talk to Mr. Smith" I would like to extract entities like the person "Smith".
I know that frameworks exist which are capable of doing so (f. ...
1
vote
2answers
131 views
What are some examples of classical AI methods?
All of the attention in the AI field over the past few years has been toward statistical AI, now that Machine Learning has been validated on hard problems.
However, I want to learn more about ...
1
vote
1answer
43 views
What is the theoretical basis for the use of Cross Validation set?
So let's follow this line of reasoning. We use a MLE estimator (implementation doesn't matter) and we have a train set. We assume that we have sampled training set from a Gaussian distribution $\...
2
votes
0answers
20 views
Suitable kernels for Gaussian processes
Consider a stochastic process $\{X_t \colon t \in T\}$ indexed by a set $T$. We assume for simplicty that $T \in \mathbb{R}^n$. We assume that for any choice of indexes $t_1, \dots, t_n$, the random ...
0
votes
2answers
123 views
Understanding CNN in a few sentences
I don't know if this is the right place to ask this question. If it is not, please tell me and I remove it.
I've just started to learn CNN and I'm trying to understand what they do and how they do it....
4
votes
1answer
64 views
Is there a way to ensure that my model is able to recognize an unseen example?
My question is more theoretical than practical. Let's say that I am training my cat classifier with a dataset that I feel is pretty representative of cat images in general. But then a new breed of cat ...
5
votes
5answers
291 views
What event would confirm that we have reached AGI or a highly intelligent system?
I was listening to a podcast on the topic of AGI and a guest made an argument that if strong music generation were to happen, it would be a sign of true intelligence in machines because of how much ...
2
votes
0answers
26 views
Did people analyze dynamics of very simple LSTMs?
I wonder if researchers tried to understand how LSTMs work by analyzing the dynamics of simple LSTM (e.g. with 2 units)? For example how the hidden state evolves depending on the properties of weight ...
2
votes
1answer
122 views
What make a CNN suitable for image classification or for semantic segmentation?
I've just started with CNN and there is something that I haven't understood yet:
How do you "ask" a network: "classify me these images" or "do semantic segmentation"?
I think it must be something on ...
2
votes
0answers
265 views
What does “episodic training” mean?
I'm reading the book Hands-On Meta Learning with Python, and in Prototypical networks said:
So, we use episodic training—for each episode, we randomly sample a
few data points from each class in ...
2
votes
1answer
49 views
How many ways are there to perform image segmentation?
I'm new in Artificial Intelligence and I want to do image segmentation.
Searching I have found these ways
Digital image processing (I have read it in this book: Digital Image Processing, 4th edition)...
2
votes
0answers
77 views
Is this a problem well suited for machine learning?
The light-field of a certain scene is the set of all light rays that travel through the volume of that scene at a specific point of time. A light-field camera for example captures and stores a subset ...
4
votes
2answers
176 views
Are current AI models sufficient to achieve Artificial General Intelligence?
I read an interesting essay about how far we are from AGI. There were quite a few solid points that made me re-visit the foundation of AI today. A few interesting concepts arose:
imagine that you ...
1
vote
0answers
31 views
How would you differentiate between different on-policy reinforcement learning algorithms?
How would you differentiate between different on-policy reinforcement learning algorithms?
0
votes
1answer
98 views
When are compiled vs. interpreted languages more optimal in AI?
When are interpreted languages more optimal? When are compiled languages more optimal? What are the qualities and functions that render the so in relation to various AI methods?
2
votes
1answer
62 views
Is there anything theoretically revolutionary about Deep Neural Network?
In recent years we have seen quite a lot of impressive display of Deep Neural Network (DNN), as demonstrated most famously by AlphaGo and its cousin programs.
But if I understand correctly, deep ...
5
votes
2answers
3k views
What is convergence in machine learning?
I came across this answer on Quora, but it was pretty sparse. I'm looking for specific meanings in the context of machine learning, but also mathematical and economic notions of the term in general.
4
votes
1answer
692 views
What does 'democratizing AI' exactly mean?
In my AI literature research, I often notice authors use term 'democratizing AI', especially in the AutoML area. I think I have an idea of what this means, but I would like to ask you for some more ...
12
votes
3answers
154 views
Is there a way to understand neural networks without using the concept of brain?
Is there a way to understand, for instance, a multi-layered perceptron without hand-waving about them being similar to brains etc?
For example: it is obvious that what a perceptron does is ...
1
vote
1answer
39 views
Why is image classification tasks are dominated by minimizing cost function instead of maximizing ones?
I was watching a video of policy gradient by Andrej Karpathy at 10:00 there shows an equation for supervised learning for image classification.
$max\sum _{i}log \:p(y_i|x_i)$
I have worked with ...
5
votes
2answers
59 views
Is the gradient at a layer independent of the activations of the previous layers?
Is the gradient at a layer (of a feed-forward neural network) independent of the activations of the previous layers?
I read this in a paper titled Mean Field Residual Networks: On the Edge of Chaos (...
4
votes
0answers
64 views
Is it possible to control asymptotic behaviour of neural network models?
Is it possible to specify what the asymptotic behaviour of a Neural Networks (NN) model should be? I am thinking on NN which try to learn a mapping $\vec y=f(\vec x)$ with $\vec x$ a vector of ...
6
votes
9answers
947 views
Can Neural Networks self-optimize?
Suppose that you show a neural network its own code, and allow it to edit itself?
Can a neural network modify its own weights and architecture (the number of layers, the number of neurons per layer, ...
48
votes
4answers
13k views
Are neural networks prone to catastrophic forgetting?
Imagine you show a neural network a picture of a lion 100 times and label with "dangerous", so it learns that lions are dangerous.
Now imagine that previously you have shown it millions of images of ...
2
votes
0answers
43 views
Why do neural networks have bias units?
Why do neural networks have bias units? Why is it sometimes okay to opt them out?
1
vote
1answer
2k views
What is an agent in Artificial Intelligence?
While studying artificial intelligence, I have often encountered the term "agent" (often autonomous, intelligent). For instance, in fields such as Reinforcement Learning, Multi-Agent Systems, Game ...
2
votes
1answer
65 views
Is explainable AI more feasible through symbolic AI or soft computing?
Is explainable AI more feasible through symbolic AI or soft computing?
How much each paradigm, symbolic AI and soft computing (or hydrid approaches), adresses explanation and argumentation, where ...
2
votes
2answers
155 views
Would an artificial general intelligence have to be Turing complete?
Artificial general intelligence is the intelligence of a machine that could successfully perform any intellectual task that a human being can.
Would an artificial general intelligence have to be ...
2
votes
1answer
63 views
What are the known ways to test AI black boxes?
Is there any conmprehensive paper or site that describes an overview of the known black box testing techniques?
1
vote
0answers
17 views
Theoretical grounding for ease of training with a prior
If we have a neural network that learns the generative model for P(A, B, C)
And now, we want to learn the generative model for P(A, B, C, D)
Is there any theory that says learning P(A,B,C) and then ...
0
votes
0answers
27 views
Superhuman AI to disguise as human players
TL;DR below.
You entered a online mini video game.
It's a fast-paced hypercasual game. But it's a ranked one - top 10 get prizes. The leaderboard resets and prizes are handed out every couple of ...
2
votes
1answer
95 views
Neural nets need a short term memory of some sort
I notice with the recent revelation of severe limitations in some AI domains such as self driving cars that NNets behave with the same sort of errors as in simpler models. Ie: They may be ~100% ...
1
vote
1answer
38 views
Is Intelligence a naturally occurring function of Information Technology?
Here intelligence is defined as any analytic or decision making process, regardless of strength (utility), and, potentially, any process of computation that produces output, regardless of the medium.
...
1
vote
2answers
72 views
Are winning and imitating the primary indicators of intelligence?
Conjecture 1: The smartest chess playing system is the one that wins the tournament.
Conjecture 2: The computer system that imitates human dialog is as smart as the human whose dialog was imitated.
...
3
votes
1answer
90 views
How would a quantum computer potentially facilitate artificial consciousness, assuming it is possible?
How would a quantum computer potentially facilitate artificial consciousness, assuming it is possible?
3
votes
0answers
28 views
Batch PTA stopping condition
I am reviewing my Neural Network lectures and I have a doubt: My book's (Haykin) batch PTA describes a cost function which is defined over the set of the misclassified inputs.
I have always been ...
3
votes
0answers
72 views
An idea I had to create the first Humanoid using Deep Learning
I've come up with an idea on how we could use a combination of Deep Learning and body sensors to create a walking talking living humanoid. Here goes:
First, we will recruit 1 billion people and have ...
7
votes
2answers
127 views
What does learning mean?
Can someone explain what is the process of learning? What does it mean to learn something?
0
votes
1answer
87 views
How can I systematically learn about the theory of neural networks?
I have seen a few articles about neural nets. Mostly they went along these lines: we tried these architectures, these meta parameters, we trained it for $x$ hours on $y$ CPUs, and it gave us these ...
