28
Yes, there is some research on this topic, which can be called adversarial machine learning, which is more an experimental field.
An adversarial example is an input similar to the ones used to train the model, but that leads the model to produce an unexpected outcome. For example, consider an artificial neural network (ANN) trained to distinguish between ...
28
Here's a snippet from an article by Gary Marcus
In particular, they showed that standard deep learning nets often fall
apart when confronted with common stimuli rotated in three dimensional
space into unusual positions, like the top right corner of this
figure, in which a schoolbus is mistaken for a snowplow:
.
.
.
Mistaking an ...
23
Explainable AI and model interpretability are hyper-active and hyper-hot areas of current research (think of holy grail, or something), which have been brought forward lately not least due to the (often tremendous) success of deep learning models in various tasks, plus the necessity of algorithmic fairness & accountability.
Here are some state of the ...
19
There are multiple papers on the topic because there have been multiple attempts to prove that neural networks are universal (i.e. they can approximate any continuous function) from slightly different perspectives and using slightly different assumptions (e.g. assuming that certain activation functions are used). Note that these proofs tell you that neural ...
19
In theory, most neural networks can approximate any continuous function on compact subsets of $\mathbb{R}^n$, provided that the activation functions satisfy certain mild conditions. This is known as the universal approximation theorem (UAT), but that should not be called universal, given that there are a lot more discontinuous functions than continuous ones,...
16
AIXI is a Bayesian, non-Markov, reinforcement learning and artificial general intelligence agent that is incomputable, given the involved incomputable Kolmogorov complexity. However, there are approximations of AIXI, such as AIXItl, described in Universal Artificial Intelligence: Universal Artificial Intelligence: Sequential Decisions based on Algorithmic ...
15
Good Mathematics Foundation
Begin by ensuring full competency with intermediate algebra and some other foundations of calculus and discrete math, including the terminology and basic concepts within these topics.
Infinite series
Logical proofs
Linear algebra and matrices
Analytic geometry, especially the distinction between local and global extremes (minima ...
15
In our deep learning lecture, we discussed the following example (from Unmasking Clever Hans predictors and assessing what machines really learn (2019) by Lapuschkin et al.).
Here the neural network learned a wrong way to identify a picture, i.E by identifying the wrong "relevant components". In the sensitivity maps next to the pictures, we can see that the ...
14
Does anyone know any paper regarding this subject?
I'm not familiar with any off the top of my head... I do know that the vast majority of Reinforcement Learning literature focuses on settings with a fixed action space (like robotics where your actions determine how you attempt to move / rotate a particular part of the robot, or simple games where you ...
13
Sophia uses ChatScript.
You can read about what ChatScript can do here.
ChatScript keeps track of conversations with each user; can record
where it is in a conversational flow and what facts it has learned
about a user (you have to tell it what facts to try to learn). You can
optionally keep logs of the conversations (either on a ChatScript
...
13
Exact Bayesian inference is (often) intractable (i.e. there is no closed-form solution, or numerical approximations are also computationally expensive) because it involves the computation of an integral over a range of real (or even floating-point) numbers, which can be intractable.
More precisely, for example, if you want to find the parameters $\mathbf{\...
12
Here is a few that might be what you are looking for:
Deep Sets, https://papers.nips.cc/paper/6931-deep-sets.pdf
BRUNO: A Deep Recurrent Model for Exchangeable Data, https://arxiv.org/pdf/1802.07535.pdf
Deep Learning with Sets and Point Clouds, https://openreview.net/pdf?id=HJF3iD9xe
Permutation-equivariant neural networks applied to dynamics prediction, ...
12
Sometimes if the rules used by an AI to identify characters are discovered, and if the rules used by a human being to identify the same characters are different, it is possible to design characters that are recognized by a human being but not recognized by an AI. However, if the human being and AI both use the same rules, they will recognize the same ...
11
The two tech reports below both call RNNs explicitly "recurrent net(work)s".
Rumelhart, David E; Hinton, Geoffrey E, and Williams, Ronald J (Sept. 1985). Learning internal representations by error propagation. Tech. rep. ICS 8504. San Diego, California: Institute for Cognitive Science, University of California.
Jordan, Michael I. (May 1986). ...
11
Yes there are, for instance one pixel attacks described in
Su, J.; Vargas, D.V.; Kouichi, S. One pixel attack for fooling deep
neural networks. arXiv:1710.08864
One pixels attacks are attacks in which changing one pixel in input image can strongly affect the results.
10
This question gets at a really interesting fact about AI research in general: AI is hard.
In fact, almost every AI problem is computationally hard (typically NP-Hard, or #P-Hard). This means that most new areas of AI research starts out by characterizing some problem that is intractable, and proposing an algorithm that technically works, but is too slow to ...
9
Traditionally, due to the way the network is structured, each input has a set of weights, that are connected to more inputs. If the inputs switch, the output will too.
Approach 1
However, you can build a network that approaches this behaviour. In your training set, use batch learning and for each training sample, give all possible permutations to the network ...
9
I work as a professor, and recently designed the mathematics requirements for a new AI major, in consultation with many of my colleagues at other institutions.
The other answers, particularly this one do a good job of cataloging all the specific topics that might be useful somewhere in AI, but not all of them are equally useful for understanding core topics. ...
9
As far as I know, no AGI system has yet been created, so that's why there aren't yet many courses on AGI. However, there are a few courses that attempt to address AGI as the main topic but from different perspectives. Below, I will mention the ones that I found and partially followed, and give some info about them.
MIT 6.S099: Artificial General Intelligence
...
8
Douglas Hofstadter's CopyCat architecture for solving letter-string analogy problems was deliberately engineered to maintain a semantically-informed notion of 'salience', i.e. given a variety of competing possibilities, tend to maintain interest in the one that is most compelling. Although the salience value of (part of) a solution is ultimately represented ...
8
Yes, many people have worked on this sort of thing, due to its obvious industrial applications (most of the ones I'm familiar with are in the pharmaceutical industry). Here's a paper from 2013 that claims good results; following the trail of papers that cited it will likely give you more recent work.
8
I would recommend to start by reading this blogpost.
You can probably cannibalise the code to create a RNN that takes in one statement of a dialogue and then proceeds to output the answer to that statement.
That would be the easy version of your project, all without word vectors and thought vectors. You are just inputting characters, so typos don't need to ...
8
A simple form of sarcasm involves a direct reversal of the literal meaning of the statement, eg "Great weather we're having" (during a thunderstorm), "just what I needed" (when something goes wrong).
The problem with doing this in random sentences is that you may have no context to establish the reversal of the literal meaning.
You could possibly construct ...
7
Yes, it is possible to combine probabilistic / bayesian reasoning and a traditional "knowledgebase". And some work along those lines has been done. See, for example, ProbLog ("Probabilistic Prolog") which combines logic programming and probabilistic elements. See:
https://dtai.cs.kuleuven.be/problog/tutorial/mpe/01_bn.html
Another project to look at ...
7
Over the last few years, evolutionary computation research has shown increasing interest in including some aspect of epigenetics. For example:
A 2008 paper by Tanev and Yuta
Work from Lee Spector's genetic programming group
A recent paper by Ricalde and Banzhaf
7
There is stuff like the Universal Approximation Theorem.
There are also investigations into the loss surface of neural networks.
And classics like this explanation of the vanishing gradient problem.
But I'm afraid the mathematical theory of neural networks only exists in bits and pieces in many different papers. And many of the most important questions ...
7
Although I have only partially read (or not read at all) some of the following resources and some of these resources may not cover more advanced topics than the ones presented in the book you are reading, I think they can still be useful for your purposes, so I will share them with you.
I would also like to note that if you understand the contents of the ...
6
Concentration, perhaps easier to grasp as "focus" or "attention", has quite some history in AI. This answer mentions CopyCat, and there was work with neural networks in the 80s as well (e.g. from Fukushima, creator of the Neocognitron).
More recently, attention in neural networks is gaining momentum. The mechanisms are applied to learning in deep neural ...
6
Yes! This most certainly can be done. Since you have a labeled dataset, that makes it all the more simple!
I would take a look at this project and that should get you where you need to go.
The implementation details should be pretty straightforward. Let me know if I can help further.
6
You could say that NAS fits into the domain of Meta Learning or Meta Machine learning.
I've pulled the NAS papers from my notes, this is a collection of papers/lectures that I personally found very interesting. It's sorted in rough chronological descending order, and *** means influential / must read.
Quoc V. Le and Barret Zoph are to good authors on the ...
Only top voted, non community-wiki answers of a minimum length are eligible