5

Randomising just b sort of works, but setting w to all zero causes severe problems with vanishing gradients, especially at the start of learning. Using backpropagation, the gradient at the outputs of a layer L involves a sum multiplying the gradient of the inputs to layer L+1 by the weights (and not the biases) between the layers. This will be zero if the ...


5

As it can be easily pointed out that true random numbers cannot be generated fully by programming and some random seed is required. This is true. In fact, it is impossible to solve using software. No software-only technique can generate randomness without an initial random seed or support from hardware. This is also true for AI software. No AI design that ...


4

Inherently, no. The MLP is just a data structure. It represents a function, but a standard MLP is just representing an input-output mapping, and there's no recursive structure to it. On the other hand, possibly your source is referring to the common algorithms that operate over MLPs, specifically forward propagation for prediction and back propagation for ...


4

In many cases, a production-ready model has everything it needs to make predictions without retaining training data. For example: a linear model might only need the coefficients, a decision tree just needs rules/splits, and a neural network needs architecture and weights. The training data isn't required as all the information needed to make a prediction is ...


3

No. This is currently out of the scope for any language processing system. It requires a general understanding of abstract concepts which is not possible for machines at present. In order to recognise a self-fulfilling prophecy, you first need to identify that something is a prophecy. So it needs to be something that expresses a possible future state, for ...


3

Your confusion is a result of Moravec’s Paradox; the just tell it to... step is enormously more difficult than it sounds. So what researchers do is attempt to find a general approach to that problem. The problem is expressed as a matrix of numbers (a grid that may be a million on each side these days) and “learning” is the process of computing the values for ...


3

Sure, you can define plenty of things we don't generally need to regard as recursive as so. An MLP is just a series of functions applied to its input. This can be loosely formulated as $$ o_n = f(o_{n-1})$$ Where $o_n$ is the output of layer $n$. But this clearly doesn't reveal, much does it?


3

1): The intuition is based on the concept of value iteration, which the authors mention but don't explain on page 504. The basic idea is this: imagine you knew the value of starting in state x and executing an optimal policy for n timesteps, for every state x. If you wanted to know the optimal policy (and it's value) for running for n+1 timesteps in each ...


2

Such a great question. I would concur with Dennis Soemers comment that humans are not great at thinking of random numbers (just think about any card trick). However, we are very good at creating randomness through our actions. If you consider moving a computer mouse, the stock market, or playing a lottery, humans are very good at creating randomness through ...


2

To paraphrase: "Is there an analogy between client/server in web development and agent/environment in reinforcement learning?" The answer is "not really". There is no useful analogy here that allows any insight into RL from web server knowledge or vice versa. However, you could set up an agent where the goal was to collect information, and the available ...


2

Most of the explanations given for choosing something or not choosing something (like hyper-parameter tuning) in deep learning, are based on empirical studies like analysing the error over a number of iterations. So the answer by @Joe S is what people in deep learning side give. Since you have asked for a mathematical explanation, I suggest you to read this ...


2

I don't know if my intuition is correct but I will give it a try. You could see weights as how much important one thing is, the problem is to understand what that thing represents. When I say thing I'm referring to the output of a specific neuron. I don't think that we can say what the output of a neuron represents in the real world unless we directly ...


2

Welcome to AI.SE @rudreshdwivedi! This is a great question, and I hope to see many more like it. Demster-Shafer Theory and Bayesian Networks were both techniques that rose to prominence within AI in the 1970's and 1980's, as AI started to seriously grapple with uncertainty in the world, and move beyond the sterilized environments that most early systems ...


2

I’ll have a stab at this. Cognitive performance in narrow domains is determined by competency, efficiency and speed. Take calculating numbers, extremely narrow domain but compared to humans the ability of a calculator to calculate numbers exceeds normal human performance, it is much competent in terms of speed. In a bit broader domain, AlphaGo has defeated ...


1

I am reading Bostrom's book "Superintelligence". I have only read the first 2 chapters, but I think he doesn't want to define super-intelligence is a precise way, but he leaves the reader the option to define it in a "sensible" way. However, I think that, in his thoughts, there's the (clear) assumption that a super-intelligence will necessarily need to be ...


Only top voted, non community-wiki answers of a minimum length are eligible