# Tag Info

Accepted

### Why is the derivative of the activation functions in neural networks important?

Consider a dataset $\mathcal{D}=\{x^{(i)},y^{(i)}:i=1,2,\ldots,N\}$ where $x^{(i)}\in\mathbb{R}^3$ and $y^{(i)}\in\mathbb{R}$ $\forall i$ The goal is to fit a function that best explains our dataset....

### Is it possible to train the neural network to solve math equations?

Not really. Neural networks are good for determining non-linear relationships between inputs when there are hidden variables. In the examples above, the relationships are linear, and there are no ...

### Is it possible to train the neural network to solve math equations?

It is possible! In fact, it's an example of the popular deep learning framework Keras. Check out this link to see the source code. This particular example uses a recurrent neural network (RNN) to ...

### Can neural networks be used to prove conjectures?

Your idea may be feasible in general, but a neural network is probably the wrong high level tool to use to explore this problem. A neural network's strength is in finding internal representations ...

### Can neural networks be used to prove conjectures?

Not in such straight forward way as described, but neural networks are successfully applied to guide the search of proof. There are automated theorem provers. What they do look roughly like this: Get ...
Accepted

### Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The derivative of $\mathcal{L_1}(y, x) = (\hat{y} - y)^2 = (f(x) - y)^2$ with respect to $\hat{y}$, where $f$ is the model and $\hat{y} = f(x)$ is the output of the model, is \begin{align} \frac{d}{...

### Which function $(\hat{y} - y)^2$ or $(y - \hat{y})^2$ should I use to compute the gradient?

The MSE can be defined as $(\hat{y} - y)^2$, which should be equivalent to $(y - \hat{y})^2$ They are not just "equivalent". It is actually the exact same function, with two different ways to write ...

### Why is the derivative of the activation functions in neural networks important?

If what you are asking is what is the intuition for using the derivative in backpropagation learning, instead of an in-depth mathematical explanation: Recall that the derivative tells you a function'...

### Why do activation functions need to be differentiable in the context of neural networks?

No, it is not necessary that an activation function is differentiable. In fact, one of the most popular activation functions, the rectifier, is non-differentiable at zero! This can create problems ...

### Can neural networks be used to prove conjectures?

It's possible, but probably not a good idea. Logical proof is one of the oldest areas of AI, and there are purpose-built techniques that don't need to be trained, and that are more reliable than a ...
Accepted

### Is it ok to struggle with mathematics while learning AI as a beginner?

I think the key part of your question is "as a beginner". For all intents and purposes you can create a state of the art (SoTA) model in various fields with no knowledge of the mathematics what so ...
Accepted

### What does the Markov assumption say about the history of state sequences?

A stochastic process has the Markov property if the probability distribution of future states conditioned on both the present and past states depends only on the present state or, more formally, the ...

### What is Lipschitz constraint and why it is enforced on discriminator?

The Lipschitz constraint is essentially that a function must have a maximum gradient. The specific maximum gradient is a hyperparameter. It's not mandatory for a discriminator to obey a Lipschitz ...

### How does ChatGPT know math?

I think that the dataset is so large and the model so well trained that it understood the probabilistic correlation of length in a token of numbers before a dot separation, and then the influence of ...
Accepted

### How can I start learning mathematics for machine learning?

You should begin from Dr Andrew Ng machine learning course on Coursera. It's probably the most popular course for newcomers in machine learning. It's a free course. You should also grab "Elements of ...
Accepted

### What does "probabilistically" mean?

In a genetic algorithm, crossover (recombination) is the analogy to mating in the real world. For example, you have some genetic information from each parent. In the case of an optimization where you ...
Accepted

### How good is AI in math?

Nim was actually one of the first games ever played by an electronic machine. It was called the Nimatron and was displayed at the 1940 New York World's Fair. It is also well known that neural networks ...

### Is the mean-squared error always convex in the context of neural networks?

Answer in short: MSE is convex on its input and parameters by itself. But on an arbitrary neural network it is not always convex due to the presence of non-linearities in the form of activation ...
Accepted

### Is recursion used in practice to improve performance of AI systems?

To my knowledge, recursion does not play a strong role in the definition of modern AI techniques, although it does feature used in Lovasz's definition of 'Local Search' and Kurzweil is certainly an ...
Accepted

### What makes multi-layer neural networks able to perform nonlinear operations?

Nonlinear relations between input and output can be achieved by using a nonlinear activation function on the value of each neuron, before it's passed on to the neurons in the next layer.