2

Interesting question, I can come with 2 explanations why we don't initialize weights with 1 mean value : It may be easier for the network to learn identity function, but we may have a similar issue about not being able to learn comparison, comparison is quite an important reasoning in my opinion, this is why having negative weight values is important, and ...


2

Quite surprising to find someone reading this same book. I read this part a week ago and the explanation is quite clear in the book : If you use successive shallow learning methods, you first train one model, then you train another model with the outputs of your first model, and then a third with the outputs of your 2nd model. The problem with that is that ...


1

It sounds like you have structured/tabular data. So, a fully-connected feedforward network should do the job.


1

It depends on what your outputs are. For example, if both outputs are similar then you can use one output branch. However, what if the two outputs are different? With two output branches you can used two different loss functions. Now your model will optimize the two branches separately. Imagine if you have a model that has to output a class label for the ...


1

In its most raw form, convolution is defined as: $(f*g)(t) = \int_{-\infty}^\infty f(\tau) \cdot g(t-\tau) d\tau$. Here, t doesn't represent the time domain. Infact, it represents the real valued argument the book is talking about. In this notion, at moment t, convolution can be thought of as a weighted average of the function $f(\tau)$ weighted by $g(–\tau)$...


1

Yes, the functionality should is there. But, don't you think you are overdoing the scales. You have at least 18 scales mentioned here. Too much of anything is bad. There is a reason it likes things divisible by 32 because at that increase in size something more meaningful will show up in the image. Spamming sizes like this won't help you at all, it would ...


1

I commonly use softmax for all 2-class or k-class problems, basically, because I always like to have an output node for each class. For sigmoid, i.e., logistic, you cannot estimate MSE for each sample using the relationship $E_i = \sum_c^C (y_c - \hat{y}_c)^2$, where $C$ is the number of classes, $y_c$ is 0 or 1 for true class membership, and $\hat{y}_c$ is ...


1

Sigmoid is used for binary cases and softmax is its generalized version for multiple classes. But, essentially what they do is over exaggerate the distances between the various values. If you have values on a unit sphere, apply sigmoid or softmax on those values would lead to the points going to the poles of the sphere.


1

The paper has sources, that contains these plots in pdf. I've converted those to svg and managed to rip the actual data: xdata = [-0.6,-3.33,-1.93,0.67,-2.63,-3.1,-5.93,-1.87,-3.93,-3.47,1.0,-3.43,-3.83,0.6,-0.47,-3.6,-1.37,-4.87,-3.63,3.53,4.33,1.73,2.63,4.13,-0.13,2.33,-1.6,-6.83,-1.5,-6.37,3.73,0.2,-0.73,-3.17,2.43,-0.3,-1.2,-1.97,-0.3,-2.9,-2.13,-1.4,4....


1

No one mentioned Planning chemical syntheses with deep neural networks and symbolic AI (published in Nature - here's arxiv link). Very impressive application of deep reinforcement learning - they use Monte Carlo Tree Search with a policy network (a-la AlphaZero) to do chemical synthesis planning. Authors claim that double blind test shown that professional ...


Only top voted, non community-wiki answers of a minimum length are eligible