6

You are talking about two different types of 'size'. The size of the input for a FFNN and a RNN must always remain fixed for the same network architecture, i.e. they take in a vector $x \in \mathbb{R}^d$ and could not take as input for instance a vector $y \in \mathbb{R}^b$ where $b \neq d$. The size you refer to in the context of the RNN is the length of ...


4

According to wikipedia of backpropagation: In fitting a neural network, backpropagation computes the gradient of the loss function during supervised learning with respect to the weights of the network for a single input–output example, and does so efficiently, unlike a naive direct computation of the gradient with respect to each weight individually. ...


3

Yes, there are different ways. What I think you are looking for is under the research field of Localization and Mapping. Which divides in the following subfields: For getting current (the robot) position and trajectory go to models for Odometry Estimation For getting a representation of the world around the robot go to models for Mapping If you want both of ...


3

As you stated, it's popular to have some form of a rectified linear unit (ReLU) activation in hidden layers and the output layer is often a softmax or sigmoid (depending also on the problem: multi-class or binary classification, respectively), which provides an output that can be viewed as a probability distribution. You could generalize this further to ...


1

Rosenblatt was probably discussing a specific architecture, for which there are many. However, for general purpose feed-forward back-propagation ANNs used for function aproximation and classification analysis, you can use whatever activation functions you want on the input-side, hidden layers, and output-side. Examples are identity, logistic, tanh, ...


1

Yes, it is possible. What you have shown in case of ANN is what happens in a regression model using NNs. What you have shown in case of RNN is what happens when you are doing sequence-to-sequence translation (like French to English). If you want to get single values like in case of ANN, suppose you are doing regression, then, in the end, you will flatten the ...


1

Deep Learning by Goodfellow et. al is a good book for anything related neural networks, and it's freely available online. Backpropagation is covered in Chapter 6.5.


1

Firstly, concatenate only works on identical output shape of the axis. Otherwise, the function will not work. Now, your function output size is (None, 32, 50) and (None, 600, 1). Here, '32' and '600' must be same when you want to concatenate. I would like to suggest some advice based on your problem. You can flatten both of them first and then concatenate. ...


1

As you say, the outputs are modeled as a vector, each output in one vector component. In regression problems: The most common loss function, like in the scalar case, is the square error. Skipping constants, it is defined as: $$E=\sum_i ||\mathbf{y_i}-\mathbf{\hat{y_i}}||^2 = \sum_i (\mathbf{y_i}-\mathbf{\hat{y_i}})(\mathbf{y_i}-\mathbf{\hat{y_i}})$$ where: $...


1

Assuming you're using softmax on the last layer for classification, it sounds like a simple application of cross entropy loss from here on out: https://datascience.stackexchange.com/questions/20296/cross-entropy-loss-explanation Edit:


Only top voted, non community-wiki answers of a minimum length are eligible