Questions tagged [dense-layers]
The dense-layers tag has no usage guidance.
16
questions
1
vote
1
answer
29
views
Does a second-order fully-connected layer have any uses?
I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this:
$X$ is the input data, shaped $(features, batch\_number)$.
$w0$ is the bias, shaped $(...
- 113
0
votes
0
answers
16
views
U-Net with freezed encoder and customize decoder
I want to use a U-Net architecture, but I want only to use the feature extraction part and train the decoder for my task, but I'm not sure if the decoder of U-Net would be enough for learning new ...
- 33
0
votes
2
answers
56
views
Correctly input additional values into CNN
I understand that in order to add additional inputs to a CNN, e.g. in self driving, I can append the data to a flattened layer after the convolutions and before the fully connected layers.
However, a ...
- 101
1
vote
0
answers
34
views
Are there any benefits of adding attention to linear layers?
Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
- 13
2
votes
1
answer
148
views
What gets optimized in convolutional neural network?
In a convolutional neural network, the hyperparameters such as number of kernels and stride, kernel size, etc are determined. After some combination of convolutions, ReLU and pooling layer there is ...
- 23
0
votes
1
answer
45
views
How to explain that a same DNN model have radically different behaviours with each new initialization and training?
I'm trying to predict the continuous values of a variable $y$ using a Fully Connected Neural Network while providing it with data from a $(3300, 13)$ matrix $X$ where $X[i, :]=[0,...,1,...,0,x_{i}]$. ...
- 489
0
votes
1
answer
134
views
Why my Fully Connected Neural Network outputs the same prediction?
I have a relatively small data set comprised of $3300$ data points where each data point is a $13$ dimensional vector where the $12$ first dimensions depict a "category" by taking the form ...
- 489
0
votes
1
answer
61
views
Must all CNNs and RNNs not have a fully connected layer in order to be considered as such?
In the paper Wrist-worn blood pressure tracking in healthy free-living individuals using neural networks, the authors talk about a combination of feed-forward and recurrent layers, as if FC layers ...
- 73
0
votes
0
answers
126
views
What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?
Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently.
However, do they follow the same process during the inference phase, i.e., do they ...
- 73
2
votes
1
answer
127
views
What's the purpose of layers without biases?
I noticed that the TensorFlow library includes a use_bias parameter for the Dense layer, which is set to ...
- 663
1
vote
1
answer
292
views
Why does the output shape of a Dense layer contain a batch size?
I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once.
Assuming I have an NN with a ...
- 13
2
votes
3
answers
145
views
Why does a neuron in a multi-layer network need several input connections?
For example, if I have the following architecture:
Each neuron in the hidden layer has a connection from each one in the
input layer.
3 x 1 Input Matrix and a 4 x 3 weight matrix (for the ...
- 21
1
vote
1
answer
103
views
Can fully connected layers be used for feature detection?
I need help in understanding something basic.
In this video, Andrew Ng says, essentially, that convolutional layers are better than fully connected (FC) layers because they use fewer parameters.
But I'...
- 21
1
vote
0
answers
80
views
When to use convolutional layers as opposed to fully connected layers?
I am still new to CNNs, but I would like to check my understanding between when to use convolutional layers versus fully connected layers.
From what I have read, we can use convolutional layers with ...
- 111
5
votes
1
answer
2k
views
How to add a dense layer after a 2d convolutional layer in a convolutional autoencoder?
I am trying to implement a convolutional autoencoder with a dense layer at the bottleneck to do some dimensional reduction. I have seen two approaches for this, which aren't particularly scalable. The ...
- 71
2
votes
1
answer
227
views
How do you go from the last convolutional layer to your first fully connected layer?
I'm implementing a neural network framework from scratch in C++ as a learning exercise. There is one concept I don't see explained anywhere clearly:
How do you go from your last convolutional or ...
- 123