Questions tagged [dense-layers]

The tag has no usage guidance.

Filter by
Sorted by
Tagged with
0 votes
2 answers
45 views

how to determine the number of units for dense layer for transfer learning?

I'm using MobileNetV2 for classification, and I want to add dense layers(i remove the last layer of the MobileNetV2 model). How do I choose the number of units for the dense layer after obtaining the ...
Cy Rine's user avatar
0 votes
1 answer
69 views

Time Series Classification using Transformer Encoder

Lets say I have a collection of tensors, each tensor representing a time series with 64 points and 4 features. The dimension of each tensor would be [64,4]. I am trying to classify these series. For ...
Zohaib Hamdule's user avatar
1 vote
1 answer
36 views

Number of units in Final softmax layer in VGGNet16

I am trying to implement and train neural network model VGGNet from scratch, on my own data. I am reproducing all the layers of the model. I am having a confusion about the last, fully connected ...
Dawood Ahmad's user avatar
1 vote
1 answer
165 views

Why in Multi-Head Attention implementation should we use $3$ linear layers for Q, K, V instead of $3 * h$ layers?

I have been trying to implement a Transformer architecture using PyTorch by following the Attention Is All You Need paper as well as the The Annotated Transformer blog post to compare my code with ...
Daviiid's user avatar
  • 573
1 vote
0 answers
51 views

About RNN followed by dense layer

In RNN we do get one output for each time stamp of input right i.e. if we give input as "I am Good" we get three outputs representing I followed by am and Good so if we connect a dense layer ...
Sowrabh meduri's user avatar
0 votes
0 answers
31 views

How can validation accuracy be more than test accuracy?

I have been trying to implement DenseNet on small dataset using k-fold cross validation. Training accuracy is 94% ,validation accuracy is 73% whereas test accuracy is 90%.I have taken 10% of my total ...
srij's user avatar
  • 13
0 votes
0 answers
23 views

Can I create and train a feedforward layer with only K active connections?

I had an idea for a layer, but I'm not sure if it exists already and/or if it's implementable in tensorflow. I would like to have a layer that is similar to a Dense layer, in the way that it's ...
Adrien Nivaggioli's user avatar
1 vote
1 answer
622 views

Why do transformer Key Query Value layers not have biases or activations?

Transformers use just matrices to transform input embeddings, which is halfway to being a connected dense layer (add a bias and activation). So, why don't transformers have dense layers for encoding ...
Artin Kim's user avatar
1 vote
1 answer
82 views

Does a second-order fully-connected layer have any uses?

I was thinking about implementing second-order regression via a fully-connected layer, and I came up with this: $X$ is the input data, shaped $(features, batch\_number)$. $w0$ is the bias, shaped $(...
HappyFace's user avatar
  • 113
0 votes
2 answers
85 views

Correctly input additional values into CNN

I understand that in order to add additional inputs to a CNN, e.g. in self driving, I can append the data to a flattened layer after the convolutions and before the fully connected layers. However, a ...
Benjoyo's user avatar
  • 101
1 vote
0 answers
102 views

Are there any benefits of adding attention to linear layers?

Is attention useful only in transformer/convolution layers? Can I add it to linear layers? If yes, how (on a conceptual level, not necessarily the code to implement the layers)?
pentavol's user avatar
2 votes
1 answer
671 views

What gets optimized in convolutional neural network?

In a convolutional neural network, the hyperparameters such as number of kernels and stride, kernel size, etc are determined. After some combination of convolutions, ReLU and pooling layer there is ...
user48144's user avatar
0 votes
1 answer
62 views

How to explain that a same DNN model have radically different behaviours with each new initialization and training?

I'm trying to predict the continuous values of a variable $y$ using a Fully Connected Neural Network while providing it with data from a $(3300, 13)$ matrix $X$ where $X[i, :]=[0,...,1,...,0,x_{i}]$. ...
Daviiid's user avatar
  • 573
0 votes
1 answer
1k views

Why my Fully Connected Neural Network outputs the same prediction?

I have a relatively small data set comprised of $3300$ data points where each data point is a $13$ dimensional vector where the $12$ first dimensions depict a "category" by taking the form ...
Daviiid's user avatar
  • 573
0 votes
1 answer
73 views

Must all CNNs and RNNs not have a fully connected layer in order to be considered as such?

In the paper Wrist-worn blood pressure tracking in healthy free-living individuals using neural networks, the authors talk about a combination of feed-forward and recurrent layers, as if FC layers ...
witdev's user avatar
  • 73
0 votes
0 answers
165 views

What is the difference between the forward pass of the Multi-Layer Perceptron, Deep AutoEncoder and Deep Belief Network?

Multi-Layer Perceptron (MLP), Deep AutoEncoder (DAE), and Deep Belief Network (DBN) are trained differently. However, do they follow the same process during the inference phase, i.e., do they ...
witdev's user avatar
  • 73
2 votes
1 answer
870 views

What's the purpose of layers without biases?

I noticed that the TensorFlow library includes a use_bias parameter for the Dense layer, which is set to ...
mark mark's user avatar
  • 763
1 vote
1 answer
791 views

Why does the output shape of a Dense layer contain a batch size?

I understand that the batch size is the number of examples you pass into the neural network (NN). If the batch size is 10, it means you feed the NN 10 examples at once. Assuming I have an NN with a ...
jaksnak's user avatar
  • 13
2 votes
3 answers
196 views

Why does a neuron in a multi-layer network need several input connections?

For example, if I have the following architecture: Each neuron in the hidden layer has a connection from each one in the input layer. 3 x 1 Input Matrix and a 4 x 3 weight matrix (for the ...
iwab's user avatar
  • 121
1 vote
1 answer
156 views

Can fully connected layers be used for feature detection?

I need help in understanding something basic. In this video, Andrew Ng says, essentially, that convolutional layers are better than fully connected (FC) layers because they use fewer parameters. But I'...
Justin's user avatar
  • 41
1 vote
0 answers
106 views

When to use convolutional layers as opposed to fully connected layers?

I am still new to CNNs, but I would like to check my understanding between when to use convolutional layers versus fully connected layers. From what I have read, we can use convolutional layers with ...
Jack Armstrong's user avatar
5 votes
1 answer
3k views

How to add a dense layer after a 2d convolutional layer in a convolutional autoencoder?

I am trying to implement a convolutional autoencoder with a dense layer at the bottleneck to do some dimensional reduction. I have seen two approaches for this, which aren't particularly scalable. The ...
Pavan Inguva's user avatar
2 votes
1 answer
286 views

How do you go from the last convolutional layer to your first fully connected layer?

I'm implementing a neural network framework from scratch in C++ as a learning exercise. There is one concept I don't see explained anywhere clearly: How do you go from your last convolutional or ...
Á. Márton's user avatar