A message from our CEO about the future of Stack Overflow and Stack Exchange. Read now.

Questions tagged [convolutional-neural-networks]

For questions about convolutional neural networks, also known as CNN or ConvNet.

Filter by
Sorted by
Tagged with
2
votes
0answers
29 views

What is the difference between Squeeze-and-excite and bottleneck modules from Mobilenet v2?

Squezee-and-excite networks introduced SE blocks, while MobileNet v2 introduced linear bottlenecks. What is the effective difference between these two concepts? Is it only implementation (depth-wise ...
5
votes
2answers
233 views

What is the difference between graph convolution in the spatial vs spectral domain?

I've been reading different papers regarding graph convolution and it seems that they come into two flavors: spatial and spectral. From what I can see the main difference between the two approaches is ...
2
votes
2answers
59 views

Why do we get a three-dimensional output after a convolutional layer?

In a convolutional neural network, when we apply the convolution on a $5 \times 5$ image with $3 \times 3$ kernel, with stride $1$, we should get only one $4 \times 4$ as output. In most of the CNN ...
6
votes
1answer
41 views

Is there any use of using 3D convolutions for traditional images (like cifar10, imagenet)?

I am curious if there is any advantage of using 3D convolutions on Images like Cifar10/100 or Imagenet. I know that they are not usually used on this data set, though they could because the channel ...
1
vote
0answers
49 views

How to train and update weights of filters

I have some problems with training CNN :( For example: Input 6x6x3, 1 core 3x3x3, output = 4x4x1 => pool: 2x2x1 By backpropagation I calculated deltas for output. This tutor and other tutors are ...
1
vote
1answer
71 views

What is the difference between 2d vs 3d convolutions?

I was trying to understand the definition of 2d convolutions vs 3d convolutions. I saw the "simplest definition" according to Pytorch and it seems the following: 2d convolutions map $(N,C_{in},H,W) \...
1
vote
0answers
15 views

How do I recover the 3D structure of a layer after a fully-connected layer?

I want to implement a CNN, but I want to explore what happens when my first layer is a fully-connected one. I still want to use convolutions, of course, but I want to apply them after the first layer. ...
2
votes
1answer
37 views

Tweaking a CNN for large number of input channels

I am using a CNN for function approximation using geospatial data. The input of the function I am trying to approximate consists of all the spatial distances between N location on a grid and all the ...
0
votes
0answers
22 views

Super Resolution CNN generates black dots on output images

I have been trying to train a CNN for the super-resolution task based on the work of Dong et al., 2015 [1]. The network structure built in PyTorch is as follows: ...
0
votes
1answer
56 views

What is the correct way to read and analyse images in machine learning?

I am trying to understand the best practice to read and analyze images. If your image has 10,000 pixels, your input layers will have 10,000 inputs? It sounds that my neural network will have too many ...
1
vote
1answer
159 views

What do the numbers in this CNN architecture stand for?

So I've got a neural net model (ResNet-18) and made a diagram according to the literature (https://arxiv.org/abs/1512.03385). I think I understand most of the format of the convolutional layers: ...
2
votes
2answers
74 views

How should we pad an image to be fed in a CNN?

As everyone experienced in deep learning might know, in an image classification problem we normally add borders to images then resize it to the input size of a CNN network. The reason of doing this is ...
3
votes
2answers
72 views

Are fully connected layers necessary in a CNN?

I have implemented a CNN for image classification. I have not used fully connected layers, but only a softmax. Still, I am getting results. Must I use fully-connected layers in a CNN?
2
votes
2answers
52 views

Untrained CNNs as feature extractors?

I've heard somewhere that due to their nature of capturing spatial relations, even untrained CNNs can be used as feature extractors? Is this true? Does anyone have any sources regarding this I can ...
1
vote
0answers
20 views

Literature on Sequence Regresssion

I have some rated time-sequential data and I would like to test if an ANN can learn a correlation between my measurements and ratings. I suspect I could just try a CNN where 1 Dimension is time or an ...
2
votes
1answer
32 views

What loss function is appropriate for finding “points of interest” in a array of x,y inputs

I am looking into whether a neural network is appropriate to detect "points of interest" (POI) in a set of tuples (say length, and some sensor value). A POI is essentially a quick change in the value ...
4
votes
1answer
126 views

What is the difference between asymmetric and depthwise separable convolution?

I have recently discovered asymmetric convolution layers in deep learning architectures, a concept which seems very similar to depthwise separable convolutions. Are they really the same concept with ...
2
votes
0answers
31 views

Pipeline to Estimate Measurement of Human Body Point Cloud

I am developing a Body Measurement extraction application, my current stage is able to extract the point clouds of human body in a standing posture, from every angles. Now, to be able to recognize ...
1
vote
1answer
62 views

Are the training loss and validation loss plotted per sample or per batch?

I am using a CNN to train on some data, where training size = 21700 samples, and test size is 653 samples, and say I am using a batch_size of 500 (I am accounting for samples out of batch size as well)...
5
votes
2answers
457 views

When should I use 3D convolution?

I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, ...
4
votes
1answer
159 views

What is the meaning of “stationarity of statistics” and “locality of pixel dependencies”?

I'm reading the ImageNet Classification with Deep Convolutional Neural Networks paper by Krizhevsky et al, and came across these lines in the Intro paragraph: Their (convolutional neural networks') ...
1
vote
1answer
44 views

Is there a theory behind which model is good for a classification task for the convolutional neural network?

Let say I'm trying to apply CNN for image classification. There are lots of different models to choose and we can try an ensemble, but given a limit amount of resources, it does not allow to try ...
2
votes
1answer
73 views

What does the words “coarse” & “fine” means in the context of computer vision and semantic segmentation?

I was reading the well know paper Fully Convolutional Networks for Semantic Segmentation and throughout the whole paper they talk use the term fine and coarse. I was wondering what the meant. The ...
1
vote
1answer
24 views

Model Performance and Size of Data Set

Suppose we have a data set with $4,000$ labeled examples. The outcome variable is trinary (three possible categorical values). Suppose the accuracy of a given model is "bad" (e.g. less than $50 \%$). ...
1
vote
2answers
356 views

Inception Resnet V2 Feature Extraction Layer?

I am working with Inception Resnet V2 with "Imagenet" pre-trained model for face recognition. So that I tend to ignore the Fully Connected Layer to get the extract feature. But I'm so confused of what ...
2
votes
2answers
90 views

What are the major differences between cost, loss, error, fitness, utility, objective, criterion functions?

I find the terms cost, loss, error, fitness, utility, objective, criterion functions to be interchangeable, but any kind of minor difference explained is appreciated.
0
votes
0answers
31 views

Pooling vs Subsampling: Multiple Definitions?

I have seen people using pooling and subsampling synonymously. I have also seen people use them as different processes. I am not sure though if I have correctly inferred what they mean, when they use ...
6
votes
1answer
202 views

What are the benefits of using max-pooling in convolutional neural networks?

I am reading Francois Chollet's Deep learning with Python, and I came across a section about max-pooling that's really giving me trouble. I am unable to copy-paste the content, so I've included ...
0
votes
1answer
48 views

Understanding arrangement of applying filters to input channels

I was watching a video about Convolutional Neural Networks: https://www.youtube.com/watch?v=SQ67NBCLV98. What I'm confused about is the arrangement of applying the filters' channels to the input image ...
0
votes
1answer
36 views

Use deep learning to rank video scenes

I'm new to machine learning and especially, deep learning. Given a video (and it's subtitle), I need to generate a 10-second summary out of this video. How can I use ML and DL to produce the most ...
3
votes
1answer
63 views

Clarification on definition of convolution: Is adding the Frobenius inner products between filter and input part of convolution or a separate step?

From the literature I have read so far, it is not clear how exactly the convolution operation is defined. It seems people use two different definitions: Let us assume we are given an $n_w \times n_h \...
0
votes
0answers
14 views

Why are my convolutional layer activations large and saturating the softmax function?

I implemented a neural network in python from scratch (The github repo to it.). It works reasonably well and I benchmarked different designs using convolutional layers on MNIST and it worked better ...
2
votes
0answers
50 views

Understanding CNN+LSTM concept with attention and need help

I have a question about the context of CNN and LSTM. I have trained a CNN network for image classification. However, I would like to combine it with LSTM for visualizing the attention weights. So, I ...
1
vote
0answers
10 views

Estimating camera's offset to its true position

I have the following problem: I get a 360 RGB image in a room. I've the 3D model of this room, hence, I can generate a 3D nominal mask of the room (1-wall, 2-ceiling, 3-floor, 4-door, etc..) in a ...
2
votes
1answer
38 views

How is a neural network where the majority of inputs are 0 trained?

Take alexnet. Alexnet has 1000 output nodes, each of which classify an image: The problem I have been having with training a neural network of similar proportions, is that it does what any reasonable ...
1
vote
1answer
20 views

Why are activation functions independent layers in CNNs rather than part of convolutional layers?

I have been reading up on CNNs. One of the different confusing things has been that people always talk of normalization layers. A common normalization layer is a ReLU layer. But I never encountered an ...
3
votes
2answers
45 views

Are feature maps merged or are they passed on as they are?

I am unsure about the following parts of the architecture and mechanics of convolution layers in CNNs. Possibly, this is implementation-dependent though. First question: Say I have 2 convolution ...
0
votes
1answer
58 views

If the goal of training of a GAN is to have $P_g=P_{data}$, shouldn't this produce the exact same images?

Referring to the blog, Image Completion with Deep Learning in TensorFlow, it clearly says that we would want a generator $g$ whose modeled distribution fits our dataset $data$, in other words, $P_{...
1
vote
1answer
56 views

Understanding the reconstruction loss in the paper “Anomaly Detection using Deep Learning based Image Completion”

I would like to implement the approach represented in this paper. Here they used following reconstruction loss: $$ L(X)= \frac{\lambda \cdot || M \odot (X - F(\overline{M} \odot X)) ||_{1} + (1 - \...
3
votes
1answer
52 views

Why do we normalize data in a deep neural network?

I have asked this question a number of times, but I always get confusing answers to this, like "normalized data works better", "data lives in the same scale" How can ...
0
votes
0answers
36 views

Imbalance large dataset with keras using flow from directory

I want to detect dataset bias, and for that, the first approach is to build a model that can recognize from which dataset belongs an image. I am working with Python3, with limited computing and more ...
3
votes
0answers
102 views

How can I incrementally train a Yolo model without catastrophic forgetting?

I have successfully trained a Yolo model to recognize k classes. Now I want to train by adding k+1 class to the pre-trained weights (k classes) without forgetting previous k classes. Ideally, I want ...
1
vote
1answer
74 views

Dropout causes too much noise for network to train

I am using dropout of different values to train my network. The problem is, dropout is contributing almost nothing to training, either causing so much noise the error never changes, or seemingly ...
1
vote
0answers
53 views

Can I perform image recognition using a very small dataset?

I have 3 types of classes and I have 2 images of each object (or class). Can I perform image recognition using this very small dataset?
0
votes
0answers
34 views

What to look for when CNN returns same prediction for every input?

I am trying to use a CNN to do a regression prediction on some statistical data. The data is time-series data formatted into a 2-D grid. The network I'm using looks like this: ...
1
vote
1answer
54 views

What is the best loss function for convolution neural network and autoencoder?

What is the best choice for loss function in Convolution Neural Network and in Autoencoder in particular - and why? I understand that the MSE is probably not the best choice, because little ...
2
votes
1answer
51 views

What is the most common practice to apply batch normalization?

For a deep NN, should I generally apply batch normalization after each convolution layer? Or only after some of them? Which? Every 2nd, every 3rd, lowest, highest, etc.?
4
votes
1answer
252 views

Can a vanilla neural network theoretically achieve the same performance as CNN?

I perfectly understand that CNN takes into account the local dependency of each pixel to the nearby pixels. In addition, CNNs are spatially invariant which means that they are able to detect the same ...
1
vote
1answer
71 views

Is there a simple way of classifying images of size differing from the input of existing image classifiers?

Most image classifiers like Inception-v3 accept images of about size 299 x 299 x 3 as input. In this particular case, I cannot resize the image and lose resolution. Is there an easy solution of ...
0
votes
0answers
23 views

How to implement Multiple Neural network architecture, connected in parallel and series in Keras or Pytorch

Hello Dear StackExchange members, I want to make a deep network as shown in the image. I want each 'network 1 to look at the specific part of the input and I don't want to divide my input beforehand ...