Questions tagged [convolutional-neural-networks]

For questions about convolutional neural networks, also known as CNN or ConvNet.

Filter by
Sorted by
Tagged with
3
votes
2answers
376 views

Do convolutional neural networks perform convolution or cross-correlation?

Typically, people say that convolutional neural networks (CNN) perform the convolution operation, hence their name. However, some people have also said that a CNN actually performs the cross-...
88
votes
7answers
16k views

Do scientists know what is happening inside artificial neural networks?

Do scientists or research experts know from the kitchen what is happening inside complex "deep" neural network with at least millions of connections firing at an instant? Do they understand the ...
81
votes
9answers
6k views

How is it possible that deep neural networks are so easily fooled?

The following page/study demonstrates that the deep neural networks are easily fooled by giving high confidence predictions for unrecognisable images, e.g. How this is possible? Can you please ...
53
votes
8answers
41k views

In a CNN, does each new filter have different weights for each input channel, or are the same weights of each filter used across input channels?

My understanding is that the convolutional layer of a convolutional neural network has four dimensions: input_channels, filter_height, filter_width, number_of_filters. Furthermore, it is my ...
4
votes
1answer
1k views

How can the convolution operation be implemented as a matrix multiplication?

How can the convolution operation used by CNNs be implemented as a matrix-vector multiplication? We often think of the convolution operation in CNNs as a kernel that slides across the input. However, ...
2
votes
1answer
251 views

Why do we need convolutional neural networks instead of feed-forward neural networks?

Why do we need convolutional neural networks instead of feed-forward neural networks? What is the significance of a CNN? Even a feed-forward neural network will able to solve the image classification ...
33
votes
5answers
63k views

What is the difference between a convolutional neural network and a regular neural network?

I've seen these terms thrown around this site a lot, specifically in the tags convolutional-neural-networks and neural-networks. I know that a neural network is a system based loosely on the human ...
3
votes
1answer
6k views

What is a fully convolution network?

I was surveying some literature related to Fully Convolutional Networks and came across the following phrase, A fully convolutional network is achieved by replacing the parameter-rich fully ...
2
votes
1answer
161 views

How is the depth of filters of hidden layers determined?

I am a bit confused about the depth of the convolutional filters in a CNN. At layer 1, there are usually about 40 3x3x3 filters. Each of these filters outputs a 2d array, so the total output of the ...
4
votes
2answers
4k views

How to calculate the number of parameters of a convolutional layer?

I was recently asked at an interview to calculate the number of parameters for a convolutional layer. I am deeply ashamed to admit I didn't know how to do that, even though I've been working and using ...
1
vote
1answer
3k views

What is the computational complexity of the forward pass of a convolutional neural network?

How do I determine the computational complexity (big-O notation) of the forward pass of a convolutional neural network? Let's assume for simplicity that we use zero-padding such that the input size ...
0
votes
1answer
123 views

What is a convolutional neural network?

Given that this question has not yet been asked on this site, although similar questions have already been asked in the past (e.g. here or here), what is essentially a convolutional neural network (...
70
votes
3answers
50k views

How can neural networks deal with varying input sizes?

As far as I can tell, neural networks have a fixed number of neurons in the input layer. If neural networks are used in a context like NLP, sentences or blocks of text of varying sizes are fed to a ...
34
votes
6answers
10k views

Why do CNN's sometimes make highly confident mistakes, and how can one combat this problem?

I trained a simple CNN on the MNIST database of handwritten digits to 99% accuracy. I'm feeding in a bunch of handwritten digits, and non-digits from a document. I want the CNN to report errors, so I ...
11
votes
5answers
7k views

What is the fundamental difference between CNN and RNN?

What is the fundamental difference between convolutional neural networks and recurrent neural networks? Where are they applied?
7
votes
2answers
4k views

When should I use 3D convolution?

I am new to convolutional neural networks, and I am learning 3D convolution. What I could understand is that 2D convolution gives us relationships between low-level features in the X-Y dimension, ...
4
votes
1answer
1k views

How to detect vanishing gradients?

Can vanishing gradients be detected by the change in distribution (or lack thereof) of my convolution's kernel weights throughout the training epochs? And if so how? For example, if only 25% of my ...
4
votes
2answers
199 views

Can neurons in MLP and filters in CNN be compared?

I know they are not the same in working, but an input layer sends the input to $n$ neurons with a set of weights, based on these weights and the activation layer, it produces an output that can be fed ...
4
votes
1answer
77 views

Is there anything that ensures that convolutional filters don't end up the same?

I trained a simple model to recognize handwritten numbers from the mnist dataset. Here it is: ...
3
votes
2answers
131 views

Are filters fixed or learned?

No matter what I google or what paper I read, I can't find an answer to my question. In a deep convolutional neural network, let's say AlexNet (Krizhevsky, 2012), filters' weights are learned by means ...
3
votes
1answer
215 views

Is continuous learning possible with a deep convolutional neural network, without changing its topology?

In general, is continuous learning possible with a deep convolutional neural network, without changing its topology? In my case, I want to use a convolutional neural network as a classifier of ...
3
votes
2answers
163 views

Are convolutional neural networks inspired by the human brain?

The Deep Learning book by Goodfellow et al. states Convolutional networks stand out as an example of neuroscientific principles influencing deep learning. Are convolutional neural networks (CNNs) ...
1
vote
1answer
51 views

Should one use an “other” category in image classification?

In image classification, there are sometimes images that do not fit in any category. For example, if I build a CNN in Keras to classify Dogs and Cats, does it help (in terms of training time and ...
23
votes
3answers
37k views

How do I handle large images when training a CNN?

Suppose that I have 10K images of sizes $2400 \times 2400$ to train a CNN. How do I handle such large image sizes without downsampling? Here are a few more specific questions. Are there any ...
12
votes
2answers
5k views

Which layer in a CNN consumes more training time: convolution layers or fully connected layers?

In a convolutional neural network, which layer consumes more training time: convolution layers or fully connected layers? We can take AlexNet architecture to understand this. I want to see the time ...
8
votes
2answers
6k views

How do we choose the kernel size depending on the problem?

Obviously, finding suitable hyper-parameters for a neural network is a complex task and problem or domain-specific. However, there should be at least some "rules" that hold most times for the size of ...
3
votes
2answers
347 views

What is the effect of using pooling layers in CNNs?

I know how pooling works, and what effect it has on the input dimensions - but I'm not sure why it's done in the first place. It'd be great if someone could provide some intuition behind it - while ...
2
votes
2answers
2k views

What's the role of bounding boxes in object detection?

I'm quite new to the field of computer vision and was wondering what are the purposes of having the boundary boxes in object detection. Obviously, it shows where the detected object is, and using a ...
6
votes
1answer
201 views

Are there well-established ways of mixing different inputs (e.g. image and numbers)?

I am interested in the possibility of having extra input along with the main data. For instance, a medical application that would rely mostly on an image: how could one also account for sex, age, etc.?...
6
votes
1answer
1k views

What are the benefits of using max-pooling in convolutional neural networks?

I am reading Francois Chollet's Deep learning with Python, and I came across a section about max-pooling that's really giving me trouble. I am unable to copy-paste the content, so I've included ...
4
votes
2answers
4k views

What is feature embedding in the context of convolutional neural networks?

What are feature embeddings in the context of convolutional neural networks? Is it related to bottleneck features or feature vectors?
2
votes
1answer
433 views

Can ConvNets be used for real-time object recognition from video feed?

Convolutional neural network are leading type of feed-forward artificial neural network for image recognition. Can they be used for real-time image recognition for videos (frame by frame), or it takes ...
1
vote
1answer
106 views

How many layers exists in my neural network?

I have a neural network model defined as below. How many layers exist there? Not sure which ones to count when we are asked about the number. ...
0
votes
1answer
47 views

Use deep learning to rank video scenes

I'm new to machine learning and especially, deep learning. Given a video (and it's subtitle), I need to generate a 10-second summary out of this video. How can I use ML and DL to produce the most ...
10
votes
2answers
1k views

What benefits can be got by applying Graph Convolutional Neural Network instead of ordinary CNN?

What benefits can we got by applying Graph Convolutional Neural Network instead of ordinary CNN? I mean if we can solve a problem by CNN, what is the reason should we convert to Graph Convolutional ...
6
votes
1answer
457 views

Game AI - Fast python OCR or cropped image input

I'm developing a Game AI which tries to master racing simulations. I already trained a CNN (alexnet) on ingame footage of me playing the game and the pressed keys as the target. As the CNN is only ...
6
votes
3answers
2k views

Convolutional Neural Network: does each filter in each convolution layer create a new image?

Say I have a CNN with this structure: input = 1 image (say, 30x30 RGB pixels) first convolution layer = 10 5x5 convolution filters second convolution layer = 5 3x3 convolution filters one dense layer ...
4
votes
1answer
501 views

Can a CNN be trained incrementally?

Like our human brain, we can first learn (train) the handwriting 0 and 1. After the traing (and test) accuray is good enough, we only need to study (traing) the hardwriting 2, Instead of cleaning all ...
4
votes
3answers
1k views

Convolutional neural nets and reduction of the layers

I have a very simple question about Conv nets. I understand the whole principle, but only one thing is not well explained on the Internet. If I have a 16 channels image that goes on a convolutional ...
4
votes
2answers
219 views

What is the best approach for writing a program to identify objects in a picture then crop them a specific way?

My works quality control department is responsible for taking pictures of our products at various phases through our QC process and currently the process goes: Take picture of product Crop the ...
3
votes
0answers
34 views

Image classification - Need method to classify “unknown” objects as “trash” (3D objects)

We have an image classifier that was built using CNN with faster R-CNN and Yolov5. It is designated to run on 3D objects. All of those objects have similar "features" structure, but the ...
3
votes
1answer
149 views

How are exploding numbers in a forward pass of a CNN combated?

Take AlexNet for example: In this case, only the activation function ReLU is used. Due to the fact ReLU cannot be saturated, it instead explodes, like in the following example: Say I have a weight ...
3
votes
2answers
2k views

Is pooling a kind of dropout?

If I got well the idea of dropout, it allows improving the sparsity of the information that comes from one layer to another by setting some weights to zero. On the other hand, pooling, let's say max-...
3
votes
1answer
112 views

How can we derive a Convolution Neural Network from a more generic Graph Neural Network?

Convolution Neural Network (CNNs) operate over strict grid-like structures ($M \times N \times C$ images), whereas Graph Neural Networks (GNNs) can operate over all-flexible graphs, with an undefined ...
3
votes
1answer
399 views

How does backpropagation work on a custom loss function whose components have magnitudes of different orders?

I want to use a custom loss function which is a weighted combination of l1 and DSSIM losses. The DSSIM loss is limited between 0 and 0.5 where as the l1 loss can be orders of magnitude greater and is ...
3
votes
2answers
5k views

How is the depth of a CNN layer determined?

I am looking at a diagram of ZFNet in an attempt to understand how CNNs are designed effectively. I'm working with the CIFAR10 set in pytorch. In the first layer, I understand the depth of 3 (...
2
votes
0answers
68 views

How to decide if gradients are vanishing?

I am trying to debug a convolutional neural network. I am seeing gradients close to zero. How can I decide whether these gradients are vanishing or not? Is there some threshold to decide on vanishing ...
2
votes
2answers
82 views

Is there anything that ensures that convolutional filters end up different from one another?

I found this question very interesting, and this is a follow up on it. Presumably, we'd want all the filters to converge towards some complementary set, where each filter fills as large a niche as ...
2
votes
1answer
42 views

Accuracy scores in a Deep Learning project

I'm using three pre-trained deep learning models to detect vehicles and count from an image data set. The vehicles belong to one of these classes ['car', 'truck', 'motorcycle', 'bus']. So, for a ...
2
votes
1answer
2k views

What are some good alternatives to U-Net for biomedical image segmentation?

Soon I will be working on biomedical image segmentation (microscopy images). There will be a small amount of data (a few dozens at best). Is there a neural network, that can compete with U-Net, in ...