added 11 characters in body
Source Link
nbro
  • 31.2k
  • 8
  • 66
  • 131

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer

Differences

The neurons of these two types of layers have two key differences. These are that the convolution layers implement:

  • Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
  • Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

Besides these two key differences, there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.

What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.

Sparse connectivity

On the left are two FC neural networks, on. On the right, are layers with sparse connections.   

Weight sharing

On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.


To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers usesuse these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.

However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D convconvolutional layers as well.

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer

Differences

The neurons of these two types of layers have two key differences. These are that the convolution layers implement:

  • Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
  • Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

Besides these two key differences, there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.

What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.

Sparse connectivity

On the left are two FC neural networks, on the right are layers with sparse connections.  

Weight sharing

On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.


To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers uses these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.

However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D conv layers as well.

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer

Differences

The neurons of these two types of layers have two key differences. These are that the convolution layers implement:

  • Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
  • Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

Besides these two key differences, there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.

What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.

Sparse connectivity

On the left are two FC neural networks. On the right, are layers with sparse connections. 

Weight sharing

On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.


To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers use these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.

However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D convolutional layers as well.

Improved clarity to better answer the question.
Source Link
Djib2011
  • 3.1k
  • 3
  • 13
  • 19

tl;dr The neuronsequivalent to a neuron in fullya Fully-connected layers are very similar toConnected (FC) layer is the weights in convolutionalkernel (or filter) of a Convolution layer filters, with

Differences

The neurons of these two types of layers have two key differences. The latterThese are that the convolution layers implement:

  • Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
  • Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

These two are the mainBesides these two key differences within convolutional layers and fully connected ones, however there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.

What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.

Sparse connectivity

On the left are two FC neural networks, on the right are layers with sparse connections.

Weight sharing

On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.


To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers uses these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.

However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D conv layers as well.

The neurons in fully-connected layers are very similar to the weights in convolutional layer filters, with two key differences. The latter implement:

  • Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
  • Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

These two are the main two differences within convolutional layers and fully connected ones, however there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.

What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.

Sparse connectivity

On the left are two FC neural networks, on the right are layers with sparse connections.

Weight sharing

On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.


To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers uses these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.

However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D conv layers as well.

tl;dr The equivalent to a neuron in a Fully-Connected (FC) layer is the kernel (or filter) of a Convolution layer

Differences

The neurons of these two types of layers have two key differences. These are that the convolution layers implement:

  • Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
  • Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

Besides these two key differences, there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.

What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.

Sparse connectivity

On the left are two FC neural networks, on the right are layers with sparse connections.

Weight sharing

On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.


To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers uses these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.

However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D conv layers as well.

Source Link
Djib2011
  • 3.1k
  • 3
  • 13
  • 19

The neurons in fully-connected layers are very similar to the weights in convolutional layer filters, with two key differences. The latter implement:

  • Sparse connectivity, i.e. each neuron is connected only to an area of the input, not the whole.
  • Weight sharing, i.e. similar connections end up having the same weights. This is usually visualized as the same filter traversing the image.

These two are the main two differences within convolutional layers and fully connected ones, however there are some other technical details, e.g. how the biases are implemented. Other than that they perform the same operation.

What causes some confusion is that the input of a CNN is usually 2 or 3-dimensional, while a FC is usually 1-dimensional. These aren't mandatory however. To better help visualize the differences between the two I made a couple of figures illustrating the differences between a conv-layer and a FC one, both in 1D.

Sparse connectivity

On the left are two FC neural networks, on the right are layers with sparse connections.

Weight sharing

On the left is a sparsely connected network. The colors represent the different values of the weights. On the right is the same network with weight-sharing. Note that similar weights (i.e. arrows with the same direction in each layer) have the same value.


To answer your other questions:

Are filters not the same in the way that they convert an "image" to a new "image" based on the weights that are in that filter? And that the next layers uses these new "images"?

Yes, if the input of a convolution layer is an image, so will the output. The next layer will also operate on an image.

However, I'd like to note that not all convolution layers accept images as their inputs. There are 1D and 3D conv layers as well.