What are all the possible usages of 'multilayer perceptron'?

Question

The term 'multilayer perceptron' has been used in literature in various ways in the literature.

I am presenting some of them below

1. As a feed-forward neural network [1].

2. As a fully connected feed-forward neural network [2].

3. As a fully connected feed-forward neural network in which each hidden layer has the same number of neurons.

4. As a fully connected feed-forward neural network in which each hidden layer has the same number of neurons and same activation function.

Afaik, the first definition is generally used, but it seems that there are many alternative definitions.

In this context, I want to know all the possible definitions that are floating in the literature for the word 'multilayer perception'.

I am asking this question because there can be several interpretations if we consider the words of 'multilayer perception' alone as the names suggests the only property required is multiple number of layers.

Answer 1

It is a series of weight matrices (possibly with a bias) separated by a non-linear activation function. So one layer can be described as this act_f(Wx+b). Where W is the weight matrix, b is the bias, x the input of the layer and act_f is the activation function. This is function is then repeated (taking in the output of the previous layer as x) possibly with different W, b and act_f's and this is repeated n times where n > 1 (as it's a multilayer preceptron).

This is an perceptron : act_f(Wx+b)
This is an possible example of a multilayer perceptron act_f(W*act_f(Wx+b)+b).

What kind of shape the W has doesn't really matter, you could even have skip connections would still be a mlp. It's just a word used to describe this type of architecture, just like we use the word cat to describe a cat while there are many different types of cats.

Answer 2

what you're getting at is a major tension in machine learning: there are sometimes different ways of defining the same mathematical concept, and different mathematical definitions for the same term. In this case context is key. In a class on machine learning it will refer to a feed forward network, in some conference communities it will refer to a linear layer. In either case, the math in @hal9000's answer will describe it technically. Though at times they may simplify the notation even further and describe it as a simple weighing e.g.

$$y = \sigma(Wx)$$