Hello I am new to neural network and I have a question about activation functions. People usually use their activation function such as relu, sigmoid, tanh etc. But what happens when I mix activation functions? I recently found that Google has developed Swish activation function which is (x*sigmoid). By altering activation function can it increase accuracy on small neural network problem such as xor problem?

There are several activation functions, such as ReLU, sigmoid or $\tanh$. What happens when I mix activation functions?

I recently found that Google has developed Swish activation function which is (x*sigmoid). By altering activation function can it increase accuracy on small neural network problem such as XOR problem?

Hello I am new to neural network and I have a question about activation functions. People usually use their activation function such as relu, sigmoid, tanh etc. But what happens when I mix activation functions? I recently found that Google has developed Swish activation function which is (xsigmoid). For instance if I use activation function as (sigmoid+tanh) or (sigmoidtanh) or (0.5sigmoid+0.5tanh) does it have any impact on accuracy or efficiency? By altering activation function can it increase accuracy on small neural network problem such as xor problem?

Hello I am new to neural network and I have a question about activation functions. People usually use their activation function such as relu, sigmoid, tanh etc. But what happens when I mix activation functions? I recently found that Google has developed Swish activation function which is (x*sigmoid). By altering activation function can it increase accuracy on small neural network problem such as xor problem?