Questions tagged [activation-function]

For questions related to the selection of and theory behind specific activation functions used in artificial networks.

15 questions with no upvoted or accepted answers
Filter by
Sorted by
Tagged with
3
votes
0answers
38 views

Are there any commonly used discontinuous activation functions?

Are there any commonly used activation functions (e.g. that take values in $(0,.5)\cup (.5,1)$)? Preferably for classification? Why? I was looking for commonly used activation functions on Google, ...
2
votes
0answers
28 views

Which activation functions should I use for polynomial regression?

I am a beginner in machine learning and neural networks. I have only used neural networks for classification problems. My aim is to modify it so that it can work for polynomial regression as well. In ...
2
votes
0answers
22 views

Single label classification into hierarchical categories using a neural network

I am working on a classification problem into progressive classes. In other words, there is some hierarchy of categories in such a way, that A < B < C, e.g. low, medium, high, very high. What ...
2
votes
1answer
72 views

How to choose the activation function in neuroevolution?

I am developing a NEAT flappy bird game, and it doesn't work, the system stays stupid for 300 generations. I chose tanh() for activation, just because it's included in JS. I can't find a good ...
2
votes
0answers
14 views

Language Learning feedback with AI

Is there a program under development that uses AI technology, like Siri, to "hold hands" so to speak with a language learner and coach them on accent, colloqiual expressions, or to let them guide the ...
2
votes
0answers
121 views

Non-monotonic activation function and XOR problem with perceptron

I already post the question on Stackoverflow : https://stackoverflow.com/questions/53785922/is-the-use-of-a-non-monotonic-activation-function-is-a-correct-and-viable-solut?noredirect=1#...
1
vote
0answers
22 views

If the output of a model is a ridge function, what should the activation functions at all the nodes be?

I have the following assignment. I can't understand the b part of this question in my assignment. I have completed the 1st part and understand the maths behind it, but the 2nd part has me stumped. I ...
1
vote
0answers
14 views

Poisson regression via neural network

The first neural net I wrote was a classifier. After that, I learned that neural nets can be used for regression tasks, even quantile regression. It has become clear to me that the usual games with ...
1
vote
0answers
19 views

Can SqueezeNet be used for regression?

I want a model that outputs the pixel coordinates of the tip of my forefinger, and whether it's touching something or not. Those would be 3 output neurons: 2 for the X-Y coordinates and 1, with a ...
1
vote
0answers
28 views

Are PreLU and Leaky ReLU better than ReLU in the case of noisy labels?

Let's assume I want to build a semantic segmentation algorithm, based on Multires-UNET. My GT-masks are messy and generated by a GAN, but they are getting better and better over time. The goal is ...
1
vote
0answers
26 views

Which activation functions can lead to the vanishing gradient problem?

From this video tutorial Vanishing Gradient Tutorial, the sigmoid function and the hyperbolic tangent can produce the vanishing gradient problem. What other activation functions can lead to the ...
1
vote
0answers
25 views

What are the pros and cons of the common activation functions?

I have heard that sigmoid activation functions should not be used on neural networks with many hidden layers as the gradients tend to vanish in deep networks. When should each of the common ...
1
vote
0answers
24 views

What activation functions are better for what problems?

I’ve been reading about neural network architectures. In certain cases, people say that the sigmoid "more accurately reflects real-life" and, in other cases, functions like hard limits reflect "the ...
1
vote
0answers
28 views

Activation Function of the Last Layer

I was wondering whether there is a method (not a table of recommendations) that could tell me what activation function to choose if the outputs of the neural network have some interpretation . For ...
0
votes
0answers
131 views

Selu and how to implement it?

I just came accros the "Selu" function. It looks promising, especially with the header "self normalizing neural network", since I have some serious with exploding gradient within my network with (...