I am following the tutorial in this video: https://youtu.be/cO5g5qLrLSo which implements deep reinforcement learning (DQN) to balance cart pole in OpenAI default environment.
The DQN model looks like as follows:
model = Sequential() model.add(Flatten(input_shape=(1,states))) model.add(Dense(24, activation='relu')) model.add(Dense(24, activation='relu')) model.add(Dense(actions, activation='linear'))
The implemented code is also available on Google Collabs for you to run/test here: https://colab.research.google.com/drive/1oQILItVu6Y8jOCprzwMGzwlztYmVKK-F?usp=sharing
I do understand the concept and mathematics behind using "linear" vs "non-linear" (softmax) activation function in the output layer.
But, what I am struggling to understand is that why in this target application linear activation function is used in the output layer instead of softmax? Can someone specify how to realize that which type of activation function will be the best for what type of target application using DQN?
P.S. I tried to change the activation function to softmax instead and got completely different result. Hence, I am confused as to why changing the activation function in the output layer could generate completely opposite results.