I want to initialize a parameter, which is a single real number in my model. If you want the role of the parameter in the model, you can assume it as the parameter to multiply with the output of the neural network and the resultant product will be the final output.

Does initialization value matter? If yes, are there any guidelines on initializing the single parameter?

  • $\begingroup$ Initialization always matters. There's countless papers on parameter initialization, however a common method (last I checked) is to use Kaiming initialization. In this case since it's only one parameter it should be fairly simple to initialize using this method $\endgroup$
    – Recessive
    Dec 8 '21 at 2:25
  • $\begingroup$ In this case, the most suitable initialization may also depend on the specific role of this parameter. You say you multiply it with the output of the model, but why are you doing this? Maybe you can provide more details about that. $\endgroup$
    – nbro
    Dec 10 '21 at 9:03
  • $\begingroup$ Ha @nbro I will try to do. $\endgroup$
    – hanugm
    Dec 11 '21 at 2:44

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.