I have a multi-label classification task I am implementing. I have done a hyper-parameter tuning to determine the best configuration for my neural network.

Is it valid to do this (determine the best hyper-parameters) and then do a cross validation to get a more accurate test estimation of the dataset? I don't see how this would be invalid as cross validation examples I have seen already have network architectures known a-priori, presumably because this is what they chose or feel is the best way of proceeding.

thank you

  • $\begingroup$ How are you doing your cross validation? Do you have three sets (a train, test and validation set)? Which set are you using to choose hyperparameters? $\endgroup$
    – htl
    Jun 28 at 12:54
  • $\begingroup$ I am doing cross validation using sklearn. input data, one hot encoded multilabel ([0,1,0,01,0] etc - per sample) then standard kfold as follows: kfold = KFold(n_splits=10, shuffle=True, random_state=42) and then - for train, test in kfold.split(inputs, one_hot): etc keras tuner is used on the original input and one_hot but a standard 80/20% train test split is used - So for CV I am using primarily for performance measure but of course with kfolds to consider all data... $\endgroup$ Jun 28 at 12:58

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.