I have a problem about meta pseudo labeling, I want to select the most significant pseudo-labels that minimize validation loss. Let's say i initialize a set of pseudo label denoted $Y_{pseudo}$, then i perform parameter update by gradient descent
$\theta_{t+1} = \theta_{t}-\alpha\nabla L(f(\theta_{t}),Y_{pseudo})$
I want to select a subset of $Y_{pseudo}$ that actually minimizes validation loss apart from those that increase the validation loss.A naive approach is that I have to brute force to remove a sample at a time and calculate validation loss to update parameter $\theta_{t+1}$ and calculate $L(f(\theta_{t+1}(Y_{pseudo}),Y_{val})$ until there is no sample that increases the loss. I try to figure out what way we can remove unwanted samples just by update parameters once, what's in my mind is that i can select sample that have the gradient $\frac{\partial L(f(\theta_{t+1}(Y_{pseudo})),Y_{val})}{\partial Y_{pseudo}} \le 0$ in that way i know that the loss function of some pseudo-label is going down so i can select them but in the case $\frac{\partial L(f(\theta_{t+1}(Y_{pseudo})),Y_{val})}{\partial Y_{pseudo}} \le 0$, it can be minimum or maximum value of loss function, then do i have to use hessian matrix to determine its direction? Anyone knows about this, please help me! or if you have other way to select by calculating the validation loss only once, feel free to share with me, I really appreciate it.