I am currently implementing a Pointer Network to solve a simple Knapsack Problem. However, I am a bit puzzled over the correct (or common, or "best") way to give the agent the option to stop taking the item (terminate episode). Currently, I have done it in 2 ways, adding raw dummy features or adding encoded dummy features (dummy features are all zeros). If the agent selects the dummy item, then the agent will stop taking the item anymore and the episode is terminated.
I trained both methods for 500K episodes and evaluated their performance on a single predefined test case in each episode, after adding the gradient. I found that concatenating dummy features with the encoded features yielded a higher score earlier, but also scored 0 very often. On the other hand, adding the dummy features to the raw features learned to maximize the score very slowly. Therefore, my questions are:
Is adding the raw dummy features make learning slower because of additional encoding layer learning?
What is the most correct (or common or arguably best) way to give the agent the option to terminate the episode (in this case stop taking item)?