I am comparing my coded TD3 (Twin-Delayed DDPG) and the same TD3 (same hyperparameters) but with Priority Replay Buffer instead of a normal Replay Buffer.
From what I have read, PER (Priority Experience Replay, Priority Replay Buffer) aims to improve sample efficiency. But how do I measure or quantify sample efficiency on these two? Is it who gets the highest average reward in a given number of episodes? Does it have something to do with the batch size?