Episodes are discrete, there is no need for calculus. Your "sample efficiency" metric is:
The quantity you are measuring per episode is the return (undiscounted). The sum of this over many episodes does not measure sample efficiency as the term is usually meant, although the sample efficiency of the algorithm you use should impact the numbers you see. Getting a high value of this metric, averaged over many training runs, implies two things:
The algorithm learns to exploit the environment quickly. This is related to sample efficiency.
The algorithm does not pay a high cost for exploring. This is not directly related to sample efficiency, and may in fact be in conflict with learning an environment quickly.
These are both desirable properties of a reinforcement learning algorithm. They often need to be considered in balance, this is the exploration versus exploitation dilemma in RL, which can be studied in a simplified form in mult-arm bandit problems. You may be able to take inspiration from how bandit algorithms are measured for other metrics related to efficient learning.
Your metric is most useful when considering learning agents run in a live environment, where costs of mistakes during learning are real.
If instead you are training an agent in a safe environment - e.g. in simulation - then you may not be interested in the undiscounted returns $R_x$ during training. Your goal may be to train the agent using the least CPU time, the least number of simulation runs etc. In which case, you care most about the mean return achievable by the trained algorithm after spending a certain amount of whatever resource you are managing. This is more closely related to the concept of sample efficiency, and to measure that you could plot the returns from separate test episodes at routine intervals, with exploration removed (e.g. if you were using DQN, then with $\epsilon = 0$