# How to formulate discounted return in cartpole?

I am trying to formulate a problem that aims to prolong the lifetime of the simulation, the same as the Cartpole problem. I aware that there are two types of return:

• finite horizon undiscounted return (used for episodic problems)

$$G = \sum_{t=0}^T R_t$$

• infinite horizon discounted return (used for non-episodic problems).

$$G = \sum_{t=0}^\infty \gamma^t R_t$$

However, I'm confusing that "Is Cartpole episodic task?". Ideally, the simulation lasts forever. This is my final objective (prolonging the lifetime). But it still has some termination states. Should I introduce the termination state and use it with a discounted return like:

$$G = \sum_{t=0}^T \gamma^t R_t$$

It's not true that finite horizon MDP can't use a discount factor. Any discount factor $$\gamma \leq 1$$ is fine for finite horizon.
Whether or not cartpole is a finite or infinite horizon depends on the environment implementation. The default environment in openai gym uses $$T=200$$ as the horizon. This is customizable, and most papers I see which use cartpole as a test environment use $$T=1000$$ instead. The task is considered "solved" when the agent can reach $$T$$ timesteps consistently (e.g., the last 100 episodes have all reached $$T$$ timesteps).
If you want to treat cartpole as an infinite horizon task, that's fine too. Clearly you need $$\gamma < 1$$ here, and you can just $$T$$ to some arbitrary large number.