Skip to main content
edited body
Source Link
hanugm
  • 4k
  • 3
  • 28
  • 59

I often see the terms episode, trajectory, and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?

In the following paragraphs, I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.

I think episode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of $(s, a, r)$ sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.

With trajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.

I think rollout is somewhere in between, since I commonly see it used to refer to a sampled sequence of $(s, a, r) $from$(s, a, r)$ from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.

I often see the terms episode, trajectory and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?

In the following paragraphs, I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.

I think episode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of $(s, a, r)$ sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.

With trajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.

I think rollout is somewhere in between, since I commonly see it used to refer to a sampled sequence of $(s, a, r) $from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.

I often see the terms episode, trajectory, and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?

In the following paragraphs, I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.

I think episode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of $(s, a, r)$ sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.

With trajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.

I think rollout is somewhere in between since I commonly see it used to refer to a sampled sequence of $(s, a, r)$ from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.

Bumped by Community user
added 1 character in body; edited tags; edited title
Source Link
nbro
  • 41.4k
  • 12
  • 114
  • 205

What is the difference between an episode, a trajectory and a rollout in reinforcement learning?

I often see the terms episode, trajectory and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?

In the following paragraphs, I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.

I think episodeepisode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of (s, a, r)$(s, a, r)$ sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.

With trajectorytrajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.

I think rolloutrollout is somewhere in between, since I commonly see it used to refer to a sampled sequence of (s, a, r) from$(s, a, r) $from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.

difference between episode, trajectory and rollout in reinforcement learning

I often see the terms episode, trajectory and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?

In the following paragraphs I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.

I think episode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of (s, a, r) sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.

With trajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.

I think rollout is somewhere in between, since I commonly see it used to refer to a sampled sequence of (s, a, r) from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.

What is the difference between an episode, a trajectory and a rollout?

I often see the terms episode, trajectory and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?

In the following paragraphs, I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.

I think episode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of $(s, a, r)$ sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.

With trajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.

I think rollout is somewhere in between, since I commonly see it used to refer to a sampled sequence of $(s, a, r) $from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.

Source Link
Paula Vega
  • 438
  • 4
  • 9

difference between episode, trajectory and rollout in reinforcement learning

I often see the terms episode, trajectory and rollout to refer to basically the same thing, a list of (state, action, rewards). Are there any concrete differences between the terms or can they be used interchangeably?

In the following paragraphs I'll summarize my current slightly vague understanding of the terms. Please point any inaccuracy or missing details in my definitions.

I think episode has a more specific definition in that it begins with an initial state and finishes with a terminal state, where the definition of whether or not a state is initial or terminal is given by the definition of the MDP. Also, I understand an episode as a sequence of (s, a, r) sampled by interacting with the environment following a particular policy, so it should have a non-zero probability of occurring in the exact same order.

With trajectory, the meaning is not as clear to me, but I believe a trajectory could represent only part of an episode and maybe the tuples could also be in an arbitrary order; even if getting such sequence by interacting with the environment has zero probability, it'd be ok, because we could say that such trajectory has zero probability of occurring.

I think rollout is somewhere in between, since I commonly see it used to refer to a sampled sequence of (s, a, r) from interacting with the environment under a given policy, but it might be only a segment of the episode, or even a segment of a continuing task, where it doesn't even make sense to talk about episodes.