2
$\begingroup$

I am learning about discretization of state space when applying reinforcement learning to continuous state space. In this video the instructor at 2:02 the instructor says that one benefit of this approach (radial basis functions over tile coding) is that "it drastically reduces the number of features". I am not able to deduce this in the case of a simple 2D continuous state space.

Suppose we are dealing with 2D continuous state space, so any state is a pair $(x,y)$ in the Cartesian space. If we use Tile Coding and select $n$ tiles, the resulting encoding will have $2n$ features, consisting of $n$ discrete valued pairs $(u_1, v_1) \dots (u_n, v_n)$ representing the approximate position of $(x,y)$ in the frames of the $n$ 2-D tiles. If instead we use $m$ 2-D circles and encode using the distance of $(x,y)$ from the center of each circle, we have $m$ (continuous) features.

Is there a reason to assume that $m < 2n$?

Furthermore, the $m$-dimensional feature vector will again need discretization, so it is unclear to me how this approach uses fewer features.

$\endgroup$

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Browse other questions tagged or ask your own question.