# How do we actually sample an action from a policy in policy gradient methods?

Recently I started to look at policy gradient methods and policies are represented as functions with features for larger problems with many states. Many articles and pseudocodes of algorithms mention sampling an action from the policy, but it is unclear to me how.

Actions are something we do in the environment, like going left, right, etc... And functions take some feature values and parameters, make calculations, and 'spit out' some number. So how do we actually map that number to a certain action, and how do we know what action to take?

• Input features $$x(s)$$ for any state $$s$$
• A fixed action set $$\mathcal{A}$$ of a fixed size (in reinforcement learning (RL) it is often assumed that all actions are legal in all states. In some situations, like board games, we'll filter out illegal actions manually in a postprocessing step).
• A function approximator $$\pi(x)$$ (often a neural network), which takes as input the features $$x(s)$$ of some state $$s$$, and produces as output a vector of probabilities, where the vector has a length equal to the size $$\vert \mathcal{A} \vert$$ of the action space.
Those bolded parts in the last point there are critical for your question. We do not normally have a function that "spits out some number" as you described, we typically have one that "spits out" multiple numbers; one for every action. Often, these numbers can also be directly interpreted as probabilities, because they are forced to all lie in $$[0, 1]$$ and sum up to $$1$$ (this is often done using a softmax layer at the end of a neural network for example).