Ultimately, a policy must be such that is is possible for an agent to execute it.
If the policy depends on the state, the implicit assumption is that the agent has knowledge of the state and can therefore choose its actions accordingly. This is the common case of a MDP as an underlying framework for RL.
If the state is not known to the agent, it may instead perceive observations that have some relation to the state (although they might not fully reveal the state). Then, one can condition policies on the last received observation.
However, it is useful to note that conditioning on the last observation in a partially observable setting is in general not sufficient for acting optimally. There are cases where one may need to remember a longer history of observations to decide which action is best. In general, acting optimally in such a partially observable setting requires that the policy is a function of the complete history of past actions taken and observations perceived. The underlying framework is then the partially observable MDP, or POMDP.