There is no major difference here between a POMDP and MDP. When setting reward values, you are generally trying to give the minimal information to the agent that when the sum of rewards is maximised, it solves the problem that you are posing.
In literature it is common to use one simple number as a reward, but I am not sure if this is really how you define a function. Because this way you have do define a reward for every possible Action-State combination.
Some defined value of reward has to be returned after all state, action pairs taken in the environment. The value could be $0$ of course.
Reward can depend on, or be a function of, current state, action, next state and a random factor. In a POMDP it may also be from any unobserved factor in the environment (you might know this in a simulation because you have created the environment and are choosing not to share the data with the agent).
In practice, the reward often does not have to depend on all the possible factors. In addition, the relationship between the factors and the possible reward can be very simple or sparse.
Classic examples you may find in the literature include:
Reward in a game can be simply $+1$ for winning, or $-1$ for losing, granted at the end. All other rewards are $0$
If your goal is to reach a certain state, such as exit from a maze, in minimum time, then a fixed reward of $-1$ per time step is enough to express the need to minimize the total number of steps.
For a goal of maintaining stability and avoiding a failure in e.g. CartPole, then a reward of +1 per time step without failure suffices.
All of these examples express the reward function as a simple condition plus one or two numbers. The key thing is that they allow you to calculate a suitable reward on each and every time step. They are all strictly reward functions - the general case can cover very complex functions if you wish, that will depend entirely on the goals you want to set the agent to learn/solve.