Is there a notion of exploration-exploitation tradeoff in dynamic programming (or model-based RL)?
I think there is an implicit notion of it in dynamic programming; say, if you have to make some sort of search over a subset of a state space and you are deciding whether to use BFS, breath first search, or DFS, depth first search, you are at least implicitly thinking on the best way to explore/exploit the state space.
As for model based RL, yes. There is explicit algorithms that mediate exploration and exploitation. One of them is UCB, uper confidence bound. One of the best examples of a model based reinforcement learning algorithm is AlphaGo. The algorithm uses a variation of UCB to explore the state space.