Can PDDL be utilized for action recognition?

Question

The Planning Domain Definition Language (PDDL) is known for its capabilities of symbolic planning in the state space. A solver will find a sequence of steps to bring the system from a start state to the goal state. A common example of this is the monkey-and-banana problem. At first, the monkey sits on the ground and, after doing some actions in the scene, the monkey will have reached the banana.

The way a PDDL planner works is by analyzing the preconditions and effects of each primitive action. This will answer the question of what happens if a certain action is executed.

However, will a PDDL domain description work the other way around as well, not for planning, but for action recognition?

I've searched in the literature to get an answer, but all the papers I've found are describing PDDL only as a planning paradigm.

My idea is to use the given precondition and effects as a parser to identify what the monkey is doing and not what he should do. That means, in the example, the robot ape knows by itself how to reach the banana and the AI system has to monitor the actions. The task is to identify a PDDL action that fits the action by the monkey.

Answer 1

I don't know of any work on this with respect to PDDL, but this is very similar to a conceptual dependency application called SAM (Script Applier Mechanism). Conceptual Dependency (CD) models actions using a number of primitives (which could be seen as equivalent to PDDL primitive actions): PTRANS for physical transfer, PROPEL for application of a physical force to an object, GRASP for grasping an object, etc. Their number varies around 12 or so, depending on the version of CD.

Stories are described by a sequence of primitive acts, which have slots for actor, object, etc. They are supposed to enable a program to draw inferences about what happens. A common problem when trying to understand stories is that often common knowledge about a situation is omitted; the standard example here is going to a restaurant. It is usually assumed that the listener/reader knows what commonly happens when the protagonist enters a restaurant, so that when they leave without paying this is recognised as something unusual.

The approach used to solve this problem is to encode such knowledge in scripts, sequences of primitive acts. When triggered, eg by "Manuel went to a restaurant", this script is retrieved, and the following actions are looked for in the script. Anything that is recognised is used to fill gaps in the story, eg sitting down at a table, or looking at a menu. This was the task of the SAM program.

Basically you have a sequence of primitive actions, and you try to recognise a more abstract event "going to a restaurant" from that. Obviously you'd need to have a script to recognise, but one could presumably use this to derive a sequence of more generalised events, such as "retrieving an object from a high place", or "standing on top on another object".

The theory of using scripts, plans, and goals to describe human reasoning is detailed in Schank, Roger; Abelson, Robert P. (1977). Scripts, plans, goals and understanding: An inquiry into human knowledge structures. New Jersey: Erlbaum. ISBN 0-470-99033-3.

Answer 2

This is theoretically possible for an exhaustive set of sequential, non-concurrent primitive actions:

$$ \forall s_1, s_2 \left( \exists a \left( \text{possible}(a, s_1) \land \text{do}(a, s_1, s_2) \right) \right) $$

where $s_1$ is the prior situation, $s_2$ is the result of doing action $a$ in $s_2$, and $\text{possible}(a, s_1)$ is true if it is possible to do an action in a situation (See Situation Calculus).

So, for a given situation, reduce your search space to those actions that are possible, then reduce your search space again to those that result in the following situation.