Doing What You're Told: Following Task Instructions in Changing, but Hospitable Environments

The AnimNL project (Anim ation from N atural L anguage) has as its goal the automatic creation of animated task simulations from natural-language instructions. The question addressed in this paper is how agents can perform tasks in environments about which they have only partial relevant knowledge. The solution we describe involves enabling such agents to * develop expectations through instruction understanding and plan inference, and use those expectations in deciding how to act; * exploit generalized abilities in order to deal with novel geometric situations. The AnimNL project builds on an animation system, Jack™, that has been developed at the Computer Graphics Research Lab at the University of Pennsylvania, and draws upon a range of recent work in Natural Language semantics, planning and plan inference, philosophical studies of intention, reasoning about knowledge and action, and subsumption architectures for autonomous agents.