PPT Slide
Problem. Agent-based command and control systems offer real promise in flexibility and adaptability but it is not yet clear how they can be understood and controlled. It would be desirable if we could just talk to systems of agents - but how?
About Menu-based Natural Language Interfaces (MBNLI). Try typing or speaking to a system that has a conventional type-in or speech natural language interface (NLI). Most of your questions and commands will not be understood because they overshoot the capabilities of the NLI system or the underlying application it interfaces to. In addition, you won’t really know about some things you could ask (map or statistical queries?) so your questions and commands will also undershoot the capabilities of the NLI system. Menu-based Natural Language Interface (MBNLI) technology uses standard NLI technology but in a menu-directed completion-based way to restrict the language and guide the user to just the capabilities of the NLI and underlying system. The technology fills a large unfilled niche in user interface design enabling unskilled users to make complex queries and commands.
Objective. The AgentGram project addresses human-to-agent communication via agents that understand constrained MBNLI languages. Humans can task and query one or more agents using complex but understandable commands. This technology can mix pervasively into all applications, both on the desktop and the Web, providing NLI capabilities to not just agents but also information sources, services, and generally any accessible resource.
Approach. Like other NL technology, MBNLI technology uses attribute grammars and a predictive parser. The difference is in using the grammars to predict legal sentence completions and displaying these using menus. MBNLI, using a client-server parser farm architecture, supports (a) web-based access, (b) multiple users, (c) multiple simultaenous grammars, (d) speech control via Java Speech Markup Language, (e) the ability to generate MBNLI interfaces to DBMS system given the DBMS schema, (f) a partitioning of MBNLI into client and stateless server so a very thin client can be downloaded (with no installation) only requiring Javascript, (g) LL descriptors on the web, discoverable by traders, and (h) techniques for composing MBNLI grammars and interfaces. The AgentGram prototype explores the idea of attaching NL wrappers to agents to make it possible for people to query or task them. Sub-grammars of individual agents can be traversed on-the-fly to construct commands or queries which span several agents. The web prototype shows how to make MBNLI technology pervasive on the web.
Demonstrations. In the NEO TIE, the MBNLI component is agentized as an SRI OAA user interface agent (part of the SRI Multi-Modal Map) that translates natural language queries to SQL to be executed by an information access agent (ISI Ariadne). In OBJS demos shown at the Science Fair, MBNLI was used to query the DBMS of Enviro Conference attendees and also to visualize the XML-based grid log. The AgentGram demo finds MBNLI-enabled agents using WebTrader and shows how the sub-grammars of one or more agents can be traversed to produce commands and queries. Our CoABSGrid demo illustrates one example of how our MBNLIGridAgent can be used to integrate MBNLI technology into the grid. In the demo the user is able to interact with the MBNLI web browser interface while the MBNLIGridAgent is able to provide current status regarding the interaction.
Plans. In the near term, we need to (a) port to IE, (b) re-engineer client-server MBNLI so one parser server can service (now 10-20, in the future 100-1000 simultaneous) MBNLI clients, (c) build grammar translations to FIPA-based Agent Communication Language, (d) interface AgentGram to eGents and Smart Data Channels. Longer term, (e) we want to interface AgentGram to agents and data sources in-the-wild (on the open Web and in DoD) and (f) add support for roles, dialog and context models.
Technology Transition. A MBNLI agent was used in the Science Fair NEO application coupled to SRI OAA and ISI Ariadne. We are starting to look for early adopter DoD and commercialization partners for this technology: USMTF and Intelink databases appear to be good DoD targets.
System Requirements. MBNLI web client requires Netscape browser; server requires WinNT, MySQL, ODBC, a cgi/php-capable web server.