WebTrader: Discovery and Programmed Access to Web-Based Services
Venu Vasudevan 1 and Tom Bannon 2
{venu, bannon}@objs.com
1 Object Services and Consulting Inc., 1942 E. Todd
Drive, Tempe, AZ 85283
2 Object Services and Consulting Inc., 5111 Purdue Ave.,
Dallas, TX 75209
Abstract
As an increasing number services become web-enabled, the
web needs an infrastructure to automate the discovery of and access to
web services. Without this infrastructure, we are limited to accessing
services in isolation via interactive front ends. An effective service
discovery infrastructure will enable the construction of applications that
are dynamic ecologies of web components. Platform specific traders have
been implemented in AI and distributed object architectures, but are not
capable of catering to the diversity of web service implementations. This
paper describes the design and implementation of WebTrader, a trader that
handles the heterogeneity, scalability and evolution in a web-based service
market.
Keywords
web searching; agents; service discovery; web automation; trader; crawler;
metadata
Rationale
The web is seeing increased use as a platform for distributed applications.
These applications are composed of programs that access a variety of web-enabled
services. Since the web has no centralized administrative authority, services
may come and go, and their implementations, access interfaces and properties
may change continually. Constructing stable applications over an evolving
service market requires a dynamic application architecture, where services
can be discovered and bound to at runtime, and where bindings can be dynamically
created and modified. Motivating applications for such an architecture
include comparison shoppers and other kinds of
personal
agents . One way to achieve this is to create all the bindings indirectly,
by querying a service location service (aka trader). The application
can then change its bindings by re-executing its trader queries, querying
a different trader, or modifying the binding queries to change the overall
quality of service. In the personal navigator , for
instance, a mobile application "recontexts" itself to the local environment
by talking to the trader (and therefore the services) in its immediate
vicinity. This allows the application to reside on a thin-client, as it
uses the available services in its neighborhood for a good part of the
application logic.
Keyword-based document search engines cater to human retrieval of unstructured
content, and do not directly address the trading problem of accessing and
binding to structured services. Trader proposals in OMG and AI architectures
do not handle the variety of web-based service delivery mechanisms, and
do not use web protocols and interchange mechanisms. This paper describes
WebTrader,
an XML-based trader that supports web service location, and exploits web
search engines for scalable trading. Related papers discuss WebTrader
architecture and implementation and the use of composition
in flexible trading.
Implementation
The WebTrader follows the canonical trader architecture
in providing an interchange format for services to advertise their capabilities,
an offer repository to store collections of service advertisement, and
a matchmaker to match client queries to service advertisements. To cater
to web services and clients, WebTrader uses XML as the encoding format
for client queries and service advertisements, and uses document search
engines as a scalable and widely available offer repository.
Web-based services (and information sources) export their capabilities
as an XML service advertisement. The XML DTD for the advertisement
has sections for describing the interface,
metadata (properties)
and searchKeywords. The interface section describes the operational
interface of the service, including its access protocol(s) and access methods.
The metadata section allows sets of instance-specific service properties
to be specified as name-value pairs. Typical service properties may include
the geographic location of a network gateway service, the fidelity and
accuracy of an interactive map service, the service cost and payment options
of a stock advisory service, and the languages supported by a text-to-speech
translation service. Services make themselves known to WebTrader
instances by publishing a service advertisement page (SAP) containing
one or more service advertisements, to search engines. The searchKeywords
section of a service advertisement provides keywords that are used in indexing
the SAP (perhaps are HTML META tags). WebTraders query search engines for
SAPs, and provide matchmaking services over the service advertisements
contained in the SAPs.
A client queries the WebTrader by advertising its needs in a client
advertisement. The XML structure of a client advertisement is very similar
to that of a service advertisement, with the difference being in the interpretation.
Metadata in the client advertisement states what the client needs as type
and property predicates. Type predicates indicate the interface requirements
that a matching service instance must satisfy, and property predicates
specify hard and soft requirements on the properties of a matching service
instance. A client queries the WebTrader by publishing the client advertisement
to the WebTrader, and receiving a page of matching bindings in return.
This binding data provides all the information a client needs to bind to
a matching service.
Multiple traders can cooperate by sharing service advertisements via
a search engine (indirect federation), or by advertising themselves as
services that can be used by other WebTraders (direct federation). In indirect
trader federation, a search engine operates as the common advertisement
repository that is shared by multiple traders. In direct federation, a
trader that is unable to find a satisfactory match, uses the trading mechanism
to find (and bind to) other WebTraders. Since a WebTrader service
is itself a service, direct trader federation uses standard trading mechanisms.
An orthogonal problem to the multiplicity of traders is the multiplicity
of global, semi-local and local search engines that occupy the web. This
means that traders accessing one search engine do not have access to the
advertisements that are published to other search engines.
DeepSearch
(an
application built by the second author) could be used to address this problem.
DeepSearch presents a single search engine view over multiple
search engines, by allowing search engines to advertise themselves as services
to other search engines.
Use
WebTrader is being used in DARPA's Control of Agent-Based Systems
(CoABS) program, as part of an evacuation scenario demonstration. Efficiently
evacuating civilians from a country in regional conflict requires tracing
down and contacting the individuals, and executing their evacuation in
a planned manner. Much of the information for such an operation such as
the consulate "white pages" of civilians in the country, the addresses
of the hotels they are staying in, a map of the city and locations of these
hotels, are accessible as web-based services. The WebTrader is used by
the agent computation to locate and access these services in a coordinated
manner.
References
[Etzi97] Etzioni,O., A scalable comparison-shopping
agent for the World Wide Web, In Proceedings of Autonomous Agents '97
[Hode97] Hodes,T.D. et al., Composable
Ad hoc Mobile Services for Universal Interaction, UC Berkeley BMRC
Technical Report, 1997
[Vasu98a] Vasudevan,V., A
Reference Model for Trader-Based Distributed System Architectures,
OBJS
Technical Report, 1998
[Vasu98b]Vasudevan, V., Trading-Based
Composition for Component-Based Systems, Workshop on Compositional
Software Architectures, 1998
[Vasu99] Vasudevan,V. and Bannon, T., WebTrader:
Discovery and Programmed Access to Web-Based Services, OBJS Technical
Report, 1999