Object Services Architectures
Trader Service
Project Summary
Venu Vasudevan
and Tom Bannon
Object Services and Consulting
Inc.
September 15, 1998
Executive Summary
We are seeing tremendous growth in web-enabled services, without any support
in the web infrastructure for service location and service composition.
The dynamic
repository federation problem of OSA/Annotations and the dynamic
behavior location problem in OSA/ISA are two problems that demonstrate
the need for such an infrastructure within the OSA/Scaling context. In
the former, an annotation system needs to dynamically discover and access
annotation repositories (i.e. internet-accessible repositories that contain
value-added annotation information) . In the latter, intermediaries that
augment a web interaction with one or more mediating services, must dynamically
locate the internet component that provides this service (e.g. applet,
CGI script, servlet). Traders that have been proposed in other frameworks
to address the dynamic service location problem. However, they have not
been demonstrated to work with internet services. Furthermore, they do
not address situations where none of the available service instances matches
the client request. We believe such mismatches between client needs
and available service offers can be better handled by a trader that composes
multiple service instances to provide the client-requested service.
Doing so requires a trader that understands service composition.
As part of the OSA/Trader task, we define a design space for trading,
identify the requirements for internet services trading, and motivate the
need for compositional trading. Prototyping efforts demonstrate a native
internet services trader, and the adaptation of an OMG trader to perform
internet services trading. The latter identifies changes required in the
OMG trader specification to support internet services trading.
Motivation and Problem
Web computing and e-commerce are leading to an explosive
growth in web-based interactive services, and application services that
can be accessed by other programs. Location of and access to such
services remains a problem, as the web does not provide a search facility
that is tailored to service location and access. Conventional search engines
work for document searches but are inadequate for service searches. Service
location services (aka traders) have been developed in the CORBA
and Agent frameworks, but work only within the frameworks they were
developed for. A trading capability that caters to web services needs to
be demonstrated. Such a web trading service will provide the end-user with
effective access to interactive web services (e.g. for online shopping),
and can be used by programs (e.g. comparison shopping bots) to provide
higher-level services that take advantage of a smart service location capability.
More generally, the OSA/Trader facilitates a dynamic application architecture,
whereby components of an application can bind to other components at run-time.
Trader-based binding can be used by applications to reconfigure themselves
to changing locations (i.e. application migration) or changing user context.
The dynamic binding capability provide by OSA/Trader is useful as a core
service in several OSA/Scaling project components. The OSA/Annotations
Service and OSA/Network
Performance Monitor Service both could use a trader to locate group
or public caches of metadata related to their service.
Objective
The near-term objective of this task was to implement
a trader for web-based services based on well known trading models. The
longer term objective was to identify an augmented trading model that allows
the trader to assemble more complex bindings that consist of multiple services
and are subject to QoS constraints.
Limitations of Related Work
As part of the preliminary investigation, we surveyed existing trader implementations
and published a report on a A
Reference Model for Trader-Based Distributed Systems Architectures.
This report defined a design space characterizing currently published
trader implementations. The taxonomy revealed that the OMG trader supports
only part of the design space out of the box, and that it might be a heavyweight
solution to the web trading problem. The OMG Trading model's choice of
service description language was IDL, which is suitable for CORBA but not
likely to be a popular choice for web service descriptors. XML provided
equivalent or better capabilities to IDL, and was much more likely to be
accepted by the web community. The OMG trader worked on a push model,
where services (or third-parties) pushed service advertisements
to the trader. A pull scheme is much more likely to succeed in the web
context, whereby robots or spiders scour the web for services, and pull
the advertisements out of web pages or by inspecting the service. The pull
scheme has already proven to be successful in conventional search engines.
The most serious mismatch between the OMG trading model and the needs of
the web was in the federation approach. OMG adopted direct
federation, over repository-based federation. The direct federation
approach requires traders to directly communicate to (and know of) the
traders they federate with. While this is a secure and controlled scheme,
the communication overhead in such a scheme grows proportionally to the
number of federating traders. An internet-scale federation with hundreds
of globally distributed traders is hard to set up if large numbers of communication
paths need to be established to make the federation operational. A more
scalable approach is one where federating traders share service information
via a commonly accessible service repository . Traders wishing
to share a service advertisement would publish the advertisement to the
service repository. Traders wishing to expand the services they trade with
would periodically access this service repository and obtain meta-data
about new services. The communication complexity in repository-based
federation is independent of the number of traders involved in the federation.
From the point of view of a trader in this federation, other traders are
anonymous providers of service advertisements. Since these advertisements
are published to the service repository, a trader can use the service meta-data
gathered by other traders, without directly having to communicate with
them. The challenging problem of supporting a globally available service
repository has already been solved in the internet architecture, as internet
search engines naturally provide such a repository.
Current traders focus on one-to-one matches between client request and
available service instances. This level of matchmaking is inadequate in
a number of situations, as there are format, QoS and performance mismatches
between the client request and the closest matching service instance.
A compositional trader can address this problem by composing such a service
instance with format translators and performance monitors to reduce the
mismatch.
Approach
We decided to pursue two concurrent implementations of web-based
trading to address the above questions. A WebTrader built from the
ground-up using web technologies (Java, XML and search engines) would investigate
the feasibility of building a lightweight, scalable trader of and for the
web. The MWTrader implementation is based on JTrader (an OMG-compliant
trader) would address whether an OMG trader implementation could be modified
to trade with web services, and to support repository-based federation.
It would also provide insight into the reusability of services across frameworks,
and the interoperability problems in using an ORB service in a web
context.
As part of the compositional trader research, we examined work in QoS
and reconfigurable distributed systems for examples of QoS-driven service
composition. Many of these efforts used proprietary architectures, but
provided examples that could be handled by an augmented trader. The
report on Augmenting
OMG traders to handle service composition discusses augmentations
to the OMG trading model that would allow traders to provide continuous
and end-to-end guarantees about the services it trades with.
Results
The initial WebTrader prototyping effort resulted
in an implementation built using Java, XML and search engine technologies.
As part of this prototype, we developed an XML-based service description
format that matches the expressiveness of OMG's service description
format (IDL + properties). The XML format offers the additional advantages
of being ASCII and human readable, and able to deal with multi-portal
services (services that export themselves via CORBA, CGI, RMI and other
portals simultaneously). In contrast, OMG compliant traders only
support CORBA service advertisements. XML service advertisements can also
be embedded in HTML pages, and shared by traders using search engine based
page retrieval. The prototype demonstrates an end-to-end trading capability
where advertisements are published to a search engine (or located by a
crawler/spider), imported (pulled in) by a trader, and provided
as the result of a client query. As mentioned earlier, the pull model
supported by WebTrader allows it to discover new services, without
the services having to advertise themselves to the trader.
Further work on the WebTrader explored recursive trading, whereby
traders are themselves objects (with associated metadata) that are traded
by other traders. Such scaling of the trading concept is important
in an internet scenario with thousands of specialized traders
(e.g. battlefield image trading, a trader that is good at finding the best
travel deals) that will be available on the internet. Recursive trading
either supports trade shipping (i.e. the shipping of a trader query
to other traders) or the inclusion of trader object references in the result
set that is returned to the client. The latter is a scaled down implementation
of the former, with the trade shipping responsibility handled by the client.
The idea of recursive trading is related to the direct trader federation
ideas proposed by OMG, and to the STARTS meta-search architecture proposed
for federating document search engines. Our goal is to produce a
lightweight scheme for dynamic trader federations, in contrast the OMG
scheme which uses heavyweight, highly secure contractual mechanisms
to set up federations. STARTS is geared more towards large-grained document
search engines (e.g. Excite, Yahoo), and is not directly applicable to
recursive trading with large number of finer-grained traders.
MWTrader
The first phase of the MWTrader prototyping effort involved getting
a stripped down JTrader to work with OrbixWeb, and understanding
its code. The code review was instructive in pointing out some quirks in
the OMG Trader Specification that were not obvious from reading it. The
service type descriptions which are part of an ORB's type repository
are also cached in the trader's type repository. The OMG specification
does not provide a rationale for such caching, given that type information
is fairly small, fairly static data and does not benefit greatly from caching
schemes. Also unexplained is the fact that the OMG trading service has
its own notion of properties, and does not use the OMG Property Service.
It is notable that the JTrader query engine (that accepts a client
"matchmaking query") is almost 40% of the trader code and is not ORB-specific.
This indicates that duplication of the OMG trader's constraint language
in a web trader will be a significant task.
The second phase of MWTrader prototyping focussed on enabling it to
: a) operate in pull mode, b) trade with foreign (non-ORB) services,
c) support the "soft" (approximate) matchmaking needed for QoS trading,
and d) support compositional trading. As pointed out in an earlier semi-annual
progress report, given the time constraints on the OSA/Scaling project,
we expected to identify approaches to the above and identify code insertion
points without being able to demonstrate all these ideas in a single prototype.
We prototyped IDLBuilder, an XML-to-IDL translator to demonstrate
that XML advertisements published to a search engine could be imported
by OMG traders in much the same way as the WebTrader. This is an important
architectural point, as it shows how to replace the heavyweight direct
federation mechanism proposed in the OMG trader specification with
repository based federation without being inconsistent with the OMG
specification. We defined an MWTrader API for the registration,
lookup and matchmaking of foreign services (which had a URL as a handle
instead of an ORB reference). While the code and API additions are relatively
small, they require extensions to the OMG Trader data structures to support
foreign handles, and LookupForeign and RegisterForeign APIs.
There will likely be other API additions as well, to allow the OMG trader
to be accessed via the web, and to return matching service advertisements
in a XML or other web-friendly formats.
We identified the interface between the matchmaker component in JTrader
and the rest of the JTrader code where MWTrader would introduce soft matchmaking
and compositional matchmaking function. We concluded that the mechanics
of plugging-in a new matchmaker with either soft or compositional matchmaking
semantics was fairly straightforward. However, the current OMG trader API
needs to be augmented with a soft matchmaking API, as there is no way to
pass distance functions to a trader in the current OMG trader specification.
We did not attempt to prototype a compositional trader, as this required
significant additions to the OMG trading model. Details of these augmented
trading model are covered in Augmenting
OMG traders to handle service composition.
We briefly explored the notion of servicebots, agents that explore
the internet for unadvertised services and extract their service advertisements.
Given that there are thousands of services on the net which nobody has
developed advertisements for, we see a bot that automatically generates
advertisements as a useful adjunct to a trader. Discovering the structure
of web services is a significant research challenge, and an area of active
research in the AI and machine learning communities. Work on AI-based
information agents such as ShopBot (now Jango from
Excite), ILA, BargainFinder, Information Manifold begins to address
service structure discovery for web services, but does not integrate it
with trading. These are technologies that can be used as part of a trader's
servicebot. Building on their work, and relating these ideas
to standards-based traders is a promising future research direction.
This research is sponsored by the Defense Advanced Research
Projects Agency and managed by the U.S. Army Research Laboratory under
contract DAAL01-95-C-0112. The views and conclusions contained in this
document are those of the authors and should not be interpreted as necessarily
representing the official policies, either expressed or implied of the
Defense Advanced Research Projects Agency, U.S. Army Research Laboratory,
or the United States Government.
© Copyright 1998 Object Services and Consulting,
Inc. Permission is granted to copy this document provided this copyright
statement is retained in all copies. Disclaimer: OBJS does not warrant
the accuracy or completeness of the information in this survey.
Last revised: September 15, 1998. Send comments
to Venu Vasudevan and Tom
Bannon.