Advanced Information Technology Services (AITS) Architecture
Off-Site Review
Washington D.C.
25-26 June 1997
Reviewer Report
Craig Thompson
(bio and contact info)
Object Services and Consulting,
Inc.
Initial comments posted by cwt on 6/30/97. Annotated in red by cwt
on 7/16/97 (most important comments)
-- see changes in sections
Executive Summary
AITS Architecture is on the right track.
The AITS Architecture is impressive in scope and appears to be on track
to provide DARPA and JPO/DISA technical and programmatic guidance as well
as a technology transfer route to the rest of DoD for a family of command
and control architectures covering command centers, crisis management,
planning, situation modeling, logistics and interfaces to simulation and
modeling architectures. The emphasis on an architecture-driven approach
makes excellent sense because it can focus an overarching design on
-ilities, which are desirable, system-wide qualities that the system,
system of systems, or family of systems is intended to have. Some
of these -ilities include:
-
understandability - so team members can understand more quickly
as the problem space is partitioned into parts
-
manageability - divide-and-conquer strategies provide a way to manage
large efforts, isolate the architectural drivers (fundamental or technology
limits), hard issues, and critical paths early, provide a basis for reasoned
decisions about how to grow and extend the effort, how to delegate responsibility,
and later how to test
-
affordability - at several levels of abstraction, from expertise
and history to design to code, locating repeated patterns in the
architecture can lead to reuse, which makes classes of systems cheaper
to build more quickly with more chance for commercial partnerships leading
to lower costs in the future.
-
agility - this is the ability to quickly, even dynamically, reconfigure
a system to meet changing conditions.
Some midcourse corrections. To realize the objectives of the
effort, there are some mid-course corrections that probably need to be
considered. These fall into the following areas
-
Ed Brady's Reviewers Feedback
-
Technology-related Feedback
-
Technology Transfer and
Business Model Feedback
-
Some Comments on Some Specific Presentations
-
Advanced Technologies
to Enable the Advanced Battlespace Information (ABIS) Vision: How Well
are we Doing?, Dave Signori, 18 June 1997
-
Roadmap
for the Evolution of the AITS Reference Architecture: Background, Motivation,
Directions, Rick Hayes-Roth, 25 June 1997
-
ISO
Archtecture and Roadmap Report, James Just, 25 June 1997
-
DARPA ISO Server Study,
Phil Barry, 25 June 1997
-
Dec'96
Transition Schedule/Status, 9 June 1997
-
Overview
of Implementation Guidance for DARPA-DISA Advanced Information Technology
Services (AITS) Reference Architecture (RA), Rick Hayes-Roth, June
1997
-
DMIF II Architecture, Bob
Tenney, 25 June 1997
-
Security Architecture
for the AITS Reference Architecture, Sami Saydjari, 2/10/97
-
AITS Architecture: Modeling
and Simulation (M&S), Jack Kramer, 25 June 1997
-
AITS Communication Server
Reviewers Feedback
Ed Brady (reviewer team spokesman) gave a 30 minute summary of a two-hour
reviewer meeting held the evening of day one. He covered (from my notes):
-
background of reviewers and their architectural biases
-
reviewer process - on major issues, we will provide consensus-based synthesis
but mostly reviewers will send individual comments directly. Some face-to-face
meetings will be needed. Satellite these reviewer meetings with AITS meetings.
Ed will provide periodic group written positions.
-
the group confirmed an architecture-driven framework for ISO. The pressure
on the S&T dollars argues for sharing. A large % of DoD S&T dollars
is in DARPA - so DoD will mandate sharing (so the AITS architecture is
partly self-defense, partly responsible management). How do we derive an
architecture? Most experience is top-down versus bottom up. Story:
SIMNET 22 month prototype was transitioned and prematurely became a hardened
set of mandated architectural principles until Anita Jones moved
on to High Level Architecture (HLA). So, its OK to start with JTF but try
to accommodate the part of the future you can see (don't preclude the good
ideas you can't see yet).
-
there is an opportunity for DARPA to play a leadership role in a suite
of standards groups, a principal one being OMG but also include IETF, W3C
and others. The opportunity is community leverage and direct interchange
with industry. The first version of the document should point this out
perhaps via Views of the architecture.
-
what are the drivers of the architecture. What are the fundamental limits?
heterogeneous, discontinuous bandwidth; fine-grained distribution; what
else?
-
there is real value in separation of architecture from implementation in
discussions.
-
consider different levels of maturity and classes of reuse:
things you share among yourselves but do not plan to transition. Make real
use of the JPO code repository.
The rest of this report is my individual comments in general
and on the various presentation topics.
Technology-related
Feedback
-
ISO Architecture and AITS Architecture
may not be good names for the architecture. Usually,
a reference architecture is not so much named for an organizational entity
or for an unscoped suite of technologies but for a domain, application
or technical area that it is meant to cover. A better name might
be C4ISR Reference Architecture, Command and Control Reference
Architecture, or even Application Architecture for Command and Control,
Logistics, and Modeling and Simulation. AITS seems to be an ISO
spanning architecture but is that a programmatic architecture or a
thematic one? The main message -- consider the name carefully so
it helps scope the effort.
-
scope. Creeping scope is always a danger.
You are working from a hypothesis that you can start with the JTF architecture
and extend it to cover JFACC, ALP, GENOA, BADD, even HLA as well as future
unknown architectures. The good news might be that I think you can
cover or span these architectures but it is does not follow that all architectures
can be spanned in this way so keeping the scope finite is an issue in managing
expectations. Show this approach works for a few efforts before you
generalize too far. You might want to mention this issue in the
Issues Log, one of which you need.
-
kinds of architectures. There are several
kinds of architectures -- you need to describe this up front in the description
of the effort. Here is a simplistic first cut that leaves out a lot
of detail about how architectures decompose to subarchitectures, how they
are related to specifications and implementations, how they relate to ilities,
etc.
-
business architecture - industry-wide, describes
problem space
-
reference architecture or reference model -
term used by the standards community - a high level document that describes
in English the main components and how they relate to each other for a
class of system.
-
component architecture - Each component in
turn can have a reference architecture. A reference architecture
might have a well defined template structure (some parts optional):
name, objective, audience, glossary, -ilities, architectural principles,
requirements, design dimensions, description (the main body), issues, relationship
to standards and past work, interfaces, list of compliant implementations.
-
application architecture - reference architecture
for a vertical application
-
domain architecture - reference architecture
for a vertical domain, one that an industry with many suppliers is centered
on (e.g., healthcare, manufacturing, logistics, electronic commerce)
-
technical architecture - reference architecture
for a horizontal (middleware) technology domain (which itself is actually
a kind of application domain)
-
system architecture - the architecture of
a specific implementation. Implementation guidelines can be
specified to map an application or technical architecture to an implementation
architecture. Such guidelines provide not only today's mapping but
some information on yesterday (grandfathering) and tomorrow (emerging technologies).
Also, this kind of architecture might contain information on capacities,
rates, sizes, etc.
-
operational/functional architecture - mentioned at the meeting --
which one is it? does its scope span application as well as technical architectures?
-
information architecture - is this the view of the modeled application
objects? as well as DBMS-like operations on data/objects?
-
domain-specific software architecture - these seem to be domain
architectures coupled to technical architectures.
-
This would make a useful foil showing the AITS Architecture
as consisting of a collection of application architectures that overlie
a collection of technical architectures that all map to a collection of
standards and/or implementation architectures.
-
services architectures. If you are going
to call AITS a services architecture, you need to define that term.
Some might assume an architecture for the DoD services (e.g., the Army,
etc.) whereas OMG means basic object services or middleware components.
There is a lot of confusion today with technical meanings of service,
facility, component, framework. But all those
terms are evocative and in wide-spread use. See some quick history
on the OMG Object Management
Architecture, OMG
Mini-tutorial, and a recent pitch on Services
Architectures I did for the DARPA Dynamic Database Panel (yes!
I believe in reuse at least at the presentation layer).
-
-ilities, architectural
principles, glossaries, specifications should be documented.
-
Recommendation for an Architectural Principles List. There
is a need to explicitly collect Architectural Principles. See OMG
Architectural Principles, then augment that list.
-
Recommendation for an AITS Architecture Issues Log Document:
AITS architecture should have Architectural Issues document. There should
be an issues template listing the issues, alternatives, and solution approaches.
See an
example issues log from the DARPA NIIIP Consortium.
-
Recommendation for an AITS ilities list. As
described in the Executive Summary
above and in several places in Rick Hayes-Roth's presentation, ilities
provide some of the motivation for the AITS effort. There are many
other desirable properties you will need (reliability, scalablity, evolvability,
survivability, ...). Understanding how to achieve ilities is a research
topic in the compositional architectures area. I am planning to push
OMG in the direction of introducing ilities as an explicit part of their
objective which mainly just includes interoperability right now so one
can consider designing for specific ilities. See ilities
for one list and Rick Hayes-Roth presentations for more.
-
Recommendation for a family of Glossaries. There is a need
to grow the architecture, component and domain glossaries (dare I say,
ontologies). Some concepts are needed for OMA/AITS to move forward the
componentware area itself forward (e.g., containment model, federation,
binding time, boundary -- see Componentware
Glossary for a slightly out of date glossary). Typically,
every domain and every middleware component will have its own glossary.
The overall glossary is the sum of all glossaries.
-
Recommendation that components (services and answer desks) have well-documented
interface specifications. The architecture itself should have
1-2 page descriptors for each service but in addition there should be specifications
for each. The OMG
Object Services Architecture is an example.
-
Identify technology limits. These are requirements that are
unmet that are at the edge of what we know how to do. They may involve
scaling a system in size, time or other dimensions
-
Design for Evolvability. A reference architecture must span a diverse
collection of requirements. Research projects need to "design broad and
implement narrow." It is important to identify apparent roadblocks and
their solution approaches early. Then later flesh out and populate the
design.
abstract:
REQUIREMENTS DESIGN IMPLEMENTATION
concrete:
requirements design implementation
So consider all known requirements including as many future requirements
as you can at the architecture level, design so as not to preclude future
capabilities now predicted but implement according to your current resources.
The good news: the system you design with the resources you have will not
preclude the system you can extend it to as the system evolves. The limit
is unpredicted requirements that break key system abstractions.
-
compositional architecture issues and importance. This area is promising
but poorly understood in the abstract and so even more poorly understood
in implemented systems. This is the critical assumption at the heart of
OMG, AITS, and many other programs - that reuse is possible based on components.
The alternative is spending billions on stovepipe solutions so this is
a critical path for DoD. No theory-only approach will solve the problem
nor will community efforts like OMG that do not focus on critical paths.
Instead, we need more engineering experiments that try to derive theory
while testing it in systems design. AITS is a good domain as long as it
introspects on critical questions like:
-
component containment models - so you can mix and match which you cannot
guarantee to do in OMG today.
-
federation architectures to scale systems, end-to-end federation architectures,
integration across echelons
-
preserving ilities and using wrapper technology to insert new ilities into
existing systems
-
coordinating views -- interaction of X and Y - e.g., plans and situation
assessment
-
decentralized control
-
granularity of modularity
-
inserting new cross cutting technologies e.g., "end-to-end reliability"
-
packaging - where do we cut, how we modularize and understand the range,
parametricly requested and varying implementations, finitely available
to meet a certain range of ilities (size, speed, …)
-
insulation - when is it good and when is it overhead
-
if I have system S and I want to insert technology T, can I do it? Add
in security, add in drill-down/pedigree server, management interfaces,
and can I delete these and add new ones/ Its like an overlay on the web
server, says Rick. Security unaware servers and in general X-unaware
wrappers.
-
economy of componentware - cost to harden - committing to optimization
too early - institutionalizing continuous improvement - whose fault is
it? Componentware responsibilities
-
systems depending on assumptions like QoS, footprint, … failure points
- the $ to maintain and extend X and Y depends on X when there are 1000's
of components. 100s of people extending X to be X1, Xn and no reconvergence.
-
Is there a normalized framework to see if two architectures are the same
or different?
-
recursive wrappers
-
lightweight versus heavyweight implementations of services and facilities
- what does this mean?
-
can an OODB partially implement the functionality of an object web server
-
confusion over services and facilities even at OMG
-
should security be in the backplane, bundled with the comm. server. - probably
the wrong way to think about this
-
see some additional issues in the OMG
Internet SIG's recommendations to OMG ORBOS and Common Facilities
in the area of composition architectures (based on my DARPA-funded work).
Also, see some OMG
Directions in Compositional Architectures
-
there are many other issues
-
Recommendation for a Workshop on Componentware Architectures. We
might want to assemble a componentware architect's forum to try to make
near-term progress on these topics. AITS architects should attend as well
as software architecture gurus like Garlan, projects like NIIIP and MCC
OIP, and industry thinkers in Netscape, Sun/Java, and Microsoft. I proposed
to MCC we organize a workshop in this area for November. Interested?
-
Recommendation. DARPA might need a program
on Agile Compositional Architectures (though that might be what Randy
Garret is providing -- I need to know more about this.) DARPA has
ITO programs in some ilities (evolution, survivability) and results from
these programs should impact AITS. The objective of the thrust might be
architectures and system mechanisms for "rapid application assembly from
reusable component parts" (a phrase you might want to adopt) and might
include the AITS effort as well as research efforts on componentware, ilities,
federation, etc.
-
Servers is a bad term.
Before AITS fans out too far, I recommend you rename the various servers
to be facilities in line with similar OMG term usage. The term server
is heavily overloaded and implies an implementation level partitioning
of functionality between clients and servers. OMG distinguishes primitive
services from facilities that are higher level specifications
that are composed of services (or other facilities). In OMG parlance, most
of the AITS servers seem to be facilities. See comments below on careful
separation of concerns of server specifications into separately specified
service and facility specifications.
-
Answer Desks. Answer desks seems
like a good term in that it is evocative of expertise collected into points
of view and arranging software by domain expertise. There needs to
be some discussion of how anchor desks coordinate in the architecture so
that logistics delivers goods while taking into account weather conditions.
That is, inter-view coordination and constraints.
Technology
Transfer and Business Model Feedback
Research may produce technology but its value diminishes if it is not transferred
to operational practice (widely used) in a way that becomes economically
sustainable (business model). Leverage describes the idea of targeting
transitions to solve increasingly complex problems and maximally accelerate
community progress in adopting the solutions. Currently, the
AITS Architecture appears to be the locus for a significant paradigm shift
from research programs disconnected from service needs to more focused
R&D. AITS begins to rationalize and create a transition model
between DARPA ISO application programs and the rest of the JPO, DISA and
DoD.
Right now, the AITS presentations only include a technology transfer
process for moving research to applications and fanning it out to DoD services
but is missing several ingredients. It does not include non-DARPA
researchers, industry, data flow back from DoD as partners in development,
and standards. The DARPA technology transfer process should think
big. It should include information flow back and forth among research,
industry, DoD, and standards. You might find the foil I made for
Kevin Mills' IC&V program a useful start. Its here
with some commentary here.
A useful phrase we invented some years ago with respect to participation
in industry forums and standards was "accelerating consensus leading towards
standards".
Leverage Industry and Accelerate Community Progress by Participation.
Most DARPA projects are underfunded to solve complex problems in a way
that will allow meaningful, affordable transition to the field. Now seems
to be a time to augment this process into a collection of dialogs with
industry, which can be done with a variety of mechanisms. Goals are to
take the deep knowledge of DARPA programs and accelerate high leverage
transitions of DARPA technology out to industry (and equally important
back in). The variety of mechanisms available include:
-
meet with the key industry shapers to forward common goals, either one-on-one
or via key technology workshops with targeted invitees
-
engagement with key standards groups or industry consortia, particularly
key parts of key groups where leverage value is highest (e.g., selected
interactions with OMG, W3C, IETF, ODMG, OGC/OGIS, etc.)
Though there is anecdotal evidence that some DARPA projects have attended
OMG meetings and possibly made a presentation now long forgotten, there
is limited evidence of active interchange where the DARPA-funded work made
a traceable advance. That is unfortunate because there are many opportunities
for interchange. Recommendation on Standards
Participation. Take an inventory of which teams are interacting
with which standards groups or other industry groups (e.g., consortia)
or even product groups productizing DARPA software technology. Recommendation
-- Develop an explicit Tech Transfer Plan. If you don't have
a plan, you are unlikely to succeed in this form of technology transfer.
Be proactive -- don't passively assume standards happen to you or happen
"over there" -- instead engage in and possibly lead the process. But choose
your battles -- not all standards are worth the time.
Specific Opportunities for OMG-DARPA engagement. Organizationally,
OMG can be viewed as a big data structure of communities of interest where
many of industry's better system architects and middleware technology vendors
meet (every two months) and where consensus-based standards evolve. OMG
has no real research arm. DARPA could play that role where OMG might plays
one of the major tech transfer routes out for some kinds of DARPA technology.
Additionally, DARPA interaction with industry can short-circuit adoption
of good ideas into broad communities that allow the services to adopt these
from industry. Finally, the process of interacting with experts (from industry
and also from other centers of excellence in DoD!) gives DARPA some out-of-band
ways for their experts to talk to other industry experts to share expertise
and get ideas critiqued so that the tech transfer route is DARPA to industry
to DoD rather than just DARPA to DoD to industry.
In the past and present, DARPA-funded effort has played a key if limited-bandwidth
role in moving OMG forward:
Recommendation to engage with OMG. It
would be beneficial to both OMG and DARPA if DARPA programs participated
in OMG more actively to share their expertise. Here
are specific opportunities for interchange:
-
OMA-NG (Next Generation OMG OMA architecture) - Craig/DARPA funded
is contributing here via composable architectures work. This will happen.
Would Randy Garret's work fold in? At OMG, work in this area is happening
in the Reference Model Working Group and the Internet SIG.
-
OMG Internet SIG is working on a facility like the Data Server for
Information Access -- we are not far along. We might work together
to define the facility starting with the I*3 Reference Model, JTF Data
Server, BADD, or Infosleuth architectures.
-
OMG Real-time SIG and QoS aspects - would AITS Comm Server fit here?
-
OMG Security SIG, OMG Firewalls RFP. Sami could at least present
his Security RA to this group and might consider working it into the OMG
Security Roadmap.
-
OMG Simulation SIG - DMSO HLA MITRE advocate Fred Kuhl is leading
this. There should be DARPA presence to establish how to augment OMG systems
with simulation, possibly via the HLA as a simulation architecture.
-
OMG Command and Control Working Group - AITS could co-opt this group
by inserting its architecture as a starting point. Also, the C2 Schema
work would fit here.
-
OMG Workflow Working Group - this group is allied with Workflow
Management Consortium (WfMC).
-
OMG GIS SIG - this group is closely allied with the Open GIS Consortium
(OGC) which is driven by the NIMA
services architecture and here.
The Map Server fits at OGC (and Allan Doyle is active here)
-
OMG domain task forces exist in overlap areas including Transportation,
Healthcare, Manufacturing, Telecom, Electronic Commerce
-
some OMG technology exists for sentinels and triggers (OMG event service,
interceptors, portable object adapter, and proposed Event-Condition-Action
Rules Service). This is related to DARPA Sentinels and Triggers,
Reactive X, and also Object Level Change Detection. OMG is adopting
an Agent Facility. It has a Trader Service specification.
-
no OMG groups exist for Planning, Scheduling, Logistics, Situation Modeling
- DARPA could seed these with expertise.
Similar targeting strategies are needed for AITS to engage in or closely
follow other related standards groups like IETF, W3C, OGC, WfMC, ODMG,
and more. By no means are all standards groups are worth following
so you need an investment strategy.
Dual Use Tech Transfer and Business Process. DARPA AITS is solving
a "business problem" and some kinds of agile businesses could use all or
parts of the architecture.
-
At a minimum, any enterprise's organizational memory would be enhanced
with situation models that allowed representing a landscape of areas
of interest and allowing what-if-ing.
-
Business planning or war-gaming could use some of the planning and
scheduling.
-
ALP could contribute to agile logistics. Etc.
The point is that it is possible to begin to envision a DoD-industry partnership.
The sooner systems like this become part of industry, the sooner key technologies
will become available to much larger communities, including the DoD community.
Procurement practices. Rick mentioned reward structure,
compromise, willingness, synchronousness. I'll add availability, in the
sense of low cost of entry to the community working in this area.
At one point in one presentation, the issue is raised about the "high complexity
of the overall product." Right now, the cost of newcomers into the
DARPA application programs is high and sharing less than it should be because
of reward structures. Your need to work to lower the entry bar.
A related point is that there may need for changes in procurement practices.
Most DARPA/ISO contractors are DoD software integration specialists.
But while they may be innovative they do not provide a direct route to
the part of industry that builds COTS products. Better mechanisms
are needed so that software developed for AITS does not get locked inside
monolithic or proprietary walls. Publishing specs and putting code
in the IE Repository may provide parts of the answer as will working with
OMG. [Personal note: the reason OBJS principals left TI was
that TI made a business decision not to productize Open OODB and we saw
no point in continuing to develop it as a proprietary non-product. Are
there commercial plans for AITS servers? is the source code available
for others to use? are the specifications available? if two
parties implemented the same specs would they get the same capabilities,
that is, how much semantics is described in the spec?
Think Big and Market your Work. Outside
of ISO-related application projects (and perhaps ITO and the JPO-DISA chain)
not many know much about the AITS architecture. It has so far had relatively
little impact on some other major DoD architectures like NIMA, HLA, probably
others or on community architectures like OMG's. While this is partly because
the work is young, it may also be because the effort is not advertising
its vision and impact to the communities it wants to influence. The communities
are definitely within DoD but the impact could be business-pervasive. So
think big and begin to think out the implications of your architecture
work and who may be the benefactors. As a very minor side-note, be sure
you keep your AITS web page up-to-date and current. When I tried
to come up to speed by browsing the ISO Architecture web site, I did not
find that much there.
Comments on Presentations
Off
Site, Dave Signori, 25-26 June 1997
-
concept of grid seems to be at same level as middleware,
comm., and networking but is not really explained. I'm not sure why
Transactions are given the primacy here -- they are just one of several
middleware services.
Advanced
Technologies to Enable the Advanced Battlespace Information (ABIS) Vision:
How Well are we Doing?, Dave Signori, 18 June 1997
-
wow, DARPA is covering an impressive range of technologies,
broad investment strategy. Many comments I had on this section show
up elsewhere in this report
-
might want a simple foil to show matrix of Application
Programs by Technology Areas and reuse checkmarks. We did this to
motivate Open OODB and why one wanted a component solution so customers
could mix and match the DBMS capabilities they needed. AITS is a
larger version of the same problem
-
It was once said of one of the presidential candidates
- if he thinks it, he thinks its done. There are a lot of promising
technologies in the DARPA investment portfolio. How many of these
will really bear fruit? How many are different names for the same
things? I guess this is just a reminder that along with the new ideas
there is the huge winnowing process to make sense of it all. The
AITS effort seems perfect to create the DARPA data structure around which
research results can be aggregated.
Roadmap
for the Evolution of the AITS Reference Architecture: Background, Motivation,
Directions, Rick Hayes-Roth, 25 June 1997
-
thoughtful, eloquent presentation. I couldn't
have said most of this better myself. I agree that architecture is
the single highest leverage point. You might want to move the backup
foils on What is an Architecture in-line and into the introduction of the
ISO Architecture document to provide more motivation.
-
a maximal (modulo adding more later) AITS Reference
Architecture with all services then color coded for JTF, JFACC, etc would
get across the idea of reuse.
-
might need a process foil showing: programs:
{JTF, JFACC, BADD, GENOA, ...} -> common architecture --> components and
then feedback to programs to get across the continuous mining of programs
for reusable technologies
-
a good argument for frameworks is in terms of who
benefits, for instance:
-
end users benefit because they are presented
with a common semantics (like a generalization of a common look
and feel in a Mac toolbox) so once they learn how to use some parts of
the AITS framework architecture it is easier to use other parts
-
developers get the benefit of building new
applications more quickly from tested components
-
Might want to say this in the introduction of AITS
Architecture
-
you need a business model to win the DoD architecture
wars. See business model comments earlier.
-
what does it mean "The architecture aligns with OMA
and cohabits with MS proprietary standards"? - do you mean it is
technology neutral? By the way, there are starting to be a host of
bridging technologies for Java Beans, IDL, and ActiveX. For instance,
see Java
Beans FAQ, Beans
Bring Components to Java Developers, CORBA:
Catching the Next Wave.
-
need a mapping from AITS services to OMG services.
If you find generic services that you have implemented and found useful
and have not taken to OMG, why haven't you? Recommending it and doing it
effectively are two different beasts. DARPA is not walking the walk.
-
are you really capturing your lessons learned
in trying to build JTF from and as componentware? What are the main
lessons learned that you (or I) can feedback to OMG/industry. But
the main point is, if OMG nor DARPA really are fielding separable composible
services is the emporer currently wearing clothes and how do we get him
to accelerate putting some on so we are not all embarrased? Will
the GOTS only w/o COTS transition really payoff or are we generating next
generation monoliths and calling them componentware?
-
(aside: regarding the "architecture process,"
I did not find the ISO Architecture homepage useful in coming up to speed
on this effort. To suceed you will need to get mind-share from ISO
and ITO as well as industry projects. The IC&V program has the
goal of briefing people 10X faster but one still needs briefing materials
to be logically centrally collected.)
-
there is a need for a DARPA program on composable
architectures -- see foil 15. Need theory and practice.
-
I would like to know more about
-
how you propose to insert band-width awareness
into apps, for instance.
-
the solution to the OO granularity problem
-
what does it really mean in practice to be
ooweb-based for the OOWG C2 schema
-
See other sections. Material in other
sections is relevant to this section. Jim Just should review the
other sections of this report.
ISO
Architecture and Roadmap Report, James Just, 25 June 1997
-
Document Outline? The presentation seems
more like an early outline, deep in some areas and just a listing of sections
in others. I'd prefer to see the AITS Architecture become a document
rather than a presentation because presentations can contain too many things
left unsaid or said only at the buzzword level. Still, the presentation
as is seems a reasonable first cut in terms of content or proposed content
but it does not flow well in terms of a presentation.
-
Section 1 Introduction. I think the
briefing should start with a short executive summary of the AITS Reference
Architecture
-
Objective - an overarching C4ISR architecture
into which DARPA programs JTF, JFACC, BADD, GENOA, ... fit - maybe use
bullet two and one of the Assumptions foil.
-
Problem - state the problem you ar solving
-- why use an architecture approach - wanted: an agile, reconfigurable
architcture to meet the diverse needs of small regional conflicts, large
multinational conflicts, disaster recovery, peace keeping and operations
other than war. Might just want to miniturize several Signori/RHR
foils with a line under each.
-
Approach - input is a collection of DARPA
information architectues, output is a factoring into reusable components
in an overarching architecture with a goal to leverage and accelerate industry's
move toward componentware. Show the AITS Reference Architecture no
later than here.
-
Significance - "Who is the customer?"
-- give the DoD as customer story (beeter-cheaper-faster, more-with-less)
including a brief about the tech transfer route thru to DISA
Then mention the "living architecture" is being used
to drive DARPA project efforts (and so the purpose of this document is
reflect this evolution over time)
-
Combine Section 2 in with Section 1.
You could combine sections 2 and 3, the parts relating to the significance/tech
transfer story (4th bullet above) into section 1. You want to first
highlight the AITS architecture and then tell that it fits into
the DoD larger story represented by Vision 2010 and ABIS, which are both
more conceptual architectures. I think the claim should be that AITS
is instantiating these most excellent but lofty architectures providing
the means to achieve these long-term objectives. (I would leave remove
the pseudo architecture statement in the box that says tiers above depend
on tiers below and instead focus on Effective Force Employment depends
on Battlespace Awareness which depends on the Information Grid. If
you do want to push the idea of an information grid, you need to say later
how it relates to the AITS architecture. I would bundle these points
into the Intro section.
-
Section 3 AITS Reference Architecture.
-
begin as you are with AITS Reference Architecture
foil -- but simplify picture leaving out Security and Other COTS/GOTS
-
I do not like Security as a backplane extension,
as I mention below. It is one and not the only one of several cross-cutting
services - systems management, QoS/optimization, even versioning, distribution,
persistence, replication can be similarly cross cutting. It fits
naturally into the Core Services. In general, I
would remove the Extensions notion from the architecture.
-
The diagram shows applications, vertical domains,
and horizontal technology. The concept of reuse of the generic domain
vertical and horizontal technology among DoD applications did not come
across in the presentation. I suggest you delete the words on the left
and color code that into into a legend:
-
color blue - Domain-Specific Applications
-
color green - Domain-specific Generic services
-
color yellow - Generic Infrastructure services include
User Environments as yellow
-
Attaching the COTS/GOTS to the side makes this partly
into an implementation architecture. Keep that separate -- delete
for now. Make this point with a bullet
-
either delete the Network/Enclaves/Domain view or
put in several other views or move it into the sections on
-
Section 4 & 5. Domain Architectures
& Component Layers.
-
These sections show a lot of program level
architecture pictures but do not show very well how they relate to the
AITS architecture or each other. That seems to be the main point
of the briefing and it is not made well in the current presentation.
Not your fault -- its a lot of work to do this.
For instance, the JFACC and JTF architectures contain not just domain but
also component services. Neither the ALP nor BADD pictures look much
like this. Show how the architectures are related. It seems
that there are endless variants of the JTF/AITS picture, some with workflow,
others with security, some with today's anchor desks and some with proposed
ones. It would be nice to have a most general one and to color code
it for specific applications to show subsetting. That is what we
did for Open OODB in our IEEE Computer paper to show the general design
could provide a collection of useful subsets. Also, it is somewhat
hard to tell Domain Architectures from Component Layers.
-
remove C++ from Joint Situation Assessment &
Planning RA. Why are (generic) monitors and triggers in the application
layer here and in the server layer in the JTF/JFACC slides.
-
you need a foil mapping the AITS architecture onto
the OMG RA. It might have JTF, BADD, ... in the applications corner;
Logistics, Meterology, Command and Control Schema, ... in the Domain generic
corner; Planning, Scheduline, Workflow, ... in the Common Facilities corner;
and middleware services like security in the Basic Object Services Corner.
-
I find it odd that you hardly mention the OMG
basic object services, .e.g., the COSSservices -- these include events,
naming, persistence, queries, transactions, trader, and several others.
See OMG
Mini-tutorial.
-
Differences in architectures - Federation.
While JFACC looks like JTF, ALP does not,
rather it looks more like HLA in its replication federation pattern.
It is my experience, mentioned elsewhere in this report, that federation
architectures are a main kind of scalability architecture. The JTF
architecture view does not show off the federation pattern. But JTF
systems would need to federate too, both along geographic and command hierarchies.
That is, they must share info across boundaries with others of their kind.
[By federation, I mean the repetitive pattern of having a service or facility
repeated in an architecture in such a way as the instances can intercommunicate.
A much weaker notion of federation is just that different systems can interoperate.
I mean the stronger notion. Also, I am not really emphasizing the
centrality of control, which some people associate with federation, where
they prefer the term confederation for decentralized control. [PS
How can i get to know more about federation in ALP.]
-
We need a definition or archicture and an architecture
glossary with terms like system of systems before we label slides like
the one on Battlefield Awareness. The idea has merit as a componentware
packaging notion but is used as a buzzword on the Battlefield Awareness
foil.
-
Section 5 Component Layers.
-
I would almost rather see all the JFACC, BADD, ...
subarchitectures in one section than have them divided across sections
4 and 5. I'd rather see the services described in terms of pure AITS divorced
from program. Separately, one
could have a matrix showing AITS services x DARPA programs, so BADD check
marks appear in the BADD column.
-
Data Server vs BADD DDS/Infosleuth vs I*3 -- It is
worthy getting asingle Information Access Architecture (Data Server Architecture)
that relates queries and results from diverse data sources. I understand
from David Wells that there is an effort to do this partly made of BADD,
Data Server, and I*3. OMG/we are also working in this area so its
a good area for technology collaboration.
-
Section 6. By extensions do you mean
new areas you are going to extend AITS to cover -or- do you mean an architectural
dimension that cuts across others and needs a "third dimension" for the
architecture. The first is reasonable. The second needs much
better justification. OMG does not separate security nor system management
as a third dimension .
-
See my comments on the security architecture below.
-
Simulation and Planning seem similar in that they
both operate on the C2 Schema objects to consider alternative futures (using
somewhat different technology bases). So why is one a component service
and the other an extension?
-
System management will be a hot topic in the next
year or so since industry is holding back on investing in componentware
until it knows it can instrument it. OMG does have a sysman spec,
which I have never read, which supports "managed object" interfaces to
system objects that need a sysadmin interface.
-
Section 8. This is my favorite section
of the paper. It represents a thoughtful start at an Issues Log,
as recommended below. You do need explanations of your issues.
Why is Java an issue, for instance? Good idea to list Possible
new services. Would be even better if we had a list of services
that we have identified. Good idea to list priorities, success factors,
etc. Good section.
-
Other comments.
-
The AITS Reference Architecture is missing
-- see above
-
a Componentware Glossary
-
a list of architectural principles
-
a list of desirable ilities you want to preserve
-
a list of all services and a descriptor for
each -- I think that is what "Illustrative Data Services Details" is that
it is a descriptor -- so now identify the services and fill in the descriptors
in the document
-
an Issues Log - issue title, #, description,
alternatives, resolution.
-
This should also list the research gaps and
service gaps that might define new DARPA ITO programs.
-
a separate section on an implementation architecture
showing a mapping from services to implementations
-
Right now the document is mainly about architecture
and not much about roadmaps.
-
See other sections. Material in other
sections is relevant to this section. Jim Just should review the
other sections of this report.
DARPA
ISO Server Study, Phil Barry, 25 June 1997
-
useful overall study - good job. I read the whole brief including
the backup. Some of my comments are on the brief and some just appended
where I had questions or wanted to raise issues regarding individual servers.
-
publish the various server specifications available centrally from
the AITS homepage (point to them)
-
servers as services or facilities. For each server, determine if
its specification is a composite of more primitive services or a primitive
service spec. If the former, then publish each primitive service specification
independently. For instance, OMG is beginning to consider (hallway conversations)
separate specifications for a replication service, cache service, index
service, change management service. Is the web server really a composite
of these?
-
Data Server - might mention Gunnings's I*3 Program, MCC
Infosleuth (slow, they tell me, but I like the architecture), OMG
Query Service, other related OMG services like Persistence 2.0, Transactions,
…, ODMG OQL, the Dynamic Database (DDB) need for queries, Global Infotek's
work to unify Stanford Tsimmis and Harvest, as well as Microsoft OLE DB,
which publishes a set of DBMS internal interfaces. BTW, I know of no commercial
equivalent COTS product. Might want to reference our Web + DBMS Integration
report and Semantic/Object File System report as well as OMG work on an
Object Transmission and Manipulation Project (OTAM) Facility, which could
benefit from a Generalized Data Access Facility, which is part of what
the Data Server is. By the way, I think the data server is a composite
spec with several separable subspecifications that need to be separated
out for Ontologies, schema management and schema federation, cache manager,
Harvest-like gatherers/result composers, etc. You might want to run a workshop
in this area to get the architecture of the data server better enunciated.
Most desirable is an open framework for adding new capabilities - much
historic research here - so as to accommodate JTF, BADD, and other future
similar query needs. Also, query is a separate capability from data access
from multiple sources. I'm a relative expert in this area. This is an obvious
area in which federation architecture are being used to access remote data
sources.
-
Object Web Server - sounds very interesting but I understand webobjects
are not the same as IDL or JavaBeans or … but rather a primitive semantic
net of nodes, arcs, and lists. If this is still so, then I question building
huge amounts of the representation from this sort of datastructure alone.
Probably I do not understand something since I know you are representing
the C2 Schema in IDL. At any rate, again there seems to be a mixin of capabilities
in one level (replication, versioning, persistence, crash recovery, consistency)
and there need to be separate specs even if the webserver is a composite
of these. Replication is a separable spec, so is versioning. The question
about using an OODBMS is a different level of concern - more an implementation
concern. Might well make sense.
-
Plan Server - would like to see this specification.
-
Comm Server - would like to see this specification and how the economic
model is being specified. OMG has a Real-time SIG and just issued an RFP
in this area this week (probably). How are bandwidth and real-time constraints
specified? Also, just like security, these sorts of constraints must federate
end-to-end across comm server boundaries as well as top to bottom within
some bandwidth aware applications.
-
Situation Server - I am most interested in this. I'd think the entire
web will benefit from this idea to help communities organize information
in richer data structures than just HTML or Java. So what does this bring
to the party?
-
Map Server - This needs to be (and is, I think) coordinated with
the JMTK.
There are many, many community-wide formats for map data (naïve).
The GIS
standards landscape is pretty rich.
-
Model Server. What is this one? Do you mean M&S
models? Or Model-View-Controller models? Or schema modeling
in the OMWG sense (in which case why is this called a server)? On
these topics: it does seem high payback if the AITS C4ISR and M&S
community modeled can use the same (IDL) entity definitions (schemas).
Also, is OMWG and C2WG the same group? How is it related to the DoD
Data Dictionary and DoD MIDB
schema (several hundred pages long and modeling 95% of all-source C4I entites
at the theater level (I think)) and other C4I schemas? If M&S
and C4ISR used the same OMWG schema then one could conceivably query a
simulation in progress to say "Show me the enemy planes within 50 miles
of my position" and dynamically see map displays where planes came and
went during the simulation using the Map Server and Data Server working
together.
-
Information Repository - rename this foil. During the briefing,
I thought you were suggesting to develop an Info Repository Service. (If
you think this is a good idea, talk to me before doing any such thing.)
What you are recommending is the same thing I am - namely to publish the
specifications as well as auxiliary information. The documentation template
makes sense for the JPO repository of code and specs.
-
Noah's Ark/Reference Implementations. IETF requires two or more
interoperable reference implementations available to industry before escalating
a protocol to a standard. OMG has a commercial requirement though it is
poorly enforced. This is a really good idea. Recommend: Be sure
to test all AITS servers and anchor desks in testbed environments before
escalating them to LES. Recommend: Make reference implementations
available in source code form on servers available to the DARPA community
-- you are paying for them. This facilitates potential extensions by others
than the original producers. There is a major procurement issue here which
I have had bitter experience, namely, many DARPA and DoD contractors are
primarily integration contracts and will guard their code's intellectual
property rights to the hilt and then never go on to productize. This is
almost the rule and not the exception. That means most AITS code will eventually
sit on shelves or be used as competitive barriers to potential other DARPA
contractors who want to be involved in the AITS project. This is a critical
issue since barriers that discourage entry into this community virtually
guarantee that the DoD will continue to pay and pay again for similar technology
and no COTS componentware economy can get a strong foothold, which need
to a primary part of the AITS-JPO-DII process. (Right now, I would like
to try out GlobalInfotek's Tsimmis/Harvest/IDL and Data Server and might
like to try out Situation Server and other servers.) Finally, the OMG community
would do well to adopt the IETF precept of two or more reference implementations.
And DARPA would be adding real value to the commercial community if it
funded a collection of OMG services implementations that were portable
to various ORBs (beyond the state of practice today) and composable into
facilities (also beyond the state of practice). Amazingly, there are no
really convincing examples of validating the OMG componentware approach
- though many a project has internal implementations of several narrow
path OMG services. This is an impediment to AITS, NIIIP, MCC OIP and many,
many other projects and threatens the open systems foundation DII COE and
industry is trying to build on. (End of this opinionated but true harangue!)
-
Final backup foil on Suggested Additional Servers/Services.
-
A security server or anchor desk makes sense. It is a subkind of sys admin
or system management function. So there is an issue about bundling.
-
OMG is working on messaging. By which they mean queueing and asynchronous
method dispatch mechanisms as the term is used in the Message Oriented
Middleware (MOM) community. If you mean Military Messaging (like parsing
natural language) then qualify the Messaging Server name.
-
A Policy Server seems like it is a system management function. Every service
might have policies attached. OMG needs to adopt the word policy generally
- they have done so in their Security spec. It is a general idea.
-
Object Name Server - OMG has a name service. Similarly OMG has a system
management spec, which I doubt if anyone has implemented. The idea is right
but the details might not be. OMG has a Concurrency Service and Transactions
Service.
-
You need a paragraph on each of these services. Ontology Server and Schema
management is related. Performance is not per se a service but a metric.
Do you mean (again) System Management, the ability to instrument a service
with network management functions. If so, this is a really critical missing
OMG service that real industry requires before they move to CORBA.
speaking of which, real industry needs firewall negotiation services
for ORBs before they need some of the other security services. So will
you as you scale the AITS architecture implementations. Apropos to this,
OMG passed a Firewalls RFP this week (I think).
Overview
of Implementation Guidance for DARPA-DISA Advanced Information Technology
Services (AITS) Reference Architecture (RA), Rick Hayes-Roth, June
1997
-
Regarding the viewgraph on "Influencing Entities on the RA". add
industry componentware trends including Java Beans, ActiveX.
-
Regarding the viewgraph on "Tiers". reorder the right column
bottom to top to match the left.
-
Regarding "Core Guidance".
-
Need rationale, criteria. If you have not done so, include rationale
for down-selecting to certain technology choices and provide criteria
for making selections is important.
-
Java in general. Permit Java use more broadly than GUIs. Why? Because
it is "a better C++" (which is a poor, inflexible but popular language
for building large systems), it is likely to supplant Smalltalk so its
a good choice to inherit some of the best of both worlds, it has a huge
groundswell of interest, tools are developing rapidly, and I and others
are going to push OMG toward supplanting IDL with Java.
-
Common Lisp. Consider the many merits of Common Lisp. Long live
Howie Shrobe! The point is, you need flexibility to solve hard problems
and C++ is a straightjacket that truly damps the ability to think, scale,
maintain, extend.
Dec'96
Transition Schedule/Status, 9 June 1997, Don Eddington
-
Servers transition to LES? It would seem
to me you'd need some field experience with the various servers before
you transition them to become LES. The transition schemes seem aggressive
to me.
-
Federation of Servers/Anchor Desks.
Right now, the servers have a sort of component composition architecture
but do not seem to have a scalable federation architecture so servers and
anchor desks can talk to others of their own kind. This is needed
so that one can communicate with another when mobile entities tracked by
one move into the grid of another. See tracking
DBMS.
-
Scalable Architecture. What does Scalable
Architecture mean on foil #3 LES 4.X
-
Categories. Some items listed are whole
categories, like Digital Library or Security APIs. Putting them into
the diagram gives a kind of odd impression since presumably there will
always be new Security APIs. So which ones are adopted when.
That is, categories is pretty broad if this is meant to be a phased acceptance
chart.
-
IE Segment Repository. I like the idea
of this but do not like the name segment though I understand the
COE uses that (I gather to mean a configuration). Keep the experimental
software repository as light-weight and web accessible as you can so it
matches the way research products as well as COTS products provide downloadable
copies off their web sites. The repository could almost be a directory
of references to interesting technology. Also, DARPA (and DoD) is
only one source of leading edge technology. There is increasingly
much of this technology now available on the web so just categorizing and
sorting through this is pretty useful. Also, it demonstrates an aspect
of scalability that this repository effort is not quite apprehending, namely,
there are now so many sources of brand new software that it is perhaps
wrong to imagine copying it into any central place. See our Internet
Tools Survey. This is another argument for including pointers
outside IE Segment Repository including to non-DARPA, non-DoD, industry
efforts. Most categories list a large number of leading edge efforts
and we have only surveyed some of the kinds of advanced technology that
are needed in large (or small) organizations as we Internet-enable them
to scale them. One other effort that is successfully logging useful
software in a more limited environment is the National
HPCC Software Exchange. You could publish a web registration
procedure. An annotation scheme is needed so you can record experience
with IE Repository entries -- perhaps associate newsgroups with each "product"
(entry) so users can provide comments to developers that other users can
see. Basically, there is a philosophy issue of whether the IE Segment
Repository should be a controlled repository or an open junk yard.
The latter can be remarkably useful and the former need good criteria for
what goes in and is kept out, and the barrier of entry, maintenance, ...
can be higher. Explicit thoughts about who has access, what
kind, ownership, is use tracked, ... are all worthwhile at this stage.
Eddington's Issue: if we download shared code that others change, then
proliferation hurts. How should we handle this?
-
Advertising. If the IE Segment Repository
is to be valuable, you need to advertise it to ISO and ITO at least.
If the IE Segment Repository can be somewhat self-maintaining, better for
you.
DMIF II
Architecture, Bob Tenney, 25 June 1997
-
much of this I cannot comment on not being a sensor
guy. One note: Sensors appear to be a kind of stream
data source that various filters operate on and fuse. So the I*3
work on federating information sources for query purposes might apply here
too.
-
pedigree server. Its clear to me that
pedigrees will be needed throughout the AITS systems, from planning,
to situation modeling, to ... Also, pedigrees like annotations
are a special use of relationship with some particular semantics
associated to record derivation history, certainty, etc. An architectural
puzzle is: if you had a system without pedigrees and wanted to insert
this capability so the system now tracked pedigrees, how would you do it?
The answer cannot be that you would rewrite the system from scratch.
Hint: people often say security must pervade the entire system in
just this way, which I doubt though it will be true of legacy systems not
designed to admit behavioral-extension wrappers.
-
Bob mentioned "Many views - Many tools to turn on and off lots of
views of the situation. Issue is to maintain consistency between different
layers." This issue is also true of inter-anchor desk coordination.
-
Bob mentioned "Coarse grained vs. Fine-grained management of fusion
engines." This might be another federation architecture.
What is the DMIF experience on granularity?
Security
Architecture for the AITS Reference Architecture, Sami Saydjari, 2/10/97
-
Recommendation. Talk to OMG sooner rather
than later. This work appears complementary to OMG's Security
specification. You should get on the calendar of the OMG Security
SIG to talk to them about it and enlist them as reviewers. Contact
is Bill Herndon. Possibly
they might eventually inherit the DARPA Security Architecture.
-
See Joint
Meeting of OMG Internet SIG and Security SIG for recent thoughts
on where the OMG Security SIG might target their efforts in the
future. Also FYI see miscellaneous
notes on Security presentations.
-
End-to-end and Bottom-to-Top. I
am not a security expert but ... There is something about levels in the
slide 4th from the end that I think leads us too far down the layered architecture
path. You mention on the previous page that it is current security
philosophy to "push security functions as low as possible in the architecture"
but this may not always leave the right impression. Security can
"wrap" not only the communications/ORB and below but also applications,
that is not only end-to-end but top-to-bottom. Furthermore, it might
wrap higher level functions but not necessarily the networks they deliver
over, so you can still secure higher level communications over unsecured
communications paths, for instance. At least, my compositional architectures
work leads me to believe you can do this. On the other hand, it is
probably the most important thing to get security as the network/ORB layer
and app-level top-to-bottom security is probably lower priority.
-
Anchor Desks and Servers. You posit
a Security Anchor Desk. I think this is reasonable. But we
need to understand this idea of adding Pedigree Servers and Security Anchor
Desks architecturally. I think I can see how to do this. It
is worth pointing out that Security Anchor Desk could be a part of System
Management Anchor Desk. And could in turn have the Authorization
Anchor Desk and the Virus Control Anchor Desk report to it. In other
words, in small C4I systems there might just be one person who mans the
Weather, Sysadmin, etc anchor desks and in large systems role-splitting
might mean there are several specializations. Also, one person might
do several aspects of several roles. So this is a dimension of the
architecture that needs to be better understood. (Its not a Security-specific
issue but more general.) Another key issue with respcet to anchor
desks is coordination and consistency across these different views of the
system assuming some of them allow not just viewing but modification of
these aspects of the system since they can have side-effects on other aspects
of the system.
-
Rick Hayes-Roth's Roadmap presentation 5th foil from
the end on Information Assurance seems like it would be a nice addition
to the Security Architecture briefing. Similar or same foil in Signori's
briefing titled "Wrappers and Composition Assurance"
-
I'd like to get a copy of the Security RA Version
.6.
AITS
Architecture: Modeling and Simulation (M&S), Jack Kramer, 25 June
1997
-
OMG now has a Distributed Simulation SIG and is focused
on HLA. HLA is getting a lot of press -- there are many, many HLA
SBIRs in this round of SBIR funding and some push the HLA community toward
commercial partnerships. AITS might want to emulate this partnership
with industry to make M&S technology base more industry friendly.
NIMA is doing the same kind of thing.
-
as noted above, if the C4ISR and HLA communities
could use the same object models, much reuse might occur.
-
studying HLA, ALP, and other federation architectures
might pay off in helping to understand how to federate AITS. See
Federation Architectures.
AITS Communications Server
-
caveats: heard some of this but left midway -- did not get foils.
This is not really my area.
-
OMG has a Real-time SIG which just issued some RFPs and a Quality of Service
Working Group which is working on a White Paper for QoS. Gil
Hansen at OBJS has surveyed
some of the QoS work and can get you access to the OMG QoS White
Paper and also to ISO (Intl Stds Org) standard reference models on QoS.
You might want to compare your work with this other body of work.
IETF RSVP provides one QoS protocol for end-to-end QoS but not really for
how applications can control or negotiate QoS attributes. Ultimately,
in resource limited environments, management and not just competing users
must make decisions on who gets the bandwidth. Management might be
the guy with the most money in an economy-based scheme like the one described.
Routers are now providing class-based routing so there are more hooks to
work with than formerly. There are starting to be some Internet Weather
Services for measuring network activity. Finally, it seems to me
a major challenge to get bandwidth-adaptive apps since it is not just the
end-to-end comm layer that must be bandwidth adaptive but the programs
that operate in this environment. How is bandwidth-adaptivity built
in to AITS services and anchor desks?