TSIMMIS Project: Status Report

1. Overview of Current Prototype
   
   • INSPEC, BIBLIO, and SYBASE wrappers
   • Mediator connecting to BIBLIO and INSPEC
   • "Synthetic" mediator connecting two mediators acting as sources (using data from INSPEC and BIBLIO)
   • MOBIE client
   • Next major milestone (end of August):
     • stable version of improved wrapper-generatorfully functional wrappers for SYBASE, BIBLIO, INSPEC
     • stable and functional mediator specification interpreter consisting of view expander, optimizer, and execution enginemediator using real wrappers as sources, synthetic mediator

2. Status of Individual Components
   We are now where we wanted to be about 6 months ago. MSL is our internal language our wrappers support the same syntax that is emitted by the mediators, LOREL is user query language, post-processing in wrappers (making available a large class of "logically supported queries"), "user-friendly" templates written as MSL rules, object fusion in mediators.
   1. Wrapper
      • New (more modular) design with well-defined components: driver, parser, matcher, execution engine, and native component (generating a wrapper requires only changes to the native component!)
      • Support for directly supported queries, indirectly supported queries, and logically supported queriesprevious version only handled directly and some indirectly supported queries
      • Cleaner templates using MSL syntax, separation of template rules and schema
      • Diffferent views of the same source
      • Reuse of mediator execution engine:
        • post-processing: given a filter and the results of the source, filters out the objects that do not match the predicates in the filter
        • several build-in predicates (eq, ne, gt, lt, contains, etc.)
      • Total of appr. 5000 lines of mostly C++ code not counting the components that are shared by the mediator (engine: 4000, parser 1260)
        Driver: 25
        Matcher: 2275
        INSPEC: 675
        BIBLIO: 500
        SYBASE: 650
      • Marcus, Svetlozar

   2. Mediator
      • Generates and uses objet-IDs
      • New version of our mediators feature object fusion (based on object-id); no object fusion possible with previous execution engine (see below)
      • Updated query decomposition module: unification for MSL patterns
      • New version of query optimizer (join ordering based on cost information about the sources)
      • New, more reliable execution engine:
        • executes a query plan (generated by the optimizer)
        • fuses objects based on OIDs
        • returns the answer of the query
      • The mediator specification interpreter currently consists of roughly 7500 lines of (mostly) C++ code
        Engine: 4000
        Parser: 1250
      • Yannis, Ramana

   3. MOBIE
      • New, re-designed look-and-feel: three frames that handle query input, result output, as well as controls (i.e., FAQ, formatting, navigation buttons)
      • Streamlined the way users can change the formatting of results
      • Ability to input MSL as well as LOREL queries-MOBIE will automatically convert LOREL queries into an equivalent MSL query by invoking a translator (Jun Yang)
      • Total of appr. 4500 lines of C code
      • Andy

      Total lines of TSIMMIS code: 6000 (SSL/CSL) + 4500 (MOBIE) + 5000 (wrappers) + 7500 (mediator spec. interpreter) = 23,000

3. General
   • Introducing software engineering methodologies into the design & implementation process
     • better documentation, better organization code versioning tool, etc.
     • increased communication among team members (e.g., regular meetings, write-ups)
     • code reviews
     
     FYI, I have compiled a document describing the TSIMMIS development process…
   • Less than a month away from reaching a major milestone (code freeze)
   • Moving into phase II of the Tsimmis project: DARPA contract officially ends this summer, Dave Gunning has promised us additional funding , focus on tech transfer (AFV Project with TASC, Global InfoTech) and various joint projects with ISX, Lockheed (SILK proposal). The important goal of DARPA is to "ruggedize" our code and insert into BADD.

4. Future Plans
   1. Status Information
      Each server has a corresponding "status server", server and "status server" communicate via file (for now), MOBIE connects to both.
      This work is done by Rohan

   2. "Bi-lingual" Components
      Every component in Tsimmis can accept either MSL or LOREL queries. Need to build MSL2LOREL translator (Jun)

   3. Wrapper for US Marshalls Web Site
      We are currently building a wrapper for a WWW site (US Marshalls-15 Most Wanted Criminals) that will be used by ISX in a prototype for a commercial product (ISLE project). Arturo has build a configurable Python script that extracts the desired information form specified HTML pages. The output of Arturo's script is a file that is used as a source to our wrapper (much like the existing BIBLIO wrapper).
      This work is done by Arturo (script) and Andy (wrapper)

   4. Transition from CSL/SSL to CORBA
      Out of our own frustration with the existing communications protocol and due to pressure from our funding agencies we have decided to move to CORBA.
      • Based on some implementation of CORBA (ORBIX, ILU, etc.)
      • Simpler than CSL/SSL (fetch operations)
      • Includes server for continuous queries
      • Based on DL libraries protocol
      • Need to develop together with partners (ISX, Lockheed, etc.)

        Task force: Joachim, Ramana, Tom, Roy

   5. Additional Functionalities
      • Continuous queries: "run this query once a day and send me the new changes"
      • Client caching
      • etc.