The tremendous research investment the government has made in Model-Based Design and Dynamic Real-time Embedded Systems (DRE's) will not be fully realized until engineers and researchers are able to obtain and build tool-chains suitable for their application domain. This critical step is being addressed by means of the ESCHER tool repository. Most of the tools have been created under government funding and many are offered under some form of open source license. A much bigger issue than tool availability, however, is tool quality.

In this context, the term quality is being used loosely to describe the complete set of characteristics a tool must exhibit to make it a useful tool chain component. Eight criteria have been initially identified as a working set for the purpose of preliminary tool evaluation.

For a list of frequently asked questions, please see the Tool Qualification FAQ.

For information about what ISIS (a tool developer) is doing towards ESCHER compliance, click here.

Intellectual Property (IP) Rules

Tools in ESCHER Repository satisfy the following IP baseline:

1. Unrestricted use (in research and/or in product development) company wide.
2. Open source licensing model

In general, the licensing model chosen by the tools must satisfy the following requirements:

1. Disclaim liability for other's use of OSS
2. Free redistribution of OSS as part of aggregate
3. Source code made available
4. Derived work allowed and freely distributable subject to Trademark restrictions
5. Derived works may be required to have different name than the original OSS
6. Costs of distribution may be charged
7. Access without discrimination
8. OSS license must not restrict software distributed by licensee along with OSS

A tool developer may choose from one of the accepted open source software (OSS) licensing models as recommended by Open Source (www.opensource.org). Examples for these models are the following:

1. Academic Free License
2. BSD
3. Common Public License (CPL)

However, a tool using a licensing model that is a viroid, such as the GNU General Public License (GPL), will be excluded from the Repository.

Tool Dependencies

Tools should explicitly disclose all dependencies on other tools, software and platforms. This disclosure should contain enough information to enable the user to determine implications of using a tool vis-à-vis:

1. The user should be fully informed of any copyright issues involved in using a tool because of its dependencies. Also, usage of a tool should not change the user's rights to the system that the user is developing.
2. If a tool uses proprietary software, user should be made aware of any licensing costs associated with that software.
3. The user should be made aware of any platform compatibility issues.
4. The number of other tools that will need to be installed and the effort and skills required.
5. The hardware, networking, etc. configuration and resources needed.

Functional Integrity

A tool must demonstrate its functionality and claimed features. For this, the tools developers must make available:

1. Functional specifications (or logical design) and any other documents that clearly outline the features and expected behavior of the tool and its interfaces.
2. Test suites for the tool. The test suites may consist of automated test scripts/executables and manual test scripts.
3. The developer will be responsible for testing the system and generating reports about features tested/passed.

ESCHER, or anyone else, should be able to use the test suites to verify the developer's claims. Feedback from the user community will be used to adjust the rating.

Integratability

Tools in the Repository will be required to contain interfaces that conform to the following tool integration frameworks: 

1. OTIF: The Open Tool Integration Framework (OTIF) is software infrastructure for building tool integration solutions. OTIF provides a meta-model driven infrastructure for design tool integration, which facilitates the semantic interoperability across the elements of a tool chain. OTIF is based on open standards such as OMG's MOF and CORBA/IDL and is itself an open standard.
2. Eclipse: Eclipse is an open source software development project dedicated to providing a robust, full-featured, commercial-quality, industry platform for the development of highly integrated tools. It provides a focal point for diverse tool builders to ensure the creation of best of breed tools for the Eclipse Platform and provides new channels for open source developers, researchers, academics and educators to participate in the on-going evolution of Eclipse.

The tools must also include test suites, as for Functional Integrity criteria, to determine compliance. The developers should

1. Clearly state the specific interfaces that the tool conforms to. A tool may conform to only a subset of the interfaces that OTIF and Eclipse define. The tool will be evaluated for successful implementation of these interfaces only .
2. Provide test suites for the interfaces. The test suites may consist of automated test scripts/executables and manual test scripts.
3. The developer will be responsible for testing the interfaces and generating reports about the test results.

ESCHER, or anyone else, should be able to use the test suites to verify the developer's claims. Feedback from the user community will be used to adjust the rating.

Documentation 

The tools in the Repository must provide the following documentation:

1. User Manuals
2. Reference Manuals

The above two artifacts should be made available in paper, electronic and online forms.

User Support Criteria 

A tool should provide artifacts that aid the user, such as:

1. Download and versioning facilities, such as a CVS Repository for the tool.
2. Discussion boards/mailing lists with active participation by the tool developers.
3. Bug Reporting mechanisms, e.g., Bugzilla. The bug reports must be monitored and responded to by the developer. The developers must publish plans for addressing the bugs and incorporate the bug fixes in periodic, scheduled releases and hotfix releases as needed.
4. Mechanisms for submitting features requests (could be part of the bug reporting system). Feature requests should be handled in the same manner as bug reports, with possibly lower priority.
5. Tutorials (print media, electronic and/or wed-based)
6. Training
7. FAQs
8. Online Help Development

QA Processes

The development processes used for tool development must be transparent and must employ some commonly used QA procedures and mechanisms, such as:

1. A well defined process for requirements gathering, design, development, testing and stabilization of the tool, with specific milestones for each stage.
2. Clear separation of conceptual, logical and physical design.
3. Coding standards.
4. Code reviews.
5. Daily builds.
6. Testing and stabilization.

Domain Applicability 

This criterion will classify the tools according to the domain and problems that they are applicable to. The tools will be evaluated using demonstrated application (through test suites, etc.) to challenge and benchmark problems.