ACATS 4.0 User's Guide
Section 1: Introduction
The Ada Conformity Assessment Test Suite (ACATS)
provides the official tests used to check conformity of an Ada implementation
with the Ada programming language standard (ANSI/ISO/IEC
and later corrigenda). The ACATS User's Guide is part of
the ACATS and is distributed with the test programs and testing support
packages. It explains the contents and use of the test suite.
The ACATS is an important part of the conformity
assessment process described in ISO/IEC-18009, Ada: Conformity of a Language
. This standard provides
a framework for testing language processors, providing a stable and reproducible
basis for testing. The Ada Resource Association (ARA) has sponsored an
instantiation of that process since October 1998. The process is managed
by the Ada Conformity Assessment Authority (ACAA).
Prior to the ISO standard, the U.S. Department of
Defense sponsored a similar conformity assessment process under the Ada
Joint Program Office (AJPO). The test suite for that process was known
as the Ada Compiler Validation Capability (ACVC).
The AJPO developed ACVC versions based on ANSI/MIL-STD-1815A-1983,
(Ada 83), which were numbered 1.x where x ranged from
1 to 11. It later developed ACVC versions based on ANSI/ISO/IEC 8652:1995
), numbered 2.0, 2.0.1, 2.1, and
When the ACAA took over Ada conformity assessment,
it adopted the ACVC as the basis for its test suite. The ACAA determined
to continue to use the same version numbering for the test suite in order
to avoid confusion. The version of the ACVC current at the time (2.1)
was initially used as ACATS 2.1. Later, the already developed but unreleased
ACVC 2.2 was released and used as ACATS 2.2. The ACAA later released
ACATS 2.3, ACATS 2.4, ACATS 2.5, and then ACATS 2.6 to include maintenance
changes and a few new tests.
In 2007, the ACAA developed ACATS version 3.0 to
check for conformity to new language features defined in ISO/IEC 8652:1995/AMD
), as well as test programs
to check for conformity to language features defined in earlier versions
of Ada, including [Ada95]
The ACAA later released ACATS 3.1 to improve the coverage and correctness
of the ACATS for features defined in [Amend1]
This version of the ACATS is 4.0. It was developed
in 2014 to test the enhancements and changes of the third edition of
the Ada Standard, ISO/IEC 8652:2012 ([Ada2012]
Subsequent maintenance or enhancement versions of the suite, if they
are required, will be numbered 4.1, etc.
The ACATS User's Guide describes the set of ACATS
tests and how they are to be used in preparation for conformity assessment.
The formal procedures for conformity assessment are described in [Pro31]
and the rules in that document govern all conformity assessments, notwithstanding
anything in this document that may be interpreted differently. Moreover,
this guide does not discuss specific requirements on processing of the
ACATS test suite, or submission and grading of results that an Ada Conformity
Assessment Laboratory (ACAL) may impose.
The User's Guide is intended to be used by compiler
implementers, software developers who maintain a version of the ACATS
as a quality control or software acceptance tool, and third-party testers
(e.g., Ada Conformity Assessment Laboratories).
of the User's Guide
for ACATS 4.0 summarizes the changes between ACATS 3.1 and ACATS 4.0.
describes test objectives and their
relationship to ACATS tests and to the rules of the Ada Standards documents.
describes the configuration of the ACATS,
including a description of the ACATS software and delivery files. Section
provides step-by-step instructions for installing
and using the test programs and test support packages, and for grading
test results. The appendices include other information that characterizes
the ACATS 4.0 release, along with information on test construction.
Refer to Annex F
for the definition of an acceptable result
and the rules for grading ACATS 4.0 test program results. Section 5.7.2
provides instructions for submitting a petition against a test program
if a user believes that a deviation from the acceptable results for a
given test program is in fact conforming behavior.