ACATS 4.1 User's Guide
Section 1: Introduction
The Ada Conformity Assessment Test Suite (ACATS)
provides the official tests used to check conformity of an Ada implementation
with the Ada programming language standard (ANSI/ISO/IEC
and later corrigenda, including ANSI/ISO/IEC
). The ACATS User's Guide is part of the ACATS
and is distributed with the test programs and testing support packages.
It explains the contents and use of the test suite.
The ACATS is an important part of the conformity
assessment process described in ISO/IEC-18009, Ada: Conformity of a Language
. This standard provides
a framework for testing language processors, providing a stable and reproducible
basis for testing. The Ada Resource Association (ARA) has sponsored an
instantiation of that process since October 1998. The process is managed
by the Ada Conformity Assessment Authority (ACAA).
Prior to the ISO standard, the U.S. Department of
Defense sponsored a similar conformity assessment process under the Ada
Joint Program Office (AJPO). The test suite for that process was known
as the Ada Compiler Validation Capability (ACVC).
The AJPO developed ACVC versions based on ANSI/MIL-STD-1815A-1983,
(Ada 83), which were numbered 1.x where x ranged from
1 to 11. It later developed ACVC versions based on ANSI/ISO/IEC 8652:1995
), numbered 2.0, 2.0.1, 2.1, and
When the ACAA took over Ada conformity assessment,
it adopted the ACVC as the basis for its test suite. The ACAA determined
to continue to use the same version numbering for the test suite in order
to avoid confusion. The version of the ACVC current at the time (2.1)
was initially used as ACATS 2.1. Later, the already developed but unreleased
ACVC 2.2 was released and used as ACATS 2.2. The ACAA later released
ACATS 2.3, ACATS 2.4, ACATS 2.5, and then ACATS 2.6 to include maintenance
changes and a few new tests.
In 2007, the ACAA developed ACATS version 3.0 to
check for conformity to new language features defined in ISO/IEC 8652:1995/AMD
), as well as test programs
to check for conformity to language features defined in earlier versions
of Ada, including [Ada95]
The ACAA later released ACATS 3.1 to improve the coverage and correctness
of the ACATS for features defined in [Amend1]
In 2014, the ACAA developed ACATS version 4.0 to
test the enhancements and changes of the third edition of the Ada Standard,
ISO/IEC 8652:2012 ([Ada2012]
). This version
of the ACATS, version 4.1, was released to improve the coverage and correctness
of the ACATS for features defined in [Ada2012]
including features originally defined in earlier versions of the Ada
Standard. Subsequent maintenance or enhancement versions of the suite,
if they are required, will be numbered 4.2, etc.
The ACATS User's Guide describes the set of ACATS
tests and how they are to be used in preparation for conformity assessment.
The formal procedures for conformity assessment are described in [Pro31]
and the rules in that document govern all conformity assessments, notwithstanding
anything in this document that may be interpreted differently. Moreover,
this guide does not discuss specific requirements on processing of the
ACATS test suite, or submission and grading of results that an Ada Conformity
Assessment Laboratory (ACAL) may impose.
The User's Guide is intended to be used by compiler
implementers, software developers who maintain a version of the ACATS
as a quality control or software acceptance tool, and third-party testers
(e.g., Ada Conformity Assessment Laboratories).
of the User's Guide
for ACATS 4.1 summarizes the changes between ACATS 4.0 and ACATS 4.1.
describes test objectives and their
relationship to ACATS tests and to the rules of the Ada Standards documents.
describes the configuration of the ACATS,
including a description of the ACATS software and delivery files. Section
provides step-by-step instructions for installing
and using the test programs and test support packages, and for grading
test results. The appendices include other information that characterizes
the ACATS 4.1 release, along with information on test construction.
Refer to Annex F
for the definition of an acceptable result
and the rules for grading ACATS 4.1 test program results. Section 5.7.2
provides instructions for submitting a petition against a test program
if a user believes that a deviation from the acceptable results for a
given test program is in fact conforming behavior.