ACATS 4.0 User's Guide
5.2.5 Allowed Test Modifications
Class B tests have one or more errors that implementations
must identify. These tests are structured such that, normally, implementations
can report all included errors. Occasionally, an implementation will
fail to find all errors in a B-test because it encounters a limit (e.g.,
error cascading, resulting in too many error reports) or is unable to
recover from an error. In such cases, a user may split a single B-test
into two or more tests. The resulting tests must contain all of the errors
included in the original test, and they must adhere as closely as possible
to the style and content of the original test. Very often, the only modification
needed is to comment out earlier errors so that later errors can be identified.
In some cases, code insertion will be required. An implementation must
be able to demonstrate that it can detect and report all intended
Splits may also be required in executable tests,
if, for example, an implementation capacity limitation is encountered
(e.g., a number of generic instantiations too large for the implementation).
In very exceptional cases, tests may be modified by the addition of an
attribute definition clause (to alter the default size of a collection),
or by the addition of an elaboration Pragma (to force an elaboration
Tests that use configuration pragmas (see 188.8.131.52
may require modification since the method of processing configuration
pragmas is implementation dependent.
Some tests include foreign language code (Fortran,
C, or COBOL). While the features used should be acceptable to all Fortran,
C, and COBOL implementations, respectively, some implementations may
require modification to the non-Ada code. Modifications must, of course,
preserve the input-output semantics of the (foreign language) subprogram;
otherwise, the ACATS test will report a failure.
All splits and modifications must be approved in
advance by the ACAL (and, if necessary, the ACAA) before they are used
in a conformity assessment. It is the responsibility of the user to propose
a B-test split that satisfies the intention of the original test. Modified
tests should be named by appending an alphanumeric character to the name
of the original test. When possible, line numbers of the original test
should be preserved in the modification.
All tests must be submitted to the compiler as distributed
(and customized, if required). If a test is executable (class A, C, D,
E) and compiles successfully, then it must be run. Modified tests or
split tests may be processed next. Only the results of the modified tests
will be graded.
If the ACAA has issued an ACATS Modification List
(see Section 5.1.1
), then the modified versions
of tests with modifications must
Either the original version or the modified version of a test with an
allowed modification may be used.