ACATS 3.1 User's Guide
3.2.2 General Coverage Guidelines
As noted in Section 1.2
the ACATS strives for complete coverage of the Ada standard. However,
complete coverage does not mean that every sentence in the standard has
an associated test objective and test. Not all rules are testable; one
important way to determine testability of a rule is to check its category.
The next clause (3.2.3
) will discuss how the
category of a rule affects its mapping to test objectives.
There are also additional considerations that don't
apply to specific rule categories.
Text sometimes contains definitions (especially in
Static Semantics rules). Definitions are hard to handle, because they
usually are not testable by themselves. The definition has to be used
in another rule to make them visible. For instance, a categorization
rule such as "something is either this, that, or fuzzy" cannot
be tested by itself. "Something" has to be used in some other
rule in order for the categorization to matter. That being the case,
it usually makes sense to test the definition as part of the other rule(s).
Such rules are marked as having a kind of "Widely Used" or
"Subpart" in the coverage documentation. "Subpart"
rules are tested as part of specific other objectives, while "Widely
Used" rules are thought to be tested by many tests indirectly (and
no attempt is made to verify that). In some cases, especially those that
otherwise would have a combinatorial explosion, it may make more sense
to test the definition as directly as possible (in which case objectives
will be assigned to the definition).
Text sometimes includes sentences marked as redundant
by the AARM. Such sentences should be given normatively elsewhere, and
the testing should be done in the place where they are normatively given.
The AARM often will indicate this place.
Combinational explosion is always a problem for visibility
rules. Rules that define the scope or visibility of something should
always be tested in place (even though other rules will need to be used
to accomplish that).