Version 1.5 of ais/ai-00236.txt

Unformatted version of ais/ai-00236.txt version 1.5
Other versions for file ais/ai-00236.txt

!standard D (02)          00-11-19 AI95-00236/02
!class confirmation 00-05-09
!status WG9 approved 01-10-05
!status ARG Approved 8-1-0 00-11-19
!status work item 00-05-09
!status received 00-05-09
!priority Low
!difficulty Medium
!subject Minimum criteria for metrics documentation
!summary
There is no formal way to determine whether documentation is adequate.
!question
Is there any criteria that can be used to determine if a vendor has formally met the requirements to document metrics described in D(2-6)? (No.)
!response
D(2) defines Metrics as a documentation requirement.
There is no formal way to determine if a particular implementation meets a documentation requirement. These requirements can be provided in a natural language, a formal language, or any combination of them.
Thus, we have to leave meeting these requirements to the "market", that is to the individual implementors and their customers.
!appendix

From: Ken Garlington
Sent: Wednesday, May 03, 2000 7:23 PM

!topic Minimum criteria for metrics documentation
!reference RM95-D(2)
!from Author Name 00-05-03
!keywords metrics, documentation, real-time
!discussion

Is there any criteria that can be used to determine if a vendor has formally
met the requirements to document metrics described in D(2-6)? For example,
which (if any) of the following conditions would be considered acceptable?

1. The vendor states that the metrics are not currently available.

2. Same as #1, but vendor provides source code for its implementation of
interfaces to an underlying operating system.

3. Same as #2, but vendor explicitly states that source code is provided to
meet requirement.

4. Same as #3, but vendor provides a list of which source code files are
applicable.

****************************************************************

From: Robert Dewar
Sent: Monday, May 08, 2000 2:00 PM

Seems pretty clear to me that the requirement for metrics can be met
simply by suggesting that the user run appropriate test programs in
accordance with RM ..

19   The implementation may choose to document implementation-defined
behavior either by documenting what happens in general, or by providing some
mechanism for the user to determine what happens in a particular case.


The mechanism is pretty clear here I would say ... I don't think this
comment is worth much effort (for background, and a far more extensive
explanation of my position, see the current thread on comp.lang.ada).

****************************************************************

From: Ken Garlington
Sent: Friday, May 05, 2000 7:55 PM

!topic Non-Requirement Requirements
!reference RM95-1.1.2(24-39)
!from Ken Garlington 00-05-05
!keywords requirements, advise
!discussion

The referenced paragraphs define several different categories of the
standard. Some of these categories use the term "requirements," although
they are not considered requirements by the Ada vendor community. There
should be an explicit list of which categories can be considered
requirements (i.e., it is possible for a vendor to "fail" to meet it). All
other categories should be described in the terms currently used for
"Implementation Advice."

****************************************************************

From: Randy Brukardt
Sent: Tuesday, May 09, 2000 6:16 PM

> The referenced paragraphs define several different categories of the standard.
> Some of these categories use the term "requirements," although
> they are not considered requirements by the Ada vendor community. There
> should be an explicit list of which categories can be considered
> requirements (i.e., it is possible for a vendor to "fail" to meet it). All
> other categories should be described in the terms currently used for
> "Implementation Advice."

(Start editor's rant) I really hate it when people pose general questions, and
let us guess what it is that they are concerned about. I suppose that this
comment comes out of some other discussion that not everyone here has read.
Please be specific next time. (End editor's rant)

(Start editor's guess of commentor's question) Based on previous comments, I
would guess that you are referring to documentation requirements (and metrics,
which are defined to be documentation requirements in Annex D). (End editor's
guess)

(Start ACAA response to editor's guess of commentor's question. :-)
Documentation requirements are clearly requirements just like all other
requirements in the Ada reference manual. However, they are untestable, in that
they can be provided in virtually any form: one or more natural languages, one
or more formal languages (including Ada), or some combination.

Since the requirements are not formally testable (and indeed aren't tested, see
below), implementors have given these requirements lower priority than the
others in the standard. That's understandable: you can't fail conformity
assessment (validation) because of bad documentation, but you can fail because
of bad resolution (for example).

In the early days of Ada 83 validations, the AVO (Ada Validation Office)
required the testing laboratories to inspect the documentation (Appendix F) for
completeness (but not correctness, which is really beyond the capabilities of
the laboratories). This proved to be a major pain for everyone involved (in part
because not everyone organized their documentation into a giant Appendix F,
leading to lengthy arguments as to whether the documentation existed or not),
and eventually the requirement was dropped.

When the AVO turned the responsibility for maintaining the policies and test
suite over to the ACAA, I inherited this policy. I don't think there that any
significant benefit to users has been demonstrated as to why the policy ought to
be changed, but I (and the ACAA Advisory Board) are willing to listen to
suggestions. (End ACAA response)

			Randy Brukardt
			ARG Editor
			ACAA Technical Agent

****************************************************************

From: Robert Dewar
Sent: Tuesday, May 09, 2000 7:43 PM

Note that in the case of documentation requirements, Ada 95 is even
more permissive, because of RM 1.1.3(19)

19   The implementation may choose to document implementation-defined
behavior either by documenting what happens in general, or by providing some
mechanism for the user to determine what happens in a particular case.


This would appear to open the door for a claim that the compiler and the
generated code itself provide such a mechanism. This is surely true of
the metrics for example. And depending on what "determine" means, the
scope of para 19 is pretty broad. For example, can a user determine
dataflow information by examining the generated code. Most certainly
the answer is yes, so it would appear that if the generated code is
examinable, then this particular documentation requirement is automatically
met. I think the concern here is with accessibility, quality and
convenience of the documentation, but these are issues which even the
RM's documentation requirements do not try to address. I agree that no
action is required here.

(As I mentioned earlier, for more background, and to find out where this
 rather vague report is coming from, see the thread on CLA).

****************************************************************

From: Robert A Duff
Sent: Wednesday, May 10, 2000 11:23 AM

> (Start editor's rant) I really hate it when people pose general questions,
> and let us guess what it is that they are concerned about. I suppose that
> this comment comes out of some other discussion that not everyone here has
> read. Please be specific next time. (End editor's rant)

The background is: This comment comes out of a long discussion on
comp.lang.ada, mostly between Robert Dewar and Ken Garlington,
about the meaning of the Metrics, and also other Documentation Requirements.
I believe it was triggered by a statement in the GNAT documentation
that says information about some metric or other is "not yet available".
Ken was outraged that the GNAT folks could get away with such an
obvious violation of the RM requirements.  ACT ended up changing
their documentation to more clearly satisfy the requirements.

I answered Ken that, yes, the Doc Requirements are real requirements,
but we have only a vague idea of what they mean.  Therefore, the
validation process doesn't bother to test them.  If you think an
implementation violates a Doc Requirement, send in a bug report, and if
they agree, they will fix it.  In this case, ACT *did* fix it.  I'm not
sure why Ken wasn't satisfied with that answer.

I think the ARG should classify this AI as "!no action", and not waste
any more time on it.  If Ken doesn't agree with that, then let *him*
propose an answer -- I'm sure he will find that it's easy to say that
these are requirements, but impossible to say precisely what they mean.

If the ARG tried to precisely define what it means to satisfy a Doc Req,
the only possible answer it would come up with would be "totally
implementation defined".  But if you want the Doc Reqs to cause
implementers to do something, it's better to leave them vague than to
point out precisely how vacuous they really are.

(Actually, I guess I mean "totally implementation dependent" above, not
"totally implementation defined".  Otherwise I'm being circular!  ;-))

I think the Doc Reqs should never have been in the RM.  And I feel even
more strongly that way about the Metrics.  But now that they're there, I
don't see any harm in leaving these vague requirements alone.

****************************************************************

From: Robert A Duff
Sent: Wednesday, May 10, 2000 3:29 PM

> 19   The implementation may choose to document implementation-defined
> behavior either by documenting what happens in general, or by providing some
> mechanism for the user to determine what happens in a particular case.

I had a specific case in mind when I wrote that: The compiler is
supposed to document elaboration order.  Instead of documenting the
algorithm used by the binder, it seems better to have a switch that
prints out the order chosen for the particular program being linked.

I agree with Robert that this paragraph should be interpreted pretty
broadly.

****************************************************************

From: Ted Baker
Sent: Wednesday, May 10, 2000 5:04 PM

Subject: Re: [Ada-Comment] Non-Requirement Requirements

I do not propose that we add another burden to Ada implementors by
tightening validation testing to include verification that all
the documentation specified by the RM is provided.

Speaking only with regard to history, I believe it was intended in
Ada 9X that if a "mechanism" is provided to obtain the metrics and
to resolve implementation dependences it should be more than just
the compiler and generated code.  That is, if the mechanism is to
run some tests, then the user should not have to invent the test
method and write the code.  The test code should be provided,
along with instructions on how to use it to obtain the
information.

****************************************************************

From: Robert A Duff
Sent: Thursday, May 11, 2000 9:43 AM

Ted,

> Speaking only with regard to history, I believe it was intended in
> Ada 9X that if a "mechanism" is provided to obtain the metrics and
> to resolve implementation dependences it should be more than just
> the compiler and generated code.  That is, if the mechanism is to
> run some tests, then the user should not have to invent the test
> method and write the code.  The test code should be provided,
> along with instructions on how to use it to obtain the
> information.

Speaking informally, the above is perfectly reasonable.
Of course when we say you have to document certain things,
useless documentation violates the spirit of the law.

All I'm saying is that we don't know exactly where to draw the line, and
we shouldn't bother trying.

Here's an extreme analogy: The RM says it's up to the implementation how
you invoke the compiler.  Suppose the implementation says the way you
invoke the compiler is to first type in an Ada compiler written in C,
compile it, and then invoke it.  Legalistically, that's a conforming
implementation.  Informally and reasonably speaking, however, it's
completely bogus.

****************************************************************


From: Ken Garlington
Sent: Wednesday, May 10, 2000 6:21 PM

Robert Duff posted:

> Ken was outraged that the GNAT folks could get away with such an
> obvious violation of the RM requirements.

Actually, I stated several times that the question was not directed at any
vendor.

> ACT ended up changing
> their documentation to more clearly satisfy the requirements.

Irrelevant to the question, which would be closer to "Did they _need_ to
make any changes to satisfy the requirements?"

> I answered Ken that, yes, the Doc Requirements are real requirements,
> but we have only a vague idea of what they mean.  Therefore, the
> validation process doesn't bother to test them.  If you think an
> implementation violates a Doc Requirement, send in a bug report, and if
> they agree, they will fix it.  In this case, ACT *did* fix it.  I'm not
> sure why Ken wasn't satisfied with that answer.

Because it was irrelevant to the question -- see above.

> I think the ARG should classify this AI as "!no action", and not waste
> any more time on it.  If Ken doesn't agree with that, then let *him*
> propose an answer -- I'm sure he will find that it's easy to say that
> these are requirements, but impossible to say precisely what they mean.

Since I haven't requested a _precise definition_, this statement is
irrelevant.

If it is necessary for me to answer my own question, here's a start. Publish
an AI with the following text:

"The RM does not imply any minimum implementation criteria with respect to
the following categories:
* Implementation Requirements
* Documentation Requirements
* Metrics"

I suspect this is neither necessary nor sufficient, but if I had the _right_
answer, I really wouldn't need to ask the question.

As to classifying it as "no action," I assume that this has already
happened, and I'm just being humored. Seems like a waste of time, but I'm
still willing to discuss it as long as you want. I provided the requested
examples in a previous post, and now have proposed an answer. If you have
further questions, let me know.

> If the ARG tried to precisely define what it means to satisfy a Doc Req,
> the only possible answer it would come up with would be "totally
> implementation defined".

If everyone agrees to this, then this would be a perfectly satisfactory
answer to my question (rephrased: "Is there any way for a vendor to fail a
Doc Req?" "No".)

Are there other sections of the RM (or categories) which share this
attribute, beyond the ones I listed in my previous response?

Also, Ted Baker posted:

> I do not propose that we add another burden to Ada implementers by
> tightening validation testing to include verification that all
> the documentation specified by the RM is provided.

Neither do I (although this seems to be yet another popular redirection of
the question.)

(Although this didn't seem to help when I said it in comp.lang.ada:) Try
reading the question not as a "springboard" to something nefarious, like
bashing the language, or particular vendors, or changing the validation
suite, or rewriting/removing sections of the RM. Try taking it at face
value, and see if it makes more sense that way.

I'm also willing to stipulate that I'm an idiot who doesn't know what the RM
is for, or how to read it, etc., since that also seemed to be a prominent
feature in prior discussions.

> Speaking only with regard to history, I believe it was intended in
> Ada 9X that if a "mechanism" is provided to obtain the metrics and
> to resolve implementation dependences it should be more than just
> the compiler and generated code.  That is, if the mechanism is to
> run some tests, then the user should not have to invent the test
> method and write the code.  The test code should be provided,
> along with instructions on how to use it to obtain the
> information.

This would also be an appropriate (if partial) answer to my question, if
everyone agreed.

****************************************************************

From: Robert A Duff
Sent: Thursday, May 11, 2000 8:00 AM

> Robert Duff posted:
>
> > Ken was outraged that the GNAT folks could get away with such an
> > obvious violation of the RM requirements.
>
> Actually, I stated several times that the question was not directed at any
> vendor.

I apologize if I've misstated your point of view.  I thought the GNAT
docs were where you discovered the offending lack of documentation,
although I will freely admit that AverStar's documentation probably has
similar bugs.

> > ACT ended up changing
> > their documentation to more clearly satisfy the requirements.
>
> Irrelevant to the question, which would be closer to "Did they _need_ to
> make any changes to satisfy the requirements?"

The answer is clear: It is a matter of opinion whether they needed to.
Apparently, Robert Dewar thought it was a good idea, because he made the
change.

> > I answered Ken that, yes, the Doc Requirements are real requirements,
> > but we have only a vague idea of what they mean.  Therefore, the
> > validation process doesn't bother to test them.  If you think an
> > implementation violates a Doc Requirement, send in a bug report, and if
> > they agree, they will fix it.  In this case, ACT *did* fix it.  I'm not
> > sure why Ken wasn't satisfied with that answer.
>
> Because it was irrelevant to the question -- see above.

I don't see that.  To me, it seems to exactly answer your question: the
answer is that the requirements are vague, and each person can have
their own opinion as to whether the requirements are met.  That's is a
bit odd for a rule in a language standard, I admit.

> > I think the ARG should classify this AI as "!no action", and not waste
> > any more time on it.  If Ken doesn't agree with that, then let *him*
> > propose an answer -- I'm sure he will find that it's easy to say that
> > these are requirements, but impossible to say precisely what they mean.
>
> Since I haven't requested a _precise definition_, this statement is
> irrelevant.

Well, the ARG isn't (or shouldn't be) in the business of answering
random questions.  Their job is to interpret the RM -- ie define the
semantics of Ada in cases where the RM is unclear or wrong.  Just giving
a few examples ("Is it OK to produce the documentation in French, when
some compiler users speak only English?  Yes.  Is it OK to give the
documentation in Klingon?  No.  ...etc") doesn't define anything.

> If it is necessary for me to answer my own question, here's a start. Publish
> an AI with the following text:
>
> "The RM does not imply any minimum implementation criteria with respect to
> the following categories:
> * Implementation Requirements
> * Documentation Requirements
> * Metrics"

I think that sends the signal that implementations shouldn't bother to
do anything to meet these requirements.  I think it's better to have a
vague rule, than an explicit invitation to ignore the rule.  I think I
would vote against anything other than "!no action" on this issue.

Besides, most of the Implementation Requirements are precise and
testable.  Just because you found a vague one doesn't mean we should
throw out all similarly-labeled rules!

> I suspect this is neither necessary nor sufficient, but if I had the _right_
> answer, I really wouldn't need to ask the question.

The ARG doesn't have the "right" answer, either.  I asked you to come up
with an answer so you could see why.

> As to classifying it as "no action," I assume that this has already
> happened, and I'm just being humored.

The ARG meets only two or three times a year.  I suspect your question
won't even be put on the agenda for a couple of years, and then the ARG
will take two or three more meetings to deal with it.  The ARG is not
fast.  And frankly, this issue is pretty low priority.

I wasn't trying to "humor" you -- I was trying to get you to see that
it's impossible to come up with a satisfactory answer to your question.

>... Seems like a waste of time, but I'm
> still willing to discuss it as long as you want.

Hmm.  Not very long, I think.  ;-)

>... I provided the requested
> examples in a previous post, and now have proposed an answer. If you have
> further questions, let me know.
>
> > If the ARG tried to precisely define what it means to satisfy a Doc Req,
> > the only possible answer it would come up with would be "totally
> > implementation defined".
>
> If everyone agrees to this, then this would be a perfectly satisfactory
> answer to my question (rephrased: "Is there any way for a vendor to fail a
> Doc Req?" "No".)

But that's wrong.  You *can* fail to meet these requirements.  It's just
that the ARG can't tell you exactly when.  I'll know it when I see it.

> Are there other sections of the RM (or categories) which share this
> attribute, beyond the ones I listed in my previous response?

The Implementation Advice is where we *tried* to put all vague
(non-testable in a formal sense) rules.  But you've found some that
didn't get treated that way.  You might be able to find more -- most
likely in Chapter 13.  I can't get too excited about that -- we know the
RM is not perfect, and we try to fix the bugs that are most important to
users, and not worry too much about the rest.

> Also, Ted Baker posted:
>
> > I do not propose that we add another burden to Ada implementers by
> > tightening validation testing to include verification that all
> > the documentation specified by the RM is provided.
>
> Neither do I (although this seems to be yet another popular redirection of
> the question.)
>
> (Although this didn't seem to help when I said it in comp.lang.ada:) Try
> reading the question not as a "springboard" to something nefarious, like
> bashing the language, or particular vendors, or changing the validation
> suite, or rewriting/removing sections of the RM. Try taking it at face
> value, and see if it makes more sense that way.

At face value, the question seems pointless (to me).  That's probably
why people were trying to guess what agenda was behind it.  I mean,
usually people ask questions for *some* reason.

> I'm also willing to stipulate that I'm an idiot who doesn't know what the RM
> is for, or how to read it, etc., since that also seemed to be a prominent
> feature in prior discussions.
>
> > Speaking only with regard to history, I believe it was intended in
> > Ada 9X that if a "mechanism" is provided to obtain the metrics and
> > to resolve implementation dependences it should be more than just
> > the compiler and generated code.  That is, if the mechanism is to
> > run some tests, then the user should not have to invent the test
> > method and write the code.  The test code should be provided,
> > along with instructions on how to use it to obtain the
> > information.
>
> This would also be an appropriate (if partial) answer to my question, if
> everyone agreed.

The problem is that we don't know how to go from partial to complete.
And it seems pointless to give one example of how to violate the rules;
there is an infinite variety.

****************************************************************

From: Robert Dewar
Sent: Wednesday, May 10, 2000 7:47 PM

OI agree that !no action is the appropriate categorization here.

****************************************************************

From: Randy Brukardt
Sent: Thursday, May 11, 2000 5:28 PM

Ken and Robert inadvertently sent the following messages privately to me;
I'm reposting them so that they become part of the permanent record.

-----Original Message-----
From: Ken Garlington [mailto:Ken.Garlington@computer.org]
Sent: Tuesday, May 09, 2000 8:29 PM
Subject: RE: [Ada-Comment] Non-Requirement Requirements


The reason I was not specific in my earlier post was that I had been led to
believe that there was general agreement among implementers as to which
requirements, in essence, "had no wrong answer," as opposed to requirements
that imposed some constraint on the implementer. Requirements that constrain
the implementer have been previously defined as those that define "formal
semantics", for example.

I'm not sure why testability gets wrapped into this discussion, but if a
"test" is necessary to motivate the discussion, I would propose the
following:

Input: Someone says, "Vendor ABC does not implement ARM section w.z.y(z) per
the standard, and therefore cannot claim conformance to the ARM (or a
particular annex)."

Output: After some review (I assume by the ACAA), the answer will be either
"It does conform" or "It does not conform" (or, possibly, some third
option?)

[Note that I usually refer to this as an "inspection," rather than a "test,"
but I gather that "test" is used differently in this context.]

What I am looking for is those sections for which the test output will
ALWAYS be "It does conform," regardless of the characteristics of the
implementation (that is, it is "vacuously" true).

Clearly, sections labeled as "Erroneous Execution," "Implementation
Permissions," and "Implementation Advice" would fall into this category,
based on the description in the ARM. I am led to believe that most (all?) of
the sections titled "Documentation Requirements" and "Metrics" also fall
into this category, although I have not seen that formally stated anywhere.
I also suspect that certain (all?) sections labeled "Implementation
Requirements" (e.g. for pragma Reviewable) may also fall into this category.

An Ada Interpretation characterizing the ARM in this manner would be quite
useful for those of us reading the manual, so that users such as myself will
not be misled into assumptions of the word "requirement" that are not
supported by the Ada community (see below). It could either be done by the
categories in 1.1.2, or by specific sections, or by another means easily
grasped by users.

I also request that a consideration for Ada0X include incorporating this
list into the RM, preferably by using the word "requirement" only in those
cases where it is possible to "get a wrong answer." I don't have Webster’s
Third New International Dictionary of the English Language handy, but my
dictionary includes a definition of "requirement" as "to impose a compulsion
or command on." I'd argue that if there "is no wrong answer," then nothing
has been imposed.

---------

From: Robert Dewar [mailto:dewar@gnat.com]
Sent: Tuesday, May 09, 2000 9:11 PM
Subject: RE: [Ada-Comment] Non-Requirement Requirements


<<The reason I was not specific in my earlier post was that I had been led
to
believe that there was general agreement among implementers as to which
requirements, in essence, "had no wrong answer," as opposed to requirements
that imposed some constraint on the implementer. Requirements that constrain
the implementer have been previously defined as those that define "formal
semantics", for example.
>>

I encourage everyone to look at the CLA thread in question, because although
Ken kept saying he had been "led to believe that there was general ..."
no such statement was made by me or anyone else (in fact I went to long
though clearly unsuccessful efforts to clear up this misconception).

I think it is important for Ken to be QUITE specific in the set of
requirements he is talking about. I don't see any practical way of
us trying to guess the set of requirements that he has in mind. Ken
please give a list of RM references for requirements that you think
should be addressed here.

---------

From: Ken Garlington [mailto:Ken.Garlington@computer.org]
Sent: Tuesday, May 09, 2000 10:28 PM
Subject: RE: [Ada-Comment] Non-Requirement Requirements


Since the _question_ is: "Which specific requirements meet" the criteria I
gave in the previous post, it seems odd to demand that I provide the answer
before I get a response! In particular, if there's not agreement on what
constitutes an answer among people who are the experts, why would I be
expected to have it? (By the way, "There's no consensus on this issue" would
certainly be an adequate response, assuming it's been discussed in the
past.)

However, if you need some _examples_, I suppose I could oblige, although I'm
pretty much shooting in the dark for some categories. Here's an example from
each of the three categories I assume are the most problematic:

* Metric:

"The implementation shall document... An upper bound on the real-time
duration of a clock tick."

* Documentation Requirement:

"The implementation shall document the properties of the underlying time
base used for the clock and for type Time, such as the range of values
supported and any relevant aspects of the underlying hardware or operating
system facilities used."

* Implementation Requirement:

"The implementation shall provide control- and data-flow information, both
within each compilation unit and across the compilation units of the
partition."


None of the other categories use the term "requirement," so I suppose
technically they don't fall under my submission. However, assuming they do,
here's a list. I'm assuming that the specific conditions being considered in
the "test" are something other than is explicitly included in the
conformance test.

* Bounded (Run-Time) Error:

"If one name denotes a part of a formal parameter, and a second name denotes
a part of a distinct formal parameter or an object that is not part of a
formal parameter, then the two names are considered distinct access paths.
If an object is of a type for which the parameter passing mechanism is not
specified, then it is a bounded error to assign to the object via one access
path, and then read the value of the object via a distinct access path,
unless the first access path denotes a part of a formal parameter that no
longer exists at the point of the second access (due to leaving the
corresponding callable construct). The possible consequences are that
Program_Error is raised, or the newly assigned value is read, or some old
value of the object is read."

* Dynamic Semantics:

"Any pragma that appears at the place of an executable construct is
executed. Unless otherwise specified for a particular pragma, this execution
consists of the evaluation of each evaluable pragma argument in an arbitrary
order."

* Post-compilation Rule:

"Pragma Reviewable is a configuration pragma. It applies to all
compilation_units included in a partition."

* Static Semantics:

"If a pragma Inline applies to a callable entity, this indicates that inline
expansion is desired for all calls to that entity."

* Legality Rule:

"An object_declaration without the reserved word constant declares a
variable object."

* Name Resolution Rule:

"A subtype_mark shall resolve to denote a subtype."

* Syntax:

2.1(3) - for example, a character reserved for a control function is treated
as a graphic character.

Again, the test is: "Is there any possible set of circumstances under which
an implementation would fail to meet the 'requirement'?"

I'm sure you guys could come up with better examples from each category...

****************************************************************

From: Robert Dewar
Sent: Thursday, May 11, 2000 6:46 PM

<<I apologize if I've misstated your point of view.  I thought the GNAT
docs were where you discovered the offending lack of documentation,
although I will freely admit that AverStar's documentation probably has
similar bugs.>>

Please do not assume I think there are any bugs in GNAT, similar or
otherwise :-)

<<The answer is clear: It is a matter of opinion whether they needed to.
Apparently, Robert Dewar thought it was a good idea, because he made the
change.>>

Just so people know the *massive* change we made, it was to add the
word principle in front of a sentence that said something like

The GNAT documentation is found [principly] in the xyz and zzz manuals.

Hardly an earthshaking change. Robert Dewar thought it was a good idea?
I would not go so far as that, that's giving FAR too much emphasis to
a fairly useless, but harmless change. Still if it makes one person
out there more comfortable, why not?

****************************************************************

From: Ken Garlington
Sent: Thursday, May 11, 2000 7:27 PM

Replying to the latest from Robert Duff:

> > > I answered Ken that, yes, the Doc Requirements are real requirements,
> > > but we have only a vague idea of what they mean.  Therefore, the
> > > validation process doesn't bother to test them.  If you think an
> > > implementation violates a Doc Requirement, send in a bug
> > > report, and if
> > > they agree, they will fix it.  In this case, ACT *did* fix
> > > it.  I'm not
> > > sure why Ken wasn't satisfied with that answer.
> >
> > Because it was irrelevant to the question -- see above.
>
> I don't see that.  To me, it seems to exactly answer your question: the
> answer is that the requirements are vague, and each person can have
> their own opinion as to whether the requirements are met.  That's is a
> bit odd for a rule in a language standard, I admit.

Note that your last answer _is_ an answer to my question. I would restate it
as "each person can have their own opinion as to whether the requirements
are met" as "there is no accepted minimum criteria for meeting section xxx."
Your original answer said that the validation process doesn't bother to test
them. Is it then fair to say that all requirements that are not tested by
the validation suite should not be expected to have any generally-accepted
minimum criteria (even if that criteria is as vague as a "moral"
responsibility)?

> > > I think the ARG should classify this AI as "!no action", and not waste
> > > any more time on it.  If Ken doesn't agree with that, then let *him*
> > > propose an answer -- I'm sure he will find that it's easy to say that
> > > these are requirements, but impossible to say precisely what they mean.
> >
> > Since I haven't requested a _precise definition_, this statement is
> > irrelevant.
>
> Well, the ARG isn't (or shouldn't be) in the business of answering
> random questions.  Their job is to interpret the RM -- ie define the
> semantics of Ada in cases where the RM is unclear or wrong.

Notes: My dictionary does not define "imprecise" as a synonym for "random".

It _does_ define "semantic" as "of or related to meaning in language". So if
I ask for the meaning of the term "requirement", and that term is unclear in
the RM, it seems to be within the scope of the ARG.

If you're restricting "semantics" to "those concepts that can be readily
expressed in VDM," then clearly my question is out of scope.

> Just giving
> a few examples ("Is it OK to produce the documentation in French, when
> some compiler users speak only English?  Yes.  Is it OK to give the
> documentation in Klingon?  No.  ...etc") doesn't define anything.

OK, so define it by category.

> > If it is necessary for me to answer my own question, here's a start.
> > Publish an AI with the following text:
> >
> > "The RM does not imply any minimum implementation criteria with respect to
> > the following categories:
> > * Implementation Requirements
> > * Documentation Requirements
> > * Metrics"
>
> I think that sends the signal that implementations shouldn't bother to
> do anything to meet these requirements.  I think it's better to have a
> vague rule, than an explicit invitation to ignore the rule.  I think I
> would vote against anything other than "!no action" on this issue.

Of course, the correct answer to my question would include the premise that
there is *no rule to be ignored*, but if you see an advantage in keeping
things intentionally obfuscated, I suppose you need to press on with that
policy. It does seem like a strange policy for a group whose charter is "to
interpret the RM -- ie define the semantics of Ada in cases where the RM is
unclear..."

> Besides, most of the Implementation Requirements are precise and
> testable.  Just because you found a vague one doesn't mean we should
> throw out all similarly-labeled rules!

This part answered (in advance) in the very next sentence:

> > I suspect this is neither necessary nor sufficient, but if I had the
> > _right_ answer, I really wouldn't need to ask the question.

So, remove Implementation Requirements from the list, and add "and possibly
some sections titled Implementation Requirements," if you believe that some
of the requirements of this category can in fact be failed.

> The ARG doesn't have the "right" answer, either.  I asked you to come up
> with an answer so you could see why.

Then simply say, "The ARG is not able to determine which parts of the RM
meet the conditions in the question."

If you don't know, you don't know. Seems like this would be a potential
problem in meeting the ARG charter, but I suppose a "don't ask, don't tell"
policy works just as well.

> But that's wrong.  You *can* fail to meet these requirements.  It's just
> that the ARG can't tell you exactly when.  I'll know it when I see it.

This would also be a perfectly acceptable answer to my question, i.e.
"Although the ARG cannot state the specific conditions under which a vendor
could fail to meet the sections categorized as {list here}, and in fact was
not asked to do so, there would be cases in practice which would be
considered non-conforming for these sections."

> > (Although this didn't seem to help when I said it in comp.lang.ada:) Try
> > reading the question not as a "springboard" to something nefarious, like
> > bashing the language, or particular vendors, or changing the validation
> > suite, or rewriting/removing sections of the RM. Try taking it at face
> > value, and see if it makes more sense that way.
>
> At face value, the question seems pointless (to me).  That's probably
> why people were trying to guess what agenda was behind it.  I mean,
> usually people ask questions for *some* reason.

Assuming that you're asking me why I ask the question -- which no one has
bothered to ask to date, preferring to explain to me why I'm asking -- my
reasons are quite simple: I don't want to think that someone is
non-conforming when they aren't. I also don't want to have false
expectations when I read the RM.

I am from time to time asked to explain parts of the RM. As I noted before,
clearly I'm not qualified to do so, but for some reason people ask anyway.
Therefore, when I see something I don't understand, I try to get the answer.
I have posted questions on comp.lang.ada in the past of this type, and I
usually get an answer.

****************************************************************

From: Erhard Ploedereder
Sent: Thursday, May 18, 2000 5:57 PM

Ken writes:
> Is it then fair to say that all requirements that are not tested by
> the validation suite should not be expected to have any generally-accepted
> minimum criteria (even if that criteria is as vague as a "moral"
> responsibility)?

No. That's not fair to say. For one thing, the ACATS fails to test many
requirements -- that it's the nature of a test suite that needs to be
economically feasible. And as to documentation requirements, please let's
not even think about asking validators for subjective evaluations of
documentation quality. (Language lawyers are bad enough, but please no
real lawyers in this process!)

At a minimum, the doc. requirements in the RM are a reasonably good
checklist for implementers and their documentation teams. If one wanted
to make them more precise by describing lower bounds, you'd need a second
standard twice as thick to account for all eventualities. And if you go for
the lowest denominator of all eventualities, you'd really weaken rather than
strengthen the requirements.

At best, you'd hope that implementers discharge their obligation to
satisfy the doc. requirements as good as they can with finite resources.
The RM puts this "moral obligation" on them; it can't do more without
completely undue complexity.

P.S. I also vote for !no action.

****************************************************************



Questions? Ask the ACAA Technical Agent