Version 1.2 of acs/ac-00010.txt

Unformatted version of acs/ac-00010.txt version 1.2
Other versions for file acs/ac-00010.txt

!standard 7.6.1 (16)          01-09-05 AC95-00010/01
!class confirmation 01-09-05
!status received no action 01-08-29
!subject Adjust Storage_Error gets converted to Program_Error
!summary
!appendix

From: Victor Porton
Sent: Wednesday, August 29, 2001  4:37 PM

As a consequence from the following rule (7.6.1):

"It is a bounded error for a call on Finalize or Adjust to propagate an
exception.
...
For an Adjust invoked as part of an assignment operation, any other
adjustments due to be performed are performed, and then Program_Error is
raised."

It is natural for Adjust of, to say, a linked list to allocate memory through
an allocator and raise Storage_Error. This Storage_Error converts to
Program_Error, which is an extremely wrong behavior. (Conceptually, an
out-of-memory isn't a Program_Error!).

So, the Standard should be changed about this (Assignment operations should
not convert other exceptions to "Program_Error"s) even despite of, that such
change introduces a backward incompatibility.

But we do not have (at least, I cannot find) any reasonable simple way to
finalize correctly an object, which is _partially_ adjusted. :-~ So, the only
choice I see, is to redesign Ada.Finalization completely: obsolete "Adjust"
(It was a bad idea.) and add instead:

procedure Initialize_With(out Object: Controlled; in Value: Controlled);

Explicit initialization of controlled objects should be by calling this
procedure: "A: T:=B;" (where T is a controlled type) should call
"Inititialize_With(Object=>A, Value=>B)".

A:=B should be made equivalent (if A and B do not have common parts) to

Finalize(A);
Inititialize_With(Object=>A, Value=>B);

Well, for backward compatibility we need to define "Inititialize_With" in
terms of "Adjust":

procedure Initialize_With(out Object: Controlled; in Value: Controlled) is
begin
    ... -- do memcpy() of Value to Object
    Adjust(Object);
end;

What you think about my idea? Are there any other ways to fight with nasty
"Storage_Error->Program_Error" effect?

****************************************************************

From: Robert Dewar
Sent: Wednesday, August 29, 2001  7:13 PM

We cannot even vaguely consider non-upwards compatible changes in this area,
so you need to reformulate your ideas in an upwards compatible form to even
be seriously considered. At least that's my opinion!

****************************************************************

From: Pascal Leroy
Sent: Thursday, August 30, 2001  4:29 AM

> So, the only
> choice I see, is to redesign Ada.Finalization completely: obsolete "Adjust"
> (It was a bad idea.)

As Robert pointed out, such a big incompatibility is out of the question.  At
this point, incompatibilities are only acceptable if (1) they are extremely
unlikely to occur in practice and (2) they are fixing a blatant hole in the
language, which compromises the safety of applications.  See AI 229 for a recent
example of this.

Now for the substance of the comment:

> It is natural for Adjust of, to say, a linked list to allocate memory through
> an allocator and raise Storage_Error. This Storage_Error converts to
> Program_Error, which is an extremely wrong behavior. (Conceptually, an
> out-of-memory isn't a Program_Error!).

I must say that I have little sympathy for exceptions flying out of Adjust or
Finalize.  If I were running the circus, this situation would merely be
erroneous, so that we wouldn't even have to discuss where/how/when P_E is
raised.  In the example at hand, I don't see why you don't handle S_E in Adjust,
and try to do something sensible (eg free some storage, or undo the partial work
done by Adjust, or mark the object as broken) rather than letting it fly out and
compromise the entire application.

> But we do not have (at least, I cannot find) any reasonable simple way to
> finalize correctly an object, which is _partially_ adjusted.

I don't quite understand that sentence.  Adjust and Finalize are both written by
the user.  It is the user's responsibility to write Adjust in such a way that
the object is left in a reasonable state, so that Finalize can correctly process
that same object later on.  If that means that Adjust has to handle various
exceptions and record in the object which parts were processed and which parts
weren't, then so be it.

> ... and add instead:
>
> procedure Initialize_With(Object: out Controlled; Value: in Controlled);

Even if this worked (and I don't think it does; see AARM 7.6(17.a-h)) I don't
understand how it would help.  Surely the code in Initialize_With could raise
S_E, and that exception would be turned into P_E in the caller.  And surely
Object would be "partially adjusted" (to follow your terminology) in this case
and Finalize would have to cope with it when it is ultimately called on Object.

****************************************************************

From: Robert Dewar
Sent: Thursday, August 30, 2001  4:03 PM

I entirely agree with Pascal's analysis here.

****************************************************************

From: Victor Porton
Sent: Thursday, August 30, 2001  7:26 PM

> > So, the only
> > choice I see, is to redesign Ada.Finalization completely: obsolete "Adjust"
> > (It was a bad idea.)
>
> As Robert pointed out, such a big incompatibility is out of the question.  At
> this point, incompatibilities are only acceptable if (1) they are extremely
> unlikely to occur in practice and (2) they are fixing a blatant hole in the
> language, which compromises the safety of applications.  See AI 229 for a recent
> example of this.

Probably you have not understood me correctly: I propose only obsolete (but
not remove) "Adjust". The only backward incompatibility introduced is that
Program_Error is not raised (but an other exception propagates). But program
error is (meant to) indicating an erroneous programs. So, programs which do
not indicate themselves as erroneous remain compatible.

The question is whether it is worth of backward incompatibility with such
strange programs which indicate themselves as erroneous (by raising
Program_Error from Adjust), but are really not erroneous (The Program_Error is
intended behavior.) and also incompatibility with non well written programs
which for example consider out-of-memory as an error in the program itself.
(Well, I understand, after my change such the programs would begin function
incorrectly, if out-of-memory, instead of just terminating :-( )

> Now for the substance of the comment:
>
> > It is natural for Adjust of, to say, a linked list to allocate memory through
> > an allocator and raise Storage_Error. This Storage_Error converts to
> > Program_Error, which is an extremely wrong behavior. (Conceptually, an
> > out-of-memory isn't a Program_Error!).
>
> I must say that I have little sympathy for exceptions flying out of Adjust or
> Finalize.  If I were running the circus, this situation would merely be
> erroneous, so that we wouldn't even have to discuss where/how/when P_E is
> raised.  In the example at hand, I don't see why you don't handle S_E in Adjust,
> and try to do something sensible (eg free some storage, or undo the partial work
> done by Adjust, or mark the object as broken) rather than letting it fly out and
> compromise the entire application.

If we handle Storage_Error inside Adjust then after the assignment A:=B (where
A and B are linked lists) we may get that A/=B (immediately after the
assignment!) A not enough careful programmer may assume that after the
assignment it is always A=B. So handling of Storage_Error inside Adjust is
also error-prone. (Oh, it seems that there are non-error prone way of dealing
with exceptions in assignments at all!!)

Well, we may mark the object as broken and cause any further operation (even
equality) with it to raise an exception... and cause the user to wonder why an
error-dialog related with out of memory is displayed when 200 Mb of memory are
free (at the moment of the first using of assigned value, which was after a
hour of the unsuccessful assignment) :-~

Well, why you dislike exceptions from assignments? (Certainly, I say only
about the case of a well written program which finalizes a partially assigned
object correctly; see also below.)

> > But we do not have (at least, I cannot find) any reasonable simple way to
> > finalize correctly an object, which is _partially_ adjusted.
>
> I don't quite understand that sentence.  Adjust and Finalize are both written by
> the user.  It is the user's responsibility to write Adjust in such a way that
> the object is left in a reasonable state, so that Finalize can correctly process
> that same object later on.  If that means that Adjust has to handle various
> exceptions and record in the object which parts were processed and which parts
> weren't, then so be it.

Components of a composite object are adjusted in an arbitrary order. So, if
one of "Adjust"s of _components_ raises an exception, we do not know, which of
them raised it. So, we do not know, which of them must be finalized and which
must be not finalized. (Yes, this problem can be solved by not using certain
kinds of controlled components (but using only such components which do not
change any other objects (such as results of allocators) during finalization)
in composite objects. But it is obviously very much too restrictive. Also,
even if we agree to follow this restriction, we must be very careful to not
violate the restriction unintendedly.)

> > ... and add instead:
> >
> > procedure Initialize_With(Object: out Controlled; Value: in Controlled);
>
> Even if this worked (and I don't think it does; see AARM 7.6(17.a-h)) I don't
> understand how it would help.  Surely the code in Initialize_With could raise
> S_E, and that exception would be turned into P_E in the caller.  And surely
> Object would be "partially adjusted" (to follow your terminology) in this case
> and Finalize would have to cope with it when it is ultimately called on Object.

I was not enough clear in explanation of the idea. I imply that _two_ changes
should (probably) be done:

1. Propagate exceptions through Adjust unchanged.

2. Introduce Initialize_With instead of Adjust. (It solves problem with
partially Adjusted objects, which I explained above. That it really solves it,
is seen from, that such the problem is relatively easily solvable in C++,
where user defined assignment (and copy constructors) are more like to my
Initialize_With than to Adjust. (Well, Initialize_With is a direct analog of
C++-ish copy constructor.))

Using this approach we can easily know (as oppose to the current Ada) in
Finalize which components of the assignment target was assigned before
throwing exception and which was non-assigned because of exception
(non-assigned components remain in finalized state (I mean the state just
after a finalization)).

P.S. Certainly, my idea is only for the future Ada Standard, not for changing
the current one.

****************************************************************

From: Robert Dewar
Sent: Thursday, August 30, 2001  8:08 PM

When Pascal suggested handling storage_error, he was, I assume, expecting
that once you have handled storage error, you indeed raise Program_Error
to indicate that the assignment failed (program error does not indicate
an erroneous execution, contrary to the claim made). You can attach an
appropriate message to the exception if needed.

I don't see a problem here, and consequently I don't see the need for a
complex solution. Can you point to a real application in which this has
proved problematic.

****************************************************************

From: Randy Brukardt
Sent: Thursday, August 30, 2001  8:48 PM

> Probably you have not understood me correctly: I propose only obsolete (but
> not remove) "Adjust". The only backward incompatibility introduced is that
> Program_Error is not raised (but an other exception propagates). But program
> error is (meant to) indicating an erroneous programs. So, programs which do
> not indicate themselves as erroneous remain compatible.

I realize you are new here, but what you are proposing is too large an
incompatibility for this group. Many good ideas have died here on the altar
of compatibility -- please take Pascal and Robert seriously.

We discussed issues surrounding the raising of Program_Error when we were
working on the Technical Corrigendum, and it was decided that it was
important enough to retain and test in the ACATS. Thus all Ada compilers are
mandated to do it, and user could write code that depends on it.

> If we handle Storage_Error inside Adjust then after the assignment A:=B
> (where A and B are linked lists) we may get that A/=B (immediately after the
> assignment!) A not enough careful programmer may assume that after the
> assignment it is always A=B. So handling of Storage_Error inside Adjust is
> also error-prone. (Oh, it seems that there are non-error prone way of dealing
> with exceptions in assignments at all!!)

The model for Ada 95 is the Finalize and Adjust do not propagate exceptions.
The Program_Error exists only because *something* has to happen. As Pascal
says, it probably was overspecification, but we're stuck with it. But in any
case, writing code to *depend* on that is really wrong.

The reason the rules are written the way they are is to prevent a broken
abstraction (one that propagates an exception from Adjust and/or Finalize)
from breaking any other unrelated abstraction. This is a critical property
of Ada 95, which must not be compromised in any way.

> Well, we may mark the object as broken and cause any further operation (even
> equality) with it to raise an exception... and cause the user to wonder why
> an error-dialog related with out of memory is displayed when 200 Mb of memory
> are free (at the moment of the first using of assigned value, which was
> after a hour of the unsuccessful assignment) :-~

I really do not see this problem. We've never felt in Claw that we needed to
propagate an exception from Adjust. Claw contains around 100 controlled
objects, most of which support assignment and have Adjusts.

> Well, why you dislike exceptions from assignments? (Certainly, I say only
> about the case of a well written program which finalizes a partially
> assigned object correctly; see also below.)

That's the model. It is perfectly OK to say you don't like the model, but
that alone is unlikely to gain much sympathy. The reason it is the model is
to avoid breaking other abstractions.

> Components of a composite object are adjusted in an arbitrary order. So,
> if one of "Adjust"s of _components_ raises an exception, we do not know,
> which of them raised it. So, we do not know, which of them must be
> finalized and which must be not finalized. (Yes, this problem can be
> solved by not using certain kinds of controlled components (but using
> only such components which do not change any other objects (such as results
> of allocators) during finalization) in composite objects. But it is
> obviously very much too restrictive. Also, even if we agree to follow this
> restriction, we must be very careful to not violate the restriction
> unintendedly.)

I think you completely miss the point. If you have a set of unrelated
controlled components, and the Adjust of one of them raises an exception,
ALL of the rest of them still will be Adjusted. (That's the meaning of the
"other adjustments due to be performed" part of the paragraph.) There is no
"partial" Adjustment in that case.

The only component that could be "partially" Adjusted is the one that raised
the exception. But the writer of that Adjust has complete control over how
that is handled, and simply should write their code to prevent problems. As
Pascal said, if this means putting an exception handler around every line of
code, tough. (There is a lot of that in the Claw Adjust routines.)

> > > ... and add instead:
> > >
> > > procedure Initialize_With(Object: out Controlled;
> > >     Value: in Controlled);
>
> I was not enough clear in explanation of the idea.

The idea was discussed in great detail before Adjust was introduced. Please
read AARM 7.6(17.a-h), here's a link to it:
www.adaic.org/standards/95aarm/html/AA-7-6.html

>I imply that _two_ changes should (probably) be done:
>
> 1. Propagate exceptions through Adjust unchanged.

This would introduce a large can of worms. Consider an assignment of a
record with two controlled components:

    record
	A : Cont1;
	B : Cont2;
    end record;

If the Adjust of A raises an exception, the implementation has to defer that
exception until the Adjust of B can be called and completes. (The main
reason that Program_Error is raised is so that the implementation doesn't
have to save the identity of the exception raised.)

But what happens if BOTH Adjusts raise an exception (two different
exceptions)? Which one is propagated? (They both cannot be.) If we don't
specify which one is propagated, we haven't helped anything. But we've
previously said that the order of Adjusts is unspecified. So we would have
to change that behavior and possibly introduce another incompatibility.

> 2. Introduce Initialize_With instead of Adjust. (It solves problem with
> partially Adjusted objects, which I explained above. That it really solves
> it, is seen from, that such the problem is relatively easily solvable in C++,
> where user defined assignment (and copy constructors) are more like to my
> Initialize_With than to Adjust. (Well, Initialize_With is a direct analog
> of C++-ish copy constructor.))

But it doesn't work in Ada (see the above mentioned reference). Moreover,
the model that all Adjusts are called would have to be carried over, and as
Pascal says, the whole issue reappears precisely as it was.

> Using this approach we can easily know (as oppose to the current Ada) in
> Finalize which components of the assignment target was assigned before
> throwing exception and which was non-assigned because of exception
> (non-assigned components remain in finalized state (I mean the state just
> after a finalization)).

We don't need to know this: in the current Ada, they all will be assigned
(except for those in the failing Adjust call). What's the problem??

> P.S. Certainly, my idea is only for the future Ada Standard, not for
> changing the current one.

Well, the future standard will be arrived at by changing the current one, so
I don't quite see the difference....

****************************************************************

From: Victor Porton
Sent: Friday, August 31, 2001  2:50 PM

Hm, I just found a serious mistake in the Standard (see below).

[discussion about backward compatibility skipped]

I understand that a user may write a code which depend on this. But (as
somebody other pointed above) such code is a bad programming style.

> The model for Ada 95 is the Finalize and Adjust do not propagate exceptions.
> The Program_Error exists only because *something* has to happen. As Pascal

It is very wrong model:

1. Accordingly to 11.1 _any_ construct may raise Storage_Error. So, _any_
Adjust may raise S_E (Even one with null body. And even one whose body have
the handler for S_E, because this handler may raise S_E itself!) So _every_
assignment operator of a controlled type may raise Program_Error! Hm, it was
certainly not the intention of the committee. It is even a security hole in
language: any memory-intensive program which assigns a value of a controlled
type and does not check for P_E around such assignments (even if it handles
Strorage_Error and even if the Adjust does not allocate any memory explicitly)
may crash with P_E. Now anybody here must agree that we need to somehow change
the language on this.

2. See below.

> says, it probably was overspecification, but we're stuck with it. But in any
> case, writing code to *depend* on that is really wrong.

Exactly my point! If a code depends on this it is wrong. So, changing it we
may break only wrong code.

> The reason the rules are written the way they are is to prevent a broken
> abstraction (one that propagates an exception from Adjust and/or Finalize)
> from breaking any other unrelated abstraction. This is a critical property
> of Ada 95, which must not be compromised in any way.

Certainly I understand the reason why the rules are such. But in reality they
do not prevent broken abstractions:

These rules simply force a programmer which deal with memory allocation (and
even does not deal with memory allocation: see above about "any construct"),
while copying something, to use not assignment operators, but something other
(such as user-defined "procedure Assign(out Target: T; in Source: T)") for
copying. This moves the problem from broken assignment operators to broken
user-defined "Assign" procedures and so absolutely does not make the situation
better.

The model of user-defined assignments in Ada95 should be changed because it is
absolutely useless. (As I proved above, _any_ user-defined assignment is in
wrong.)

> I really do not see this problem. We've never felt in Claw that we needed to
> propagate an exception from Adjust. Claw contains around 100 controlled
> objects, most of which support assignment and have Adjusts.

What you do in the case of out-of-memory in Adjust?

> That's the model. It is perfectly OK to say you don't like the model, but
> that alone is unlikely to gain much sympathy. The reason it is the model is
> to avoid breaking other abstractions.

As I shown above it does not avoid it!

> I think you completely miss the point. If you have a set of unrelated
> controlled components, and the Adjust of one of them raises an exception,
> ALL of the rest of them still will be Adjusted. (That's the meaning of the
> "other adjustments due to be performed" part of the paragraph.) There is no
> "partial" Adjustment in that case.

Yes, this was my error... but many other arguments for changing the thing
remain...

[skip about Initialize_With]

> This would introduce a large can of worms. Consider an assignment of a
> record with two controlled components:
>
>     record
> 	A : Cont1;
> 	B : Cont2;
>     end record;
>
> If the Adjust of A raises an exception, the implementation has to defer that
> exception until the Adjust of B can be called and completes. (The main
> reason that Program_Error is raised is so that the implementation doesn't
> have to save the identity of the exception raised.)
>
> But what happens if BOTH Adjusts raise an exception (two different
> exceptions)? Which one is propagated? (They both cannot be.) If we don't
> specify which one is propagated, we haven't helped anything. But we've
> previously said that the order of Adjusts is unspecified. So we would have
> to change that behavior and possibly introduce another incompatibility.

If we introduce Initialize_With, these rule of language should be somehow
changed (making them like to C++-ish): If Initialize_With of a component
raises an exception, other components of the object should be not
"initialized-with".

****************************************************************

From: Victor Porton
Sent: Friday, August 31, 2001  2:12 PM

> When Pascal suggested handling storage_error, he was, I assume, expecting
> that once you have handled storage error, you indeed raise Program_Error
> to indicate that the assignment failed (program error does not indicate
> an erroneous execution, contrary to the claim made). You can attach an
> appropriate message to the exception if needed.

You misunderstood me: I haven't said that Program_Error indicates erroneous
execution. I say only that P_E is meant to be used to indicate an error in the
program itself, not in the environment.

> I don't see a problem here, and consequently I don't see the need for a
> complex solution. Can you point to a real application in which this has
> proved problematic.

Client Side SSI (see my signature :-) ) It is written in C++, but if we
replace std::bad_alloc<->Storage_Error and std::logic_error<->Program_Error,
we would get the analogous situation in Ada.

(As every similar program which deals well with out-of-memory) CSSSI handles
out-of-memory exceptions which arrive during executing of the operation
(conversion of several files SHTML->HTML) which happens when a user presses
"Convert!" button, shows an error dialog and continues to work (does not exit).

It is implemented by raising an out-of-memory exception by every operation
(including possibly an assignment) for which the memory is not enough. The
out-of-memory exception (std::bad_alloc) is handled by the subprogram which is
executed in response of pressing "Convert!" button. But if the conversion
engine would raise std::logic_error (sorry, Program_Error...) on an
out-of-memory, then it would be not handled by this exception handler but
would cause the program to terminate with an error message.

What to do in Ada case (when assignments throw Program_Error's if out of
memory)? Certainly it can be done in some ways...

1. Putting around every assignment, which may call an "Adjust" throwing
Storage_Error, inside such an exception handler which will convert
Program_Error _back_ (Top of stupidness!) to Storage_Error.

2. Handling Program_Error in the subprogram which is called on "Convert!"
button in the same way as Storage_Error is handled... and losing the
reliability, because we will more not correctly act with "real"
"Program_Error"s (for example, ones arriving as result of an erroneous
execution or as result of incorrect accessibility level of an access value).

3. Not using assignments for types which allocate memory dynamically at all,
but declare all such types as limited and doing assignment by user-defined
"Assign" procedure instead of the assignment operator. (See also my reply to
Randy.)

... but no of these ways are good. So, it is needed, as you say, a "complex
solution".

****************************************************************

From: Randy Brukardt
Sent: Friday, August 31, 2001  4:27 PM

> What to do in Ada case (when assignments throw Program_Error's if out of
> memory)? Certainly it can be done in some ways...
>
> 2. Handling Program_Error in the subprogram which is called on "Convert!"
> button in the same way as Storage_Error is handled... and losing the
> reliability, because we will more not correctly act with "real"
> "Program_Error"s (for example, ones arriving as result of an erroneous
> execution or as result of incorrect accessibility level of an
> access value).

This is precisely the correct solution, although it requires some care in
debugging and designing the handler.

The reason is that the user of your application does not care why the
application stopped working. She doesn't care if the application raised
Storage_Error, Program_Error, Constraint_Error, or Bumbleshoot_Error. All
she knows is that her file didn't get converted.

Thus all exceptions should be converted into an appropriate error box. The
details of the exception should be made available somewhere (behind the
scenes, preferably). Exception_Information can be useful for this task (how
useful depends on the compilers). Thus, the information is available if
needed for debugging, but in general, the user only sees an appropriate
level of information. For debugging, you also might want to write a log.

Essentially, that's how our web server works. It just logs exceptions and
recycles to accept the next command. (The web server runs the AdaIC archive
site (archive.adaic.com)). When I recently went on vacation right after
making it live, it logged four separate crashes. But it kept on running the
whole time, and was still working fine when I returned.

****************************************************************

From: Victor Porton
Sent: Saturday, September 1, 2001  1:16 AM

> The reason is that the user of your application does not care why the
> application stopped working. She doesn't care if the application raised
> Storage_Error, Program_Error, Constraint_Error, or Bumbleshoot_Error. All
> she knows is that her file didn't get converted.

She care: her reaction on "Out of memory" and on "Internal error in the
program. Please report bug." should be very different.

So, it isn't a 100% correct method.

> Thus all exceptions should be converted into an appropriate error box. The

Huh? Should End_Of_File be converted into an error box? Recovering from errors
and debugging are not the only uses of exceptions. You probably too much
follow the usual pattern of handling exceptions, when they may be reasonably
handled in variety of ways.

Well, we are almost in offtopic. Probably you should not answer to this
message.

****************************************************************

From: Randy Brukardt
Sent: Wednesday, September 5, 2001  7:16 PM

> She care: her reaction on "Out of memory" and on "Internal error in the
> program. Please report bug." should be very different.

I disagree. Either way, her file didn't get converted. I know that my
reaction to a commercial program that refuses to do something reasonable
(for any reason) is unpleasant.

OTOH, the developer may care, but that is handled best (IMHO) by using
better algorithms so it is unlikely that you would run out of memory.

> So, it isn't a 100% correct method.
>
> > Thus all exceptions should be converted into an appropriate error box.
>
> Huh? Should End_Of_File be converted into an error box? Recovering from
> errors and debugging are not the only uses of exceptions. You probably
> too much follow the usual pattern of handling exceptions, when they may be
> reasonably handled in variety of ways.

Yes, at the GUI level, End_Error should be converted into an error box.
Internal to the code, you might handle it and do something else, but if it
propagates to the outer GUI level, it's just as much of a bug as a
Program_Error or Tasking_Error. It means someone read past the end of a file
without taking the correct precautions (which might just be handling the
error). One of the bugs in the web server I mentioned previously was exactly
that (raising End_Error when it tried to send an empty file in response to a
request). Certainly, I much preferred handling the error and logging it to
letting the server crash.

> Well, we are almost in offtopic. Probably you should not answer to this
> message.

I don't think that we're off-topic yet. If the reason someone wants an
change to the language is bogus, then we don't need to consider the change.
Anyway, some of us always want the last word... :-)

****************************************************************

From: Victor Porton
Sent: Thursday, September 6, 2001  5:03 AM

> OTOH, the developer may care, but that is handled best (IMHO) by using
> better algorithms so it is unlikely that you would run out of memory.

I am not sure whether memory needed by any memory consuming algorithm may be
very decreased changing the algorithm. (This seems to be something similar to
yet unsolved by mathematicians P=NP problem.) But I can be sure in one thing:
the cost of decreasing of used memory may be decreasing of speed. So, do not
say: we will not change language, better you create better algorithms.

****************************************************************

From: Randy Brukardt
Sent: Friday, August 31, 2001  4:16 PM

> > I really do not see this problem. We've never felt in Claw that we needed
> > to propagate an exception from Adjust. Claw contains around 100 controlled
> > objects, most of which support assignment and have Adjusts.
>
> What you do in the case of out-of-memory in Adjust?

We don't allocate memory in Adjust. It's not necessary. A lot of the causes
of trying to allocate memory there happen because someone copied a
heap-based C++ style object design into Ada, rather than taking advantage of
the fact that objects in Ada do not need to be allocated on the heap.

Some objects do have heap-based portions; these are allocated when the
object is created (or Initialized).

> [skip about Initialize_With]
...
> If we introduce Initialize_With, these rule of language should be somehow
> changed (making them like to C++-ish): If Initialize_With of a component
> raises an exception, other components of the object should be not
> "initialized-with".

We're not likely to introduce Initialize_With, unless someone can address
the technical problems noted in the AARM. You're simply ignoring the
technical problems -- you have to solve them before there is any point of
even talking about that solution. (And many people tried to solve them for
Ada 95, and failed...)

> The model of user-defined assignments in Ada95 should be changed because
> it is absolutely useless. (As I proved above, _any_ user-defined assignment
> is in wrong.)

Only if the program runs out of memory. Very few correct programs run out of
memory. The point is that Ada defines what happens when a program runs out
of memory. It is possible that it won't be possible to recover. Handling
Storage_Error is an art, and is very compiler-specific. And that standard
can't help that, because what it means to run out of memory is very compiler
and target specific.

****************************************************************

From: Victor Porton
Sent: Friday, August 31, 2001  5:04 PM

> We don't allocate memory in Adjust. It's not necessary. A lot of the causes

_Linked lists_ and similar (binary trees, sets etc.)!

> Only if the program runs out of memory. Very few correct programs run out of
> memory. The point is that Ada defines what happens when a program runs out
> of memory. It is possible that it won't be possible to recover. Handling
> Storage_Error is an art, and is very compiler-specific. And that standard
> can't help that, because what it means to run out of memory is very compiler
> and target specific.

What?! It is compiler specific? It was absolutely unknown to me :-) I always
used to solve it in a compiler independent way like this:

procedure On_A_Menu_Item is
begin
    A_Memory_Consuming_Operation;
exception
    when Storage_Error =>
        MessageBox("Sorry, out of memory.");
end On_A_Menu_Item;

Well, I understand that on _very_ low memory any program crashes (or hangs).
But I speak about the case of low (allowing only part of functionality of a
program) but not extremely low (not allowing running of a program at all)
memory.

I absolutely do not understand saying "standard can't help that".

I insist that (at least) we should change the Standard in such a way that it
will be impossible that the assignment A:=B will raise P_E in the following
procedure:

procedure May_Crash_Program is
    type Looking_Harmless is
        new Ada.Finalization.Controlled with null record;
    A, B: Looking_Harmless;
begin
    A := B; -- may raise Program_Error (and crash the program even if it
            -- handles Storage_Error gracefully)!
end May_Crash_Program;

(I retell, that because _any_ construct may cause S_E, even the null statement
in the default Adjust may raise it.)

Does somebody have an idea how exactly to change the Standard to solve this
last problem (with procedure May_Crash_Program)? (Apparently, we need some
kind of (about 100 bytes) memory reserve for Adjust... But currently I have no
idea how to formalize it.)

****************************************************************

From: Randy Brukardt
Sent: Wednesday, September 5, 2001  7:50 PM

> > We don't allocate memory in Adjust. It's not necessary. A lot of the
>
> _Linked lists_ and similar (binary trees, sets etc.)!

We link the objects together using 'Unchecked_Access. We don't need to
allocate anything.

> > Only if the program runs out of memory. Very few correct programs run
> > out of memory. The point is that Ada defines what happens when a program
> > runs out of memory. It is possible that it won't be possible to recover.
> > Handling Storage_Error is an art, and is very compiler-specific. And that
> > standard can't help that, because what it means to run out of memory is
> > very compiler and target specific.
>
> What?! It is compiler specific? It was absolutely unknown to me :-) I
> always used to solve it in a compiler independent way like this:
>
> procedure On_A_Menu_Item is
> begin
>     A_Memory_Consuming_Operation;
> exception
>     when Storage_Error =>
>         MessageBox("Sorry, out of memory.");
> end On_A_Menu_Item;

No, you are talking about HANDLING out of memory. I was talking about what
CAUSES running out of memory. That is completely compiler and target
specific. For instance, on the MS-DOS and Windows 3.1 Janus/Ada compilers,
the above message box call would have to allocate a copy of the string
constant on the heap (because the parameter pointer [a near pointer]
couldn't point at the constant area [in the code, not data, segment]). But
of the heap is out of memory, you're toast. Those implementations also used
a heap and stack that grew toward each other, so that when one ran out, so
did the other. Thus, handling Storage_Error was difficult.

Similarly, many targets pass parameters on the stack, so passing the string
parameter might raise Storage_Error if the stack is full. Some compilers
even need to allocate memory to handle an exception (to create an
Exception_Occurrence).

The problem for users is that exactly what raises Storage_Error is
compiler-specific. Thus, the only possible defensive strategy is to handle
Storage_Error at the outermost level possible (so that stack unwinding makes
some room), and to realize that it isn't possible to do it completely
reliably.

> Well, I understand that on _very_ low memory any program crashes (or
> hangs). But I speak about the case of low (allowing only part of
> functionality of a program) but not extremely low (not allowing running
> of a program at all) memory.

You're talking in language that is not appropriate to a language standard.
We don't get to talk about "kinda low memory" here; we have to describe when
an exception is raised without discussing the actual implementation.

> I absolutely do not understand saying "standard can't help that".
>
> I insist that (at least) we should change the Standard in such a way that
> it will be impossible that the assignment A:=B will raise P_E in the
> following procedure:
>
> procedure May_Crash_Program is
>     type Looking_Harmless is
>         new Ada.Finalization.Controlled with null record;
>     A, B: Looking_Harmless;
> begin
>     A := B; -- may raise Program_Error (and crash the program even if it
>             -- handles Storage_Error gracefully)!
> end May_Crash_Program;
>
> (I retell, that because _any_ construct may cause S_E, even the null
> statement in the default Adjust may raise it.)

Yes, and that seems absolutely necessary. On most targets, a subprogram call
pushes something (parameters, return address) onto the stack, and if the
stack is full, what else could possibily happen? Of course Storage_Error is
going to be raised.

Probably the only thing in the entire Ada language that could be guaranteed
not to raise Storage_Error is the null statement, but what would the point
of saying that be? You can't write anything useful out of the null statement
alone. Even an integer math operation like divide might be implemented with
a subprogram call to a library routine. (Our ancient CP/M implementation for
the Z-80 had to do this, because the Z-80 didn't have a divide instruction.)
And *ANY* subprogram call has a (very small) risk of raising Storage_Error.

> Does somebody have an idea how exactly to change the Standard to solve
> this last problem (with procedure May_Crash_Program)? (Apparently, we need
> some kind of (about 100 bytes) memory reserve for Adjust... But currently I
> have no idea how to formalize it.)

I don't think that there is a solution to this problem. It certainly was
considered before the "anything can raise Storage_Error" rule was adopted.
And it comes about because any subprogram call can raise Storage_Error, and
almost any Ada construct could be implemented with a subprogram call (and
the language's framers did not want to constrain implementations enough to
avoid it).

As a practical matter, its not worth worrying about. Just don't assume you
know exactly where an exception is raised (or could be raised) [good policy
in any case]. Any programming language that supports subprogram calls on the
Pentium (for example) will have the same problem -- it just might not be
willing to admit it. I've spent a lot of mental energy on this problem in
the context of Janus/Ada and the Pentium processor (a much more restricted
environment than the Ada standard has to offer), and I haven't found anyway
to bulletproof the implementation enough to be able to write a handler for
Storage_Error that would always work. The best I could do was to insure that
the first handling would be successful -- but that provides no help for long
running programs.

****************************************************************

From: Robert Dewar
Sent: Wednesday, September 5, 2001 11:34 PM

<<procedure On_A_Menu_Item is
begin
    A_Memory_Consuming_Operation;
exception
    when Storage_Error =>
        MessageBox("Sorry, out of memory.");
end On_A_Menu_Item;>>

This is obviously highly dubious code. To handle storage error in the
same scope where it is raised if the storage error comes from a stack
overflow is obviously unlikely to work (i.e. the call to MessageBox
may well reraise SE).

****************************************************************

From: Victor Porton
Sent: Thursday, September 6, 2001  4:54 AM

> Yes, and that seems absolutely necessary. On most targets, a subprogram call
> pushes something (parameters, return address) onto the stack, and if the
> stack is full, what else could possibily happen? Of course Storage_Error is
> going to be raised.

It seems, that in whatever reason, most members of this list think that I am
stupid. It is not true. I understand that _calling_ of Adjust may raise S_E,
but I say not about calling of Adjust (which we may considers as a part of the
assignment operation itself, not as part of Adjust). In such the case we can
guaranty that Adjust itself which (conceptually) consist of only null operator
will not raise exceptions.

> I don't think that there is a solution to this problem. It certainly was
> considered before the "anything can raise Storage_Error" rule was adopted.
> And it comes about because any subprogram call can raise Storage_Error, and
> almost any Ada construct could be implemented with a subprogram call (and
> the language's framers did not want to constrain implementations enough to
> avoid it).

For the idea of the solution see my other message. (It was sent several days
before this one.)

****************************************************************

From: Randy Brukardt
Sent: Thursday, September 6, 2001  4:54 AM

> It seems, that in whatever reason, most members of this list think that I am
> stupid. It is not true.

I wouldn't go that far. Perhaps "stubborn" and "misguided", but not "stupid".

> I understand that _calling_ of Adjust may raise S_E, but I say not about
> calling of Adjust (which we may considers as a part of the
> assignment operation itself, not as part of Adjust). In such the case we can
> guaranty that Adjust itself which (conceptually) consist of only null operator
> will not raise exceptions.

Even "null;" can raise Storage_Error on a target where interrupts are handled
on the user's stack (this was true on the Z80 and the 8086, for instance). If
an interrupt comes in, the stack may get blown without the program doing
anything at all.

> For the idea of the solution see my other message. (It was
> sent several days before this one.)

I explained (in the message that you are replying to) why that solution doesn't
make sense. Your response to it here certainly makes no effort to address the
issues.

The most basic issue is that you want us to spend a lot of time working out a
detailed model of memory operations. For instance, you claim above that the
Adjust call is part of the assignment, not the Adjust. OK, but you would have to
have RM language to that effect. And having such language would have an impact
on existing implementations (which may not be following the rules you are
thinking about).

In any case, the more complete a proposal that you make, the more likely it is
that something will happen to it. Every Ada user has a dozen ideas that they
are sure will improve the language. The ARG simply does not have the time or
energy to work out the details of every idea that is suggested. Generally, only
ideas that find champions are worked out, and even then there is no guarantee
that anything will happen (see enumeration extensions, for example).

****************************************************************

From: Tucker Taft
Sent: Friday, August 31, 2001  4:40 PM

I believe I understand your concern.  I suppose it would
be possible for the rule to say that if the only exceptions
raised by the Adjusts or Finalizes involved in an assignment
statement are Storage_Error, then Storage_Error is propagated,
otherwise Program_Error is propagated.

Or equivalently, it is a bounded error for Adjust or Finalize
to propagate any exception *other than* Storage_Error.
If Storage_Error is propagated, by one or more Adjusts/Finalizes,
then Storage_Error is what is propagated by the assignment
statement as a whole.  The same would apply to finalization,
presumably.

I do have some concern on the performance effect on implementations.
In our implementation, the run-time system does all the
handling and re-raising of exceptions associated with
Adjust/Finalize, in part because abort deferral is also
required.  However, I believe certain other implementations
have in-line abort deferral and exception handling/reraising,
so for them it might be a significant overhead to distinguish
Storage_Error from other exceptions.

****************************************************************

From: Robert Dewar
Sent: Friday, August 31, 2001  6:55 PM

<<The model of user-defined assignments in Ada95 should be changed because it
is absolutely useless. (As I proved above, _any_ user-defined assignment is
in wrong.)>>

This is a completely absurd statement. Many large scale applications are
successfully using this feature. It is pointless to use this kind of excessive
rhetoric if you want anyone to take what you are saying seriously.

****************************************************************

From: Robert Dewar
Sent: Friday, August 31, 2001  6:57 PM

<<Client Side SSI (see my signature :-) ) It is written in C++, but if we
replace std::bad_alloc<->Storage_Error and std::logic_error<->Program_Error,
we would get the analogous situation in Ada.>>

No, you miss my question, I am not asking for a hypothetical example of how
a program *might* be helped, that's always unconvincing, compared to what
I *was* asking for, which is a real existing large scale Ada application
for which this is an important issue.

****************************************************************

From: Pascal Leroy
Sent: Saturday, September 1, 2001  3:31 AM

> It is very wrong model:
>
> 1. Accordingly to 11.1 _any_ construct may raise Storage_Error. So, _any_
> Adjust may raise S_E (Even one with null body. And even one whose body have
> the handler for S_E, because this handler may raise S_E itself!) So _every_
> assignment operator of a controlled type may raise Program_Error! Hm, it was
> certainly not the intention of the committee. It is even a security hole in
> language: any memory-intensive program which assigns a value of a controlled
> type and does not check for P_E around such assignments (even if it handles
> Strorage_Error and even if the Adjust does not allocate any memory
> explicitly) may crash with P_E. Now anybody here must agree that we need
> to somehow change the language on this.

There is one subtlety that you are missing here.  As you point out, RM95
11.1(6) states that any construct may raise S_E (and we are definitely not
going to change that).  Among other things, this means that an attempt to
handle S_E might well raise S_E (maybe because entering the handler requires
some memory allocation).  So the language effectively does not guarantee
that you can ever handle S_E.  Although implementation normally try to do a
reasonable job, any handling of S_E is extremely hairy.  I would not
recommend depending on that capability in the first place.

Interestingly enough, Java has a class Error which defines exceptions from
which the program is not expected to recover.  OutOfMemoryError and
StackOverflowError fall in that category.  I think it would have been useful
if Ada had (at least informally) made the same distinction.

****************************************************************

From: Pascal Leroy
Sent: Saturday, September 1, 2001  3:32 AM

> I believe I understand your concern.  I suppose it would
> be possible for the rule to say that if the only exceptions
> raised by the Adjusts or Finalizes involved in an assignment
> statement are Storage_Error, then Storage_Error is propagated,
> otherwise Program_Error is propagated.

You would still have to decide what happens if a composite has several
controlled components, and one of the Adjust raises P_E and the other S_E.
Which exception is propagated out of the overall assignment?  Does the order
matter?  (Hopefully not.)  Also, I am not sure what the phrase "involved in
an assignment" means.  Do you mean that these subprograms would behave
differently when called in another context? (e.g. pure finalization).

And I don't see why we would want to specialize S_E in that respect.  I
could come up with an equally (un)convincing argument where Adjust could
raise Tasking_Error in some cases, and I would really want the caller to
know that, instead of getting P_E.  Why makes S_E special?

> I do have some concern on the performance effect on implementations.
> In our implementation...

Currently the only thing that you need to record when adjusting a composite
is that _some_ exception was raised, and that the entire adjustment will
therefore have to raise P_E when it's done.  Distinguishing different
exceptions would have a very sizeable implementation impact.

Incidentally, I am always dubious when we try to draw subtle distinctions
between exceptions, which exception is raised, under what circumstances,
etc.  We have been bitten by that in Ada 83 with C_E vs. N_E.  I remember
Jean saying that there should only have been one predefined exception,
raised by any failure of any language check.  Although this position is a
bit extreme, I have more sympathy for it as this discussion develops.

****************************************************************

From: Victor Porton
Sent: Saturday, September 1, 2001  1:16 AM

> I believe I understand your concern.  I suppose it would
> be possible for the rule to say that if the only exceptions
> raised by the Adjusts or Finalizes involved in an assignment
> statement are Storage_Error, then Storage_Error is propagated,
> otherwise Program_Error is propagated.
>
> Or equivalently, it is a bounded error for Adjust or Finalize
> to propagate any exception *other than* Storage_Error.
> If Storage_Error is propagated, by one or more Adjusts/Finalizes,
> then Storage_Error is what is propagated by the assignment
> statement as a whole.  The same would apply to finalization,
> presumably.

Well, if Storage_Error is propagated by at least one Adjust in an assignment
operator _and not other (non S_E) exceptions are propagated by other
"Adjust"s_, then we should propagate Storage_Error. However it is not 100%
backward compatible :-(

Well, I already probably have sent too much letters on this topic. It seems
that it is impossible to recover Ada from this shortcoming :-(

****************************************************************

From: Victor Porton
Sent: Saturday, September 1, 2001  7:31 AM

> There is one subtlety that you are missing here.  As you point out, RM95
> 11.1(6) states that any construct may raise S_E (and we are definitely not
> going to change that).  Among other things, this means that an attempt to
> handle S_E might well raise S_E (maybe because entering the handler requires
> some memory allocation).  So the language effectively does not guarantee
> that you can ever handle S_E.  Although implementation normally try to do a
> reasonable job, any handling of S_E is extremely hairy.  I would not
> recommend depending on that capability in the first place.

I didn't understood what I miss: you repeat here my own words.

But my opinion is the reverse. I argue that certain constructs should never
raise S_E (if not a requirement, then it should be an implementation advise).
These constructs probably roughly should be:

- null operator

- call of an user defined allocator (however, the allocator itself may raise
S_E)

- ? (some other constructs)

We also can introduce "memory safe" subprograms:

pragma Memory_Safe(A_Subprogram)

These subprograms shall never raise S_E. In memory safe subprograms some
operations shall be not allowed (calling of not memory safe subprograms,
recursion, allocating memory with the standard allocator...) Well, probably
only a little number of constructs should be allowed in memory safe
subprograms.

Implementation: memory for such the subprograms should be reserved on the
stack or in a separate array in advance (at the point of activating the task,
which may call such a subprogram).

> Interestingly enough, Java has a class Error which defines exceptions from
> which the program is not expected to recover.  OutOfMemoryError and
> StackOverflowError fall in that category.  I think it would have been useful
> if Ada had (at least informally) made the same distinction.

The warranty of Java clearly says that it isn't for atomic reactors :-)

P.S. Consider the following: a program which we want to make as fast as
possible and which is faster as it gets more memory (these programs really
exist in such fields as mathematics, AI, data compression etc.). For
simplicity let's consider that the system does not have swap. (E.g. many Linux
installations do not have swap.) Writing such a program we would want to
allocate as much memory as possible and stop allocating only when we get S_E.
But in Ada there are no warranty that we will able to catch S_E... C++ is more
reliable in this aspect.

****************************************************************


Questions? Ask the ACAA Technical Agent