Version 1.5 of ai05s/ai05-0142-3.txt

Unformatted version of ai05s/ai05-0142-3.txt version 1.5
Other versions for file ai05s/ai05-0142-3.txt

!standard 3.10.2(13.1/2)          09-04-27 AI05-0142-3/02
!class Amendment 09-04-10
!status No Action (9-0-0) 10-02-27
!status work item 09-04-10
!status received 09-04-10
!priority Medium
!difficulty Medium
!subject Accessors for Ada.Containers
!summary
(See proposal.)
!problem
Modifying a portion of a larger opaque object (such as a container) is not well-supported in Ada. The Ada.Containers packages provide a procedure Replace_Element for this purpose. But this procedure requires copying the element (potentially in both directions). That could be very expensive if the element is large.
The Ada.Containers packages also provide an procedure Update_Element for this purpose; this provides a writable object as a parameter to a subprogram passed into the procedure. This procedure avoids the need to copy the element, but it is hardly convenient to define a procedure for every component of the element that needs changing. The extra syntax needed obscures the real meaning of the program.
An option that was rejected for the Ada.Containers packages was to return an access to the element type. However, this is problematic as it is difficult to control the accessibility and lifetime of the returned access. If the element is removed from the container, the returned access could become dangling; continued use of the access would make the program erroneous. Moreover, the accessibility of the returned object (and thus what could be done with it) would depend on the actual implementation of the container. Bounded containers would typically only return access values with a very short lifetime, while unbounded containers would typically return access values with a much longer lifetime. Converting from an unbounded to bounded form could thus introduce new runtime errors - a serious maintenance hazard.
!proposal
(See wording.)
!wording
Pseudo-wording:
Language Design Principles: The idea of the returnable parameter (see below) is that we will ensure at the call site that a part of it can be returned as part of the function result without creating a dangling pointer. We do this with a combination of static and dynamic rules (most of the dynamic rules can actually be evaluated at compiler-time, but doing so would break privacy or the generic contract -- we don't want legality rules that do either of these things.) Note that these rules don't need to be a perfect match between the call and inside the function -- it is harmless if the call thinks a parameter if returnable while the actual function body does not (this can happen inside of a generic, for instance). On the other hand, disaster could result if the call thinks it is passing a normal parameter (and thus makes no checks) while the body assumes the parameter was already checked.
[I defined the terms "returnable parameter" etc. below in order that we don't distribute that definition all over the world. Currently, they only get used a few times, but that's fine. - RLB]
Static Semantics (Definitions): A "returnable parameter" is one for which a part (or coextension) of the parameter can be returned as a function result. An "immutably returnable parameter" is a parameter with the following characteristics:
* it is a parameter of a function with an anonymous access result, or a result subtype whose view has a part with an access discriminant;
* if the function of which it is a parameter overrides a homograph and has a controlling result, the corresponding parameter in all overridden homographs is also an immutably returnable parameter;
* the type of the parameter is tagged, or is a composite type that has a known discriminant part or is not a partial view.
[Note: If we wanted to retain the original "non-in" wording, it would go here and in the next bulleted list.]
AARM Reason: This is the definition for use in legality rules; these rules only apply to types that are composite and cannot become something else.
The part about the homographs ensures that the "immutably returnable parameter" property does not change for dispatching calls. In particular, if some tagged type does not have an part with an access discriminant, any functions with controlling results that it defines will not have returnable parameters. If a descendent of that tagged type adds an access discriminant and overrides the function, we could dispatch to the function without the special accessibility checking. In that case, the function body better not assume that checking was performed, as it could allow it to return dangling pointers. We only have to worry about controlling results, as in other cases the return type of the dispatching routine cannot change.
We need the messy wording excepting partial views as a private type is always considered composite and we don't want this to apply to parameters that have private types but are really elementary. [Note: For uses in calls, this is not strictly necessary, as it only would mean an extra check; but for uses inside of function bodies, this is critical so that parameters that were never checked are not used as if they were checked. OTOH, it is not certain that you could do anything that causes trouble with such a parameter, as it would not be passed by reference and would not allow 'Access, nor would it have any visible aliased components. So maybe we don't need the messy wording.] End AARM Reason.
A "runtime returnable parameter" is a parameter with the following characteristics:
* it is a parameter of a function with an anonymous access result, or a result subtype whose full view has a part with an access discriminant;
* if the function of which it is a parameter overrides a homograph and has a controlling result, the corresponding parameter in all overridden homographs is also a runtime returnable parameter;
* the full view of the type of the parameter is a composite type.
AARM Reason: This is the definition for use in dynamic semantic checks. The homograph part of the rule is needed for the same reason as notied previously.
Add a new Static semantics rule somewhere: For runtime returnable parameters of a function call, the master of the expression designating the actual parameter is that of the function result (rather than the master of the function call).
[The master of a function result is defined by 3.10.2(10/2). Note that if the function call is a master, these two masters will be different.]
[RLB: Not sure where to put this. It really belongs in 7.6.1(3/2), but that seems really nasty because how complex the wording there is. We also could put it into 6.4.1, after (15) [and the new rule given below] as a "Notwithstanding" rule, but that isn't very pleasant, either.]
Add a new Legality Rule after 6.4.1(6):
In a function call, the accessibility level of the actual object for each immutably returnable parameter shall not be statically deeper than accessibility level of the master of the function result.
AARM Discussion: Since immutably returnable parameters are always composite, there is always an object (possibly anonymous) to talk about. This is discussing the static accessibility level of the actual object; it does not depend on any runtime information (for instance when the actual object is a formal parameter to another call).
Add a new Dynamic Semantics after 6.4.1(15): (this would be a new, outer level, bullet)
In a function call, for each runtime returnable parameter, a check is made that the master of the accessibility level of the actual object is the same as or includes the master of the function result.
AARM To Be Honest: We're talking about the "nominal" level of the actual object. If the actual object is a formal parameter of some subprogram call C, we do not intend to require dynamic checks that depend on the master of the actual parameters to C, as that would cause nasty distributed overhead (all composite parameters would have to carry accessibility levels).
The accessibility level of a formal immutably returnable parameter in a function is that that the return object will have after a return statement completes normally.
[That level is defined by 3.10.2(10.1/2). We're talking about inside of the function body. This rule would go somewhere in 3.10.2(6-16), I'm not certain where.].
[This level needs to be defined statically as it is used in legality rules. That prevents using the runtime version for this rule, and that means that parameters of untagged private types are not going to be able to use this accessibility. That does not appear to be a problem, as such parameters do not have any visible aliased parts to return -- discriminants are not aliased; the type is private, so it doesn't have any components, and the parameter itself is not considered aliased. - RLB]
[The intent is that this level means that the checks defined in 6.5 for return statements will always succeed (or fail) without any need for any runtime garbage. But this depends on what we end up doing for AI-51; this works best with the fully dynamic version of that AI. If we use Bob's version, then we will have to make a hole in the rules to allow this scenario. One way to do that would be use immutably limited results only, in that case, the other rules given above would be changed similarly. - RLB]
Add the following to each Ada.Containers package immediately after Update_Element (with suitable substitutions for the names of types) [note that the names of the type, discriminant, and function are TBD].
type Accessor_Type (Element : not null access Element_Type) is tagged limited private;
Accessor_Type needs finalization.
A default-initialized object of type Accessor_Type propagates Program_Error.
AARM Reason: It is expected that Accessor_Type will be a controlled type, for which finalization will have some action to terminate the tampering check for the associated container. If the object is created by default, however, there is no associated container. Since this is useless, and supporting this case would take extra work, we define it to raise an exception.
function Accessor (Container : in out Vector; Position : in Cursor) return Accessor_Type;
If Position equals No_Element, then Constraint_Error is propagated; if Position does not designate an element in Container, then Program_Error is propagated. Otherwise, Accessor returns an object whose discriminant is an access to the element designated by Position. Program_Error is propagated if any operation tampers with the elements of Container while the object returned by Accessor exists and has not been finalized.
The element designated by Position is not an empty element after successful completion of this operation. [This is only needed for the Vectors case. - ED]
Add the following to Ada.Containers.Vectors and its relatives.
function Accessor (Container : in out Vector; Index : in Index_Type) return Accessor_Type;
If Index is not in the range First_Index (Container) .. Last_Index (Container), then Constraint_Error is propagated. Otherwise,
Accessor returns an object whose discriminant is an access to the element at position Index. Program_Error is propagated if any operation tampers with the elements of Container while the object returned by Accessor exists and has not been finalized.
The element designated by Position is not an empty element after successful completion of this operation.
Add the following to the Erroneous Execution section of each container package:
Execution is erroneous if the vector associated with the result of a call to Accessor is finalized before the result object returned by Accessor is finalized.
AARM Reason: Each object of Accessor_Type probably contains some reference to the originating container. If that container is prematurely finalized (only possible via Unchecked_Deallocation, as accessibility checks prevent passing a container to Accessor that will not live as long as the result), the finalization of the object of Accessor_Type will try to access a non-existent object. This is a normal case of a dangling pointer created by Unchecked_Deallocation; we have to explicitly mention it here as the pointer in question is not visible in the specification of the type. (This is the same reason we have to say this for invalid cursors.)
[Q: Should we add the read-only accessors? They would look like:
type RO_Accessor_Type (Element : not null access constant Element_Type) is tagged limited private;
function RO_Accessor (Position : in Cursor) return RO_Accessor_Type;
function RO_Accessor (Container : in Vector; Index : in Index_Type) return RO_Accessor_Type;
The names of all of these functions need work.]
!discussion
The idea is that the accessibility level of "returnable" parameters of functions is defined such that returning a part (or all) of the parameter as an anonymous access result or as an access discriminant will always succeed. That means no checks (and associated failures) need to be done within the function; we make any needed checks at the call-site instead.
The call-site checks can fail only in obscure cases, as the vast majority of objects that could be passed to a function call will outlive the function result. However, if the function result is built-in-place, the result could be made library-level simply by including a call in an allocator for a library level access type. In that case, we do not want to be returning an access to a local object. The check is normally a static one, and imposes overhead only in rare cases (such as passing a dereference of an anonymous access parameter).
The checks are (mostly) defined dynamically so that they don't break privacy. We want these rules to apply to all appropriate functions whose full definitions have the appropriate characteristics, but we don't want legality rules depending on the contents of private parts. That would cause generic contract problems, as well.
For instance, the properties of the return type are not part of mode-conformance. It would not be possible in a generic body to tell if the rule applied to a particular function or not:
generic type Cont is private; type Lim (<>) is limited private; with function Accessor (C : in out Cont) return Lim; package G is procedure Something; end G;
package body G is type ALim is access Lim; procedure Something is C : Cont; A : ALim; begin A := new Lim'(Accessor (C)); end Something; end G;
package Inst1 is new G (MV.Vector, MV.Accessor_Type, MV.Accessor); -- Better be illegal in the body. package Inst2 is new G (Natural, Natural, Some_Func); -- Body is OK.
We could try to apply an assume-the-worst rule, but that essentially turns into the original rule with no conditionality. So the complication doesn't buy much, and it would be a nightmare to implement in a shared generic.
By making all of the call checks dynamic in cases like this, we avoid the contract problems.
---
Note that the accessibility of the returned object from a call to Accessor will prevent converting the returned access discriminant to any type that lives longer than the returned object. That means that we can assume that the discriminant value cannot be accessed after the object disappears.
It is possible to use Unchecked_Deallocation to destroy the returned Accessor_Type object prematurely, while continuing to use the returned access. The accessibility check on the container argument to the Accessor function means that the returned access can't live longer than the container, but the use of Unchecked_Deallocation would allow tampering on the container. This could look like:
declare type at is access MV.Accessor_Type; procedure Free is new Ada.Unchecked_Deallocation (MV.Accessor_Type, at);
PAT : at := new MV.Accessor_Type'(Vect.Accessor (1))
Element : access Element_Type := PAT.Element; -- OK.
begin Free (PAT); -- Tampering checking ends here!
Vect.Delete (1); -- No check here. ... Element ... -- Oops, gone. end;
Obviously, this is highly unlikely to happen by accident. This is more like shooting oneself in the foot by attaching a laser target to your foot and then firing a laser-seeking missile. It doesn't seem worth worrying about someone who wants to go to these lengths to avoid a tampering check - they're just as likely to use Unchecked_Conversion or .all'Unchecked_Access or something else nasty.
---
We don't want to make this change for all parameters of function for compatibility reasons.
First, the accessibility checks needed could cause some functions used to initialize an allocator to become illegal or raise Program_Error. While that is relatively rare, it still would be hard to work around.
Second, changing the master of all parameters of functions would make function parameters live a long time in some cases (such as when the result is renamed or part of an allocator). That could be especially nasty if the parameter is an object that contains tasks that otherwise would have been terminated or releases locks when finalized. This would be the worst sort of incompatibility, where the program still works but does something different.
---
We need to be able to define the period of existence of the return object of the accessor as a time when tampering with elements is prohibited. If we didn't do this, we would be opening the door to all manner of problems (up to and including erroneousness) if an element is added or deleted to the container.
To see how this could happen, consider the following example:
Assume that Ada.Containers.Vectors has an accessor function defined as follows:
function Accessor (C : in out Vector; I : Index_Type) return access Element_Type;
Now consider the following program fragments:
package Data is type Node (D : Boolean := True) is record case D is when True => CT : Natural; when False => CF : Float; end case; end record; subtype True_Node is Node (D => True); subtype False_Node is Node (D => False);
package Node_Vector is new Ada.Containers.Vectors (Element_Type => Node, Index_Type => Positive);
Node_List : Node_Vector.Vector; end Data;
with Data; package Far_Away is procedure Some_Complex_Operation (...); end Far_Away;
package body Far_Away is procedure Some_Complex_Operation (...) is Some_Index : Positive; begin ... Some_Index := <lengthy-computation-resulting-in-value-of-1>; ... Data.Node_List.Delete (Some_Index); ... end Some_Complex_Operation; end Far_Away;
with Data; package Process is procedure Process_True (NT : in out Data.True_Node); end Process;
with Far_Away; package body Process is procedure Process_True (NT : in out Data.True_Node) is Component : renames NT.CT; -- OK, NT is constrained. begin ... Far_Away.Some_Complex_Operation (...); if Component > 10 then -- ** Oh-oh!! Component has moved. ... end if; end Process_True; end Process;
with Data, Process; procedure Main is begin Data.Node_List.Append (New_Item => (D => False, CF => 1.0)); -- Element 1. Data.Node_List.Append (New_Item => (D => True, CT => 4)); -- Element 2. Data.Node_List.Append (New_Item => (D => False, CF => 26.5)); -- Element 3. ... Process.Process_True (Data.Node_List.Accessor (2).all); ... end Main;
Now, in the canonical implementation for a vector, all of the data is stored in an array of some size, and when a component is deleted, the rest of them slide down. So, when the Far_Away operation (probably called through many levels of calls) deletes the first element of the vector, all of the rest of them move. That means that the second element passed to Process_True unexpectedly changes its value to that which was in the third element. But that element doesn't match the constraint and doesn't even have a CT component! So we get erroneous behavior. Note that the same thing would happen if an element was inserted at the front of the vector.
Note that in this example no one has copied the access value returned by the accessor. So a rule preventing copying does no good. You can also get a similar effect by renaming the access value itself and doing bad things while the renames exists.
Note that the erroneousness can't happen if you use Update_Element instead, because the tampering check will cause the deletion to raise Program_Error. Similarly, cursors have rules allowing implementations to detect this sliding and raise Program_Error.
This example code seems plausible; the record type is similar to many found in the author's compiler and using a library-level container is not unusual. So it seems reasonable that cases like this would happen in practice. Thus we conclude that a tampering check is needed for the accessor functions.
---
Alternatives:
We could define functions to have returnable parameters only for immutably limited result types. That's all we need for the containers, and it would reduce the cases where incompatibilities could arise. But it seems to be preventing the use of a useful tool.
We defined a returnable parameter as any that has a "really" composite type and is in a function with an appropriate return type. We could limit these parameters further only to cases that have a part that could be usefully returned: a tagged type (which is implicitly aliased), a type with an aliased part, or parts with access discriminants. We didn't do this as it would cause a maintenance hazard: adding "aliased" to a component in a private type could cause existing function calls that use that type (possibly far away) to start raising Program_Error. A similar issue would happen if a component with an access discriminant were added.
Note that this hazard exists somewhat for functions if the return type is changed. This is likely to be fairly rare. It could be mitigated by requiring returning a type with a visible access discriminant before the function would be considered as having returnable parameters. That's all we need to use to implement the container's accessors. But it seems restrictive.
Another option that was considered was to have a method to mark subprogram parameters that need this capability. As this compatibility is an important part of the profile that ought to figure into mode conformance, this would need to be done with syntax (having a pragma affect legality is uncomfortable).
This would have the advantage of applying the accessibility check only to parameters that actually need it, improving flexibility. But adding syntax for a rarely-used case is annoying at best:
function Accessor (Container : in out Vector use for return; Position : in Cursor) return Accessor_Type;
function Read_Only (Container : Vector use for return; Index : Index_Type) return RO_Accessor_Type;
The idea is that "use for return" would be a "mode-modifier", which would be considered as part of profile for conformance purposes. Thus it would have to be included in renames and for generic matching. That neatly gets rid of generic contract issues and all compatibility issues, but of course at the cost of extra syntax.
!examples
The new Accessor function could be used (based on a recent example from comp.lang.ada):
with Ada.Containers.Vectors; procedure Check is package Integer_Vectors is new Ada.Containers.Vectors (Index_Type => Natural, Element_Type => Integer);
package Nested_Vectors is new Ada.Containers.Vectors (Index_Type => Natural, Element_Type => Integer_Vectors.Vector, "=" => Integer_Vectors."=");
IV : Integer_Vectors.Vector; NV : Nested_Vectors.Vector; begin IV.Append(42); NV.Append(IV); NV.Accessor(0).Element.Append(43); end Check;
The dereference does not need to be explicitly given in this case, and the fact that the object returned by Accessor is limited and (probably) controlled is not visible in the code.
You could also save the entire object as long as needed:
declare My_IV : Nested_Vectors.Accessor_Type := NV.Accessor(0); begin
(since My_NV would be built-in-place).
Note that this design still works as intended even if the user renames only the limited access value:
declare My_IV : Integer_Vectors.Vector renames NV.Accessor(0).Element.all; begin
Even in this case, the wrapping finalizable object still is protecting the container, because the master of the returned object is the master of the renames: it will stick around as long as the renames does.
Moreover, splitting the access from the object usually isn't possible:
declare My_IV : access Integer_Vectors.Vector := NV.Accessor(0).Element; -- Illegal! begin
This is illegal because the accessibility check fails. The (anonymous) object returned here will have the master of the expression, which is shorter than the master of the declaration of My_IV. Thus the accessibility check preventing shorter lifetimes will fail.
It is OK to rename the access:
declare My_IV : access Integer_Vectors.Vector renames NV.Accessor(0).Element; begin
as in this case, the anonymous object will continue to exist until the renames is finalized.
This element can be passed as a parameter, as again the wrapping object will live as long as the parameter:
Operate (NV.Accessor(0).Element);
or
Munge (NV.Accessor(0).Element.all);
!ACATS test
ACATS tests would be needed for this feature.
!appendix

From: Steve Baird
Sent: Friday, April 10, 2009  11:12 AM

> I think adding "call-level" accessibility is a smaller change, and is
> relatively intuitive, so I would rather do that than add a whole
> "limited access type" concept.

I spoke with Randy yesterday and I believe that we both agree with you on this
point. At the risk of repeating much that has been said before, I'll summarize
where I think we stand now.

As you pointed out, no language changes are needed in order to get the desired
functionality in the case where the implementation of the container already uses
a level of indirection.

This is accomplished by using a function whose result type is tagged limited
private with a visible access-to-element discriminant and with a finalization
routine that updates the tampering state of the container.

Two language changes are needed in the case where the objects that need to be
referenced (as variables) in the function result have the accessibility level of
the container parameter (as opposed to the accessibility level of the container
generic instance, as would typically be the case if a level of indirection is
introduced):
   - in out parameters for functions
   - a new parameter accessibility rule which has the effect
     of allowing an immutably limited function result
     to contain references to (some) parameters; finalization of any
     referenced actual parameter objects must be correspondingly
     deferred until after the finalization of the function result object.

Both of these are needed in order to support something like

    type Element_Handle (Element_Ref : access Element)
      is tagged limited private;

    function F (C : in out Container; I : Index) return Element_Handle is
    begin
        return (Element_Ref => C.Elements(I)'Access, ...);
        -- result also includes reference
        -- to C or perhaps C.Tampering_State
    end F;

In whatever cases we decide this new accessibility rule applies, this means that
the finalization of the actual parameter may occur later than it otherwise would
in the case where the actual parameter is a function call, an aggregate, or an
allocator of an anonymous access type.

Is this compatibility issue a major concern?

****************************************************************

From: Tucker Taft
Sent: Friday, April  10, 2009  11:55 AM

> Is this compatibility issue a major concern?

Not to me.  Since I think the result type will inevitably need to be a
build-in-place type, these didn't exist before Ada 2005, so the compatibility
issues are trivial at this point.

Note that I have proposed that access results have essentially the same rules as
the access discriminants of build-in-place results, so this would allow for
un-wrapped access results to be used in those situations where no finalization
is required, accomplishing essentially everything we would get from limited
access types.

****************************************************************

From: Randy Brukardt
Sent: Friday, April  10, 2009  1:09 PM

...
> Note that I have proposed that access results have essentially the
> same rules as the access discriminants of build-in-place results, so
> this would allow for un-wrapped access results to be used in those
> situations where no finalization is required, accomplishing
> essentially everything we would get from limited access types.

This is not true, in that you have no control over the accessibility of the
result. The rules you proposed were that the accessibility of the result was
that of the type that the access result is converted to. That's not necessarily
the same as the place of the call. In the built-in-place object case, you can
*only* get the accessibility of the call.

That means that an implementation change in a package could cause functions that
previously worked to raise Program_Error. This is clearly a maintenance hazard.

To take an example, for a library level container C and access type Acc and the
following function Func:

     function Func (C : in out Container) return access Element_Type;


     declare
         Ptr : Acc := Func (C); -- (1)
     begin

This call will work if the container is implemented with nodes allocated out of
a library-level heap (as is the case with the Unbounded containers), and will
raise Program_Error if the container is implemented with an internal array of
nodes directly as part of the object (as is the case with the Bounded
containers). This means that if the implementation of the container changes,
calls might start raising Program_Error for no apparent reason.

OTOH, the access discriminant case:

    declare
        Ptr : Acc := Func (C).Element;
    begin

will statically fail an accessibilty check (as the discriminanted object has a
nested scope, and the access type's is library level).

Of course, we're not going to use the former form, so this doesn't matter for
Ada.Containers.

****************************************************************

From: Tucker Taft
Sent: Friday, April 10, 2009  1:39 PM

...
> This is not true, in that you have no control over the accessibility
> of the result. The rules you proposed were that the accessibility of
> the result was that of the type that the access result is converted
> to. That's not necessarily the same as the place of the call. In the
> built-in-place object case, you can *only* get the accessibility of the call.

I don't agree with this.  You can use a call on a build-result-in-place function
to initialize an allocator of a named access type, and you can return it from an
outer build-result-in-place function.  In both of these cases, you don't have
call accessibility.  In general, the caller must indicate whether the context
corresponds to "call-level" accessibility or something longer-lasting. On
function return, if the function returns 'Access of some part of an actual
parameter, then it will have to raise Program_Error if the caller context
indicates something longer lasting that call-level accessibility.

I believe this is exactly analogous to the case with access results.  If you
dereference the access result directly, (perhaps to define a rename), you have
call-level accessibility. If you convert it to some named type, you don't have
call-level accessibility.  Similarly, if you use a call of a
build-result-in-place function to initialize an allocator for a named access
type, then you don't have call-level accessibility.  If you pass it to another
function, then you do.

...
> OTOH, the access discriminant case:
>
>     declare
>         Ptr : Acc := Func (C).Element;
>     begin
>
> will statically fail an accessibilty check (as the discriminanted
> object has a nested scope, and the access type's is library level).

But what about:

     declare
         Ptr : Acc_T := new T'(Func(C));
     begin

This will have the same issue as the access result.

I really believe they are analogous, with similar accessibility rules, if you
map "used to initialize allocator of named access type" in the build-in-place
case to "converted to named access type" in the access result case.

> Of course, we're not going to use the former form, so this doesn't
> matter for Ada.Containers.

I think you have the same issue using a build-in-place result, but I agree that
using it to initialize an allocator of a named access type is perhaps less
likely in practice than simply converting an access result to a named access
type.  The key point in my mind is that the language rules are analogous, so
presumably similar implementation techniques can be used, and the user can shift
between returning an access result and returning a build-in-place object without
huge shifts in capability creating annoying false incentives.

****************************************************************

From: Randy Brukardt
Sent: Friday, April  10, 2009  2:05 PM

...
> I don't agree with this.  You can use a call on a
> build-result-in-place function to initialize an allocator of a named
> access type, and you can return it from an outer build-result-in-place
> function.  In both of these cases, you don't have call accessibility.

I don't see this, although that may be because I've already abandoned the entire
idea of call-level accessibility (we don't need it). I'll be making a full(er)
write-up right after lunch and you will see.

My model is that in all cases, the accessibility of the call is that of the
function result. That means in the case of an allocator the master of the call
is the master of the access type. So things had better be long-lived.

In any case, the "outer built-in-place" function case had better work without
raising any funny exceptions. It makes perfect sense to nest these guys (in the
container-of-containers case), and I consider that a critical requirement.

> In general, the caller must
> indicate whether the context corresponds to "call-level"
> accessibility or something longer-lasting.

There is nothing longer-lasting here; we're just talking about the accessibility
of the function result. Period. (The more general notion of call-level
accessibility is now dropped, we don't need it.)

> On function return, if the function returns 'Access of some part of an
> actual parameter, then it will have to raise Program_Error if the
> caller context indicates something longer lasting that call-level
> accessibility.

No. I do think this shows that there may be a need to do an accessibility check
*on the parameter* in a few unusual cases. But that check can usually be done
statically, so it is a big improvement over the normal case.

> I believe this is exactly analogous to the case with access results.
> If you dereference the access result directly, (perhaps to define a
> rename), you have call-level accessibility.
> If you convert it to some named type, you don't have call-level
> accessibility.  Similarly, if you use a call of a
> build-result-in-place function to initialize an allocator for a named
> access type, then you don't have call-level accessibility.  If you
> pass it to another function, then you do.

None of this makes any sense to me. We obviously have a different idea of the
accessibility rules here.

...
> But what about:
>
>      declare
>          Ptr : Acc_T := new T'(Func(C));
>      begin
>
> This will have the same issue as the access result.

Not unless C is a local object. The accessibility of the function result in this
case is library level. That's the master of the formal parameter as well. Since
that is the case, I think there would have to be an accessibility check on the
parameter (so that *it* cannot be a more local than the function result).

..
> I think you have the same issue using a build-in-place result, but I
> agree that using it to initialize an allocator of a named access type
> is perhaps less likely in practice than simply converting an access
> result to a named access type.  The key point in my mind is that the
> language rules are analogous, so presumably similar implementation
> techniques can be used, and the user can shift between returning an
> access result and returning a build-in-place object without huge
> shifts in capability creating annoying false incentives.

I agree with this, in that the same rules of accessibility should apply in both
cases. We just don't have quite the same idea of the accessibility rules needed.
But raising Program_Error conditionally depending on the implementation of the
containers is clearly unacceptable. If the accessibility is wrong, that should
*always* be detected (even if the actual implementation would allow it to work).
I think that the rules I have in mind would have that effect for the containers
(yours definitely do not).

****************************************************************

From: Tucker Taft
Sent: Friday, April  10, 2009  3:35 PM

...
> My model is that in all cases, the accessibility of the call is that
> of the function result. That means in the case of an allocator the
> master of the call is the master of the access type. So things had better be long-lived.

You will have to define what you mean by "accessibility of the call." I am
guessing this becomes a level that some or all of the actual parameters are
being checked against.  Perhaps the checks would apply only to those parameters
that will have some aliased part or coextension that might be designated by some
access value buried in the result object?  This sounds hard to specify.

The existing Ada 2005 rules for build-in-place clearly have the context of the
call determining where the return object is created, be it in the heap, as a
component of another object, in a declared variable, or as a temp passed to
another subprogram.  I presume you aren't planning on changing that.  Instead it
sounds like you are going to specify that (some or all?) of the (by-reference?)
parameters must live at least as long at the return object.

...
>> I believe this is exactly analogous to the case with access results.
>> If you dereference the access result directly, (perhaps to define a
>> rename), you have call-level accessibility.
>> If you convert it to some named type, you don't have call-level
>> accessibility.  Similarly, if you use a call of a
>> build-result-in-place function to initialize an allocator for a named
>> access type, then you don't have call-level accessibility.  If you
>> pass it to another function, then you do.
>
> None of this makes any sense to me. We obviously have a different idea
> of the accessibility rules here.

I think the difference is that I was presuming no restrictions were being
imposed at the call site on the parameters, whereas you are now proposing that
there might be some accessibility checks on the parameters.  That certainly
sounds like an approach worth considering.  But everything I wrote earlier has
presumed no such checks, so all of the checking had to be performed at the
return statement, potentially using information passed in from the caller.  I'm
not saying that is better, but it might explain the difference in perspective.

>
> ...
>> But what about:
>>
>>      declare
>>          Ptr : Acc_T := new T'(Func(C));
>>      begin
>>
>> This will have the same issue as the access result.
>
> Not unless C is a local object. The accessibility of the function
> result in this case is library level. That's the master of the formal
> parameter as well. Since that is the case, I think there would have to
> be an accessibility check on the parameter (so that *it* cannot be a
> more local than the function result).

Yes, I am beginning to understand your model.  It basically assumes the worst
about what might be used with 'Access in the return statement.

> ..
>> I think you have the same issue using a build-in-place result, but I
>> agree that using it to initialize an allocator of a named access type
>> is perhaps less likely in practice than simply converting an access
>> result to a named access type.  The key point in my mind is that the
>> language rules are analogous, so presumably similar implementation
>> techniques can be used, and the user can shift between returning an
>> access result and returning a build-in-place object without huge
>> shifts in capability creating annoying false incentives.
>
> I agree with this, in that the same rules of accessibility should
> apply in both cases. We just don't have quite the same idea of the
> accessibility rules needed. But raising Program_Error conditionally
> depending on the implementation of the containers is clearly
> unacceptable. If the accessibility is wrong, that should *always* be
> detected (even if the actual implementation would allow it to work). I
> think that the rules I have in mind would have that effect for the containers (yours definitely do not).

I think we aren't that far off, really.  If we can agree that access results
should piggy-back on the rules for access discriminants of build-in-place return
objects, I'm happy.  If we decide to impose some accessibility checks on the
actual parameters when calling a build-in-place function, those same checks
would be relevant for a function with an access result.

In either case, the delicate part will be defining which actual parameters are
to be checked, and that will in turn imply which formal parameters can be
referenced with 'Access in the return statement.

****************************************************************

From: Randy Brukardt
Sent: Friday, April  10, 2009  10:06 PM

Find attached a full write-up of the latest ideas on this topic. [This is /01
of this AI - ED]

I'm not completely certain that the accessibility check is harmless in the case
of functions that don't need it, but the only way to get rid of it is to add a bit
of syntax to differentiate between parameters that need it and those that don't.
(See the very end of the !discussion section.) That seems like overkill in the
absence of a real problem.

Tell me what you think.

****************************************************************

From: Tucker Taft
Sent: Sunday, April 26, 2009  10:56 PM

[Tucker was responding to an earlier draft of version /02 of this AI.]

I think what you have proposed is reasonable.  I would suggest some changes
or perhaps simply a different way of looking at the same essential rules:

  *  Make the access discriminants of the Accessor_Types
     null excluding.

  *  I don't see the need to say that the finalization
     of an Accessor_Type has no effect.  I think that is
     just misleading, especially right after we say that it
     "needs finalization."  We don't normally define exactly what
     an implementation must do to finalize a type appearing
     in a language-defined package.  We merely impose certain
     requirements (such as no loss of storage) which then
     imply what needs to occur upon finalization.

  *  If a function call has a result type that, in the caller's view,
     is anonymous access, classwide, or has a part with an access discriminant
     ("function call whose result might reference a parameter"), then
     *all* actual parameters that in the caller's view are tagged, have
     aliased subcomponents, or have parts with access discriminants
     ("actual parameters that are referenceable in caller's view"),
     should be subject to static accessibility checks as part of a
     legality rule.

  *  If a function call has a result type that, in any view, is anonymous
     access, classwide, controlling as part of a dispatching call, or
     has a part with an access discriminant, then *all* actual parameters
     that are tagged, have aliased subcomponents, or parts with
     access discriminants in any view, are subject
     to dynamic accessibility checks prior to the call.
     (Or, rewording using the phrases used above:
       If in any view a function call has a result that might reference
       a parameter, then all actual parameters that are referenceable
       (in any view) are subject to a dynamic accessibility check before
       the call.)

     If a referenceable parameter is a "new" object (aggregate or
     function call), then its master is that of the function call's
     return object.  If such a call has an actual parameter that
     is an anonymous allocator, then its master is that of the function
     call's return object also (I think this partially addresses
     concerns about the local scope of anonymous allocators).
     If the call's return object is used to initialize an allocator, and
     that object is later unchecked-deallocated, then the implementation may
     at that time also finalize and deallocate these "new" objects
     associated with the call (including anonymous allocators).

I think we are largely in agreement, and this may be just my attempt to
reexpress your suggested rules so that I understand them better.

****************************************************************

From: Randy Brukardt
Sent: Monday, April  27, 2009  11:31 PM

...
>   *  I don't see the need to say that the finalization
>      of an Accessor_Type has no effect.  I think that is
>      just misleading, especially right after we say that it
>      "needs finalization."  We don't normally define exactly what
>      an implementation must do to finalize a type appearing
>      in a language-defined package.  We merely impose certain
>      requirements (such as no loss of storage) which then
>      imply what needs to occur upon finalization.

Understood, but this is a bit of a weird case. All of the wording
discusses what happens for values that are returned from one of the
Accessor functions. We need to say *something* about the
default-initialized case (that is, if one is defined explicitly). In
particular, it *must not* have any effect on the tampering state of
some container. Perhaps an AARM Implementation note is enough.

An alternative would be to have a default-initialized object of this type
raise an exception, since it is a useless construct and represents some
sort of mistake. That would simplify implementation a bit. I think I'll
write it up that way, as it addresses your objection.

>   *  If a function call has a result type that, in the caller's view,
>      is anonymous access, classwide, or has a part with an access discriminant
>      ("function call whose result might reference a parameter"), then
>      *all* actual parameters that in the caller's view are tagged, have
>      aliased subcomponents, or have parts with access discriminants
>      ("actual parameters that are referenceable in caller's view"),
>      should be subject to static accessibility checks as part of a
>      legality rule.

I don't understand the reason for including "classwide" in the result types
here. That seems like a significant incompatibility with existing code (and
even Ada 95 code). And you can't dispatch to a routine that has such a
requirement unless the access discriminant part is known. So it doesn't seem
like anything could be done with this.

As far as the parameters go, I just applied it to any "really" composite
parameter, because anything more specific is a maintenance hazard. With the
rule you have, adding a aliased component to a private type might cause far
away calls on some unrelated functions that happen to have a parameter of
the type to start raising Program_Error. Yuck.

>      If such a call has an actual parameter that
>      is an anonymous allocator, then its master is that of the function
>      call's return object also (I think this partially addresses
>      concerns about the local scope of anonymous allocators).
>      If the call's return object is used to initialize an allocator, and
>      that object is later unchecked-deallocated, then the implementation may
>      at that time also finalize and deallocate these "new" objects
>      associated with the call (including anonymous allocators).

That sounds worse than co-extensions. This level of complexity would kill the
proposal, and it has nothing whatsoever to do with the problem at hand.

****************************************************************

From: Tucker Taft
Sent: Tuesday, April  28, 2009  8:53 AM

> ...
>>   *  If a function call has a result type that, in the caller's view,
>>      is anonymous access, classwide, or has a part with an access discriminant
>>      ("function call whose result might reference a parameter"), then
>>      *all* actual parameters that in the caller's view are tagged, have
>>      aliased subcomponents, or have parts with access discriminants
>>      ("actual parameters that are referenceable in caller's view"),
>>      should be subject to static accessibility checks as part of a
>>      legality rule.
>
> That is the intent of the definition of "immutably returnable parameters".

I think "referenceable" might be more appropriate than "returnable,"
since that is what is actually happening.  I don't find "immutably"
helpful here.  I think it is a question of whether it is referenceable in the
callers view, or only in the full view.  Immutably was used with limited
because things might become *non* limited.  Referenceable parameters never
become non-referenceable.

> Why did you add classwide types to this list? That seems wrong to me;
> both because it adds a significant possibility of incompatibility (the
> number of Ada 95 functions with access discriminants is likely to be
> far less than classwide functions), and because it doesn't seem
> useful. You'd have to have a case-statement on various extensions in
> order to use it -- dispatching would not give you (directly) any
> returnable parameters. Perhaps we need to say something about
> classwide types whose specific type is known to have a part with an
> access discriminant, but that is the only case where it could be useful.

I'm confused by the above.  Classwide results generally need to be built
in place, since the returned object might have an access discriminant.
Inside the function, even though the result type is classwide, the actual
return object will often be of a specific type, and may have access
discriminants.  It seems quite natural for one of them to refer to one
of the "referenceable" parameters.

I do understand your concern about incompatibility, but remember the only
compatibility issue is when the function is used to initialize an allocator
 -- that is the only time that there are added requirements on the actual
parameters.

> ...
>>      If a referenceable parameter is a "new" object (aggregate or
>>      function call), then its master is that of the function call's
>>      return object.
>
> Technically, that is the master of the parameter expression, which I
> already covered. And that is the same whether or not the expression has a "new"
> part; it just isn't used if it does not.

I'm not sure that the way you expressed this made sense.
What is the master of an expression?  We can talk about the master of objects,
which determines the lifetime of the object.  Perhaps you are saying that if
the expression denotes a new object, then the master of that object is ...

Can you find some existing wording that talks about the master of an expression,
so I can compare your wording with that?

>
> <      If such a call has an actual parameter that
>>      is an anonymous allocator, then its master is that of the function
>>      call's return object also (I think this partially addresses
>>      concerns about the local scope of anonymous allocators).
>>      If the call's return object is used to initialize an allocator, and
>>      that object is later unchecked-deallocated, then the implementation may
>>      at that time also finalize and deallocate these "new" objects
>>      associated with the call (including anonymous allocators).
>
> That sounds worse than co-extensions: it means initializing a storage
> pool with *every* call. Or something. Death to co-extensions!

Can you explain the issue further?  This is just an implementation permission to
*avoid* a storage leak.  That is, if there is an aggregate or a function call as
an actual parameter, and it is required to live at least as long as the return
object, then the implementation is *permitted* to finalize and reclaim the actual
parameter object when the return object is unchecked-deallocated.

Since anonymous allocators as actual parameters are treated essentially like
aggregates from the point of view of storage allocation, it seemed to make sense
to give them the same master as an aggregate and allow the same reclamation.

Which part of the above do you find objectionable, and can you explain the reason a
bit further?

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April  28, 2009  1:39 PM

> > ...
> >>   *  If a function call has a result type that, in the caller's view,
> >>      is anonymous access, classwide, or has a part with an access discriminant
> >>      ("function call whose result might reference a parameter"), then
> >>      *all* actual parameters that in the caller's view are tagged, have
> >>      aliased subcomponents, or have parts with access discriminants
> >>      ("actual parameters that are referenceable in caller's view"),
> >>      should be subject to static accessibility checks as part of a
> >>      legality rule.
> >
> > That is the intent of the definition of "immutably
> returnable parameters".
>
> I think "referenceable" might be more appropriate than "returnable,"
> since that is what is actually happening.

"referenceable" doesn't mean much to me, since you can *always* reference something,
you just might get an exception doing so. But I'm not that tied to this terminology;
I'm sure there is something better out there.

> I don't find "immutably"
> helpful here.  I think it is a question of whether it is referenceable
> in the callers view, or only in the full view.
> Immutably was used with limited because things might become
> *non* limited.  Referenceable parameters never become
> non-referenceable.

It was my attempt to differentiate "static" returnability from "dynamic" returnability.
The basic concept of returnability doesn't tell you which you are talking about. This
is a major problem with accessibility, in that it is hard to tell whether you are
talking about static or dynamic accessibility; the only clue is the presence or absence
of the word "static" in the "deeper" relationship. And it doesn't work at all if you
want to do a static level check dynamically (as in this AI).

I wanted *both* terms to carry a modifier so there never would be any confusion as to
which rules you are talking about.

> > Why did you add classwide types to this list? That seems wrong to
> > me; both because it adds a significant possibility of
> > incompatibility (the number of Ada 95 functions with access
> > discriminants is likely to be far less than classwide functions),
> > and because it doesn't seem useful. You'd have to have a
> > case-statement on various extensions in order to use it --
> > dispatching would not give you (directly) any returnable parameters.
> > Perhaps we need to say something about classwide types whose
> > specific type is known to have a part with an access discriminant, but that is
> > the only case where it could be useful.
>
> I'm confused by the above.  Classwide results generally need to be
> built in place, since the returned object might have an access
> discriminant.

This AI has nothing (directly) to do with built-in-place. It works just as well on
things that aren't built-in-place. So I don't see this as relevant.

I don't see the point about access discriminants, either. If a classwide type is
non-limited, building in place is not possible in general. We surely aren't
insisting on impossible requirements?

> Inside the function, even though the result type is classwide, the
> actual return object will often be of a specific type, and may have
> access discriminants.  It seems quite natural for one of them to refer
> to one of the "referenceable" parameters.

Natural, but pointless. The only *correct* way (by OOP dogma) to create such
an object is via a dispatching call, and any such call will not have returnable
parameters (unless the root type already had such parameters). So the is no value
to the classwide function to having such parameters in that case. You could use
some control structure to create an object of a specific type, but that would
require visibility on all of those types -- a real nightmare (I had to do that
in the Claw builder and it was a disaster). So no one would intentionally do that.

My intent is that if the specific type has a part with an access discriminant,
then the classwide type would qualify, but otherwise it would not. I'm not
certain that this is reflected in the wording.

In any case, this seems highly speculative. Remember, the primary point here
is to get accessors. Other sorts of uses are just guesses at best - I think we'd
need to use realistic examples before going too far.

> I do understand your concern about incompatibility, but remember the
> only compatibility issue is when the function is used to initialize an
> allocator -- that is the only time that there are added requirements
> on the actual parameters.

That's not true, in that the lifetime of parameters is extended. That matters for
any parameters that are function calls or other sorts of temporaries. When function
calls are renamed or passed as parameters, they can live for a long time. If locks
or tasks are involved, that could make for very significant semantics changes (even
though the program will still "work" - no exceptions will be raised). Those are
the worst sorts of incompatibilities: where a program continues to run but does
something different.

> > ...
> >>      If a referenceable parameter is a "new" object (aggregate or
> >>      function call), then its master is that of the function call's
> >>      return object.
> >
> > Technically, that is the master of the parameter expression, which I
> > already covered. And that is the same whether or not the expression has a "new"
> > part; it just isn't used if it does not.
>
> I'm not sure that the way you expressed this made sense.
> What is the master of an expression?  We can talk about the master of
> objects, which determines the lifetime of the object.  Perhaps you are
> saying that if the expression denotes a new object, then the master of
> that object is ...

I'm sure my terminology is confusing here, because the language tends to skirt
this issue and doesn't define appropriate terminology.

Every construct in Ada is associated with a master. (This is mentioned in 7.6.1(2-3)).
That master only controls the finalization of things declared within it, but surely
the execution of the entire construct is within the master. (Where else would it be?
A model where the master changes for every entity of an expression is too bizarre
for words.)

So, by "master of the expression" I simply mean that master that executes the
expression (and would be the master of any items declared by the execution of that
expression). Similarly, the "master of the function call" is the master that executes
the function call. This isn't necessarily a unique master for each construct.

> Can you find some existing wording that talks about the master of an
> expression, so I can compare your wording with that?

I don't think there is any, other than the definition in 7.6.1(3/2) that an expression
can *be* a master.

Your attempt to talk about "new objects" doesn't help, because that isn't defined by
the language, either. You could define the term something like "the master of an
expression is the master that would be used for an entity declared by the execution
of the expression." Except that I'd probably replace "expression" by "construct" here.

> > <      If such a call has an actual parameter that
> >>      is an anonymous allocator, then its master is that of the function
> >>      call's return object also (I think this partially addresses
> >>      concerns about the local scope of anonymous allocators).
> >>      If the call's return object is used to initialize an allocator, and
> >>      that object is later unchecked-deallocated, then the implementation may
> >>      at that time also finalize and deallocate these "new" objects
> >>      associated with the call (including anonymous allocators).
> >
> > That sounds worse than co-extensions: it means initializing a
> > storage pool with *every* call. Or something. Death to co-extensions!
>
> Can you explain the issue further?  This is just an implementation
> permission to *avoid* a storage leak.  That is, if there is an
> aggregate or a function call as an actual parameter, and it is
> required to live at least as long as the return object, then the
> implementation is *permitted* to finalize and reclaim the actual
> parameter object when the return object is unchecked-deallocated.

I don't think that this semantics could be implemented without amazing handstands. How
would you keep track of the source of the part of the parameter which is then returned
into an allocated object? If the same function was called passing an existing access
value, you surely would not want this behavior. And the call site can have no idea
(short of inlining) as to what the function is doing internally -- maybe those pointers
came from fresh allocations or from global variables.

You'd have to carry source information along with every anonymous access parameter
(the accessibility level would not be enough, as passing a locally allocated object
would have to be treated differently). And you'd have to carry information about the
source of the pointer along with every component that it might be placed into.

The use of anonymous allocators is just bad. Janus/Ada puts out a warning that "this
allocator will leak memory" on every such allocator (since we make no attempt to
implement any special storage pools). And I think anyone whose program does not
include "pragma Restriction (No_Anonymous_Allocators)" is nuts (since anonymous
allocators are not required to be portable - the semantics you are talking about
have never been more the Implementation Advice - the most ignorable IA in the entire
Ada standard).

> Since anonymous allocators as actual parameters are treated
> essentially like aggregates from the point of view of storage
> allocation, it seemed to make sense to give them the same master as an
> aggregate and allow the same reclamation.

This is not true; that is just IA which ought to be ignored (and surely is in our
implementation).

> Which part of the above do you find objectionable, and can you explain
> the reason a bit further?

Unless you convince people to *require* this behavior of all compilers, it is
insane to depend on it. Moreover, the inability to control pools and deallocation
for anonynous allocators make them pretty useless. The uses that you have promoted
for years are too clever to be understandable -- when I see "new", I'm expecting
an allocation into a global data structure, not something with a very short
lifetime. It's just the wrong message. If you want a stack-allocated object, declare
a stack allocated object! Or just use an aggregate directly - don't use access types
unless you have to.

Most of this really was an attempt to give the advantages (and problems) of "in out"
parameters to functions without doing that. Which was complete insanity - lets not
continue to make a bad situation worse.

****************************************************************

From: Tucker Taft
Sent: Tuesday, April  28, 2009  2:23 PM

> So, by "master of the expression" I simply mean that master that
> executes the expression (and would be the master of any items declared
> by the execution of that expression). Similarly, the "master of the function call"
> is the master that executes the function call. This isn't necessarily
> a unique master for each construct.
>
>> Can you find some existing wording that talks about the master of an
>> expression, so I can compare your wording with that?
>
> I don't think there is any, other than the definition in 7.6.1(3/2)
> that an expression can *be* a master.

Right.  Masters don't "execute" constructs, though perhaps that's just being
picky.  A master *is* the execution of a construct, and it might include the
execution of other constructs. I still don't understand how you can say that
*the* master of an expression is some far away thing.  I would have said that
*the* master of an expression is the innermost master that includes the
evaluation of the expression.  I think you are more interested in the master of
the object denoted by the expression, and that will depend on the kind of the
expression.  I think function calls and aggregates are the only things we are
granting "object-hood" to.  Whenever I say "new object" read "function call or
aggregate".

>
> Your attempt to talk about "new objects" doesn't help, because that
> isn't defined by the language, either. You could define the term
> something like "the master of an expression is the master that would
> be used for an entity declared by the execution of the expression."
> Except that I'd probably replace "expression" by "construct" here.

I really think this is misleading.  We talk about function calls and aggregates
in 7.6.1(13/3), and that is where we need to change the rules to say that the
master of the associated anonymous object is *not* the innermost master in this
case, but is rather the master of the return object.  As mentioned above, when I
say "new objects" read "aggregates and function calls" (and if you can stomach
it, anonymous allocators passed to access parameters).

>>> <      If such a call has an actual parameter that
>>>>      is an anonymous allocator, then its master is that of the function
>>>>      call's return object also (I think this partially addresses
>>>>      concerns about the local scope of anonymous allocators).
>>>>      If the call's return object is used to initialize an
>>>>      allocator, and that object is later unchecked-deallocated, then
>>>>      the implementation may
>>>>      at that time also finalize and deallocate these "new" objects
>>>>      associated with the call (including anonymous allocators).
>>> That sounds worse than co-extensions: it means initializing a
>>> storage pool with *every* call. Or something. Death to co-extensions!
>> Can you explain the issue further?  This is just an implementation
>> permission to *avoid* a storage leak.  That is, if there is an
>> aggregate or a function call as an actual parameter, and it is
>> required to live at least as long as the return object, then the
>> implementation is *permitted* to finalize and reclaim the actual
>> parameter object when the return object is unchecked-deallocated.
>
> I don't think that this semantics could be implemented without amazing
> handstands. How would you keep track of the source of the part of the
> parameter which is then returned into an allocated object? If the same
> function was called passing an existing access value, you surely would
> not want this behavior. And the call site can have no idea (short of
> inlining) as to what the function is doing internally -- maybe those
> pointers came from fresh allocations or from global variables.
>
> You'd have to carry source information along with every anonymous
> access parameter (the accessibility level would not be enough, as
> passing a locally allocated object would have to be treated
> differently). And you'd have to carry information about the source of
> the pointer along with every component that it might be placed into.

The above doesn't mention anonymous allocators.  Here I am merely talking about
the function calls and aggregates whose lifetime has been extended to match that
of the return object.  Don't you want permission to reclaim those, if the return
object is reclaimed?

As far as implementation, when a heap object has access discriminants, we
automatically associate with it a chain of other objects which are to be
reclaimed and finalized if the heap object is unchecked-deallocated. This chain
can be used for these extended-life aggregates and function calls as well.

> The use of anonymous allocators is just bad. Janus/Ada puts out a
> warning that "this allocator will leak memory" on every such allocator
> (since we make no attempt to implement any special storage pools). And
> I think anyone whose program does not include "pragma Restriction
> (No_Anonymous_Allocators)" is nuts (since anonymous allocators are not
> required to be portable - the semantics you are talking about have
> never been more the Implementation Advice - the most ignorable IA in
> the entire Ada standard).

Again, the above isn't talking about anonymous allocators.

>> Since anonymous allocators as actual parameters are treated
>> essentially like aggregates from the point of view of storage
>> allocation, it seemed to make sense to give them the same master as
>> an aggregate and allow the same reclamation.
>
> This is not true; that is just IA which ought to be ignored (and
> surely is in our implementation).

There isn't really much of an implementation burden to treat these as
aggregates.  I can believe Janus doesn't do that, but you don't need a special
storage pool, since (with the existing rules) these are just being allocated on
the (secondary) stack like other aggregates.  I also don't think this is
actually just implementation advice.  The master of an anonymous allocator
passed to an access parameter is the innermost master of the call (see
3.10.2(14/2)), meaning at the very least that finalization is required when the
other parameters to the call are finalized.  I suppose you could leak storage,
though I don't see why you would want to given the approximate equivalence with
aggregates.

My suggestion is that in the case with referenceable/returnable/ designatable
parameters, they should still be treated like aggregates, from a finalization
and storage management point of view.  That seems pretty straightforward.

In any case I accept that you can't bring yourself to waste any breath on
anonymous allocators, but I will certainly suggest the equivalence with
aggregates in the ARG discussion.

>> Which part of the above do you find objectionable, and can you
>> explain the reason a bit further?
>
> Unless you convince people to *require* this behavior of all
> compilers, it is insane to depend on it. Moreover, the inability to
> control pools and deallocation for anonynous allocators make them
> pretty useless. The uses that you have promoted for years are too
> clever to be understandable -- when I see "new", I'm expecting an
> allocation into a global data structure, not something with a very
> short lifetime. It's just the wrong message. If you want a
> stack-allocated object, declare a stack allocated object! Or just use an aggregate directly - don't use access types unless you have to.

I accept you don't like these beasts, but they are pretty well defined at this
point, and their short lifetime is required according to 3.10.2(14/2).

> Most of this really was an attempt to give the advantages (and
> problems) of "in out" parameters to functions without doing that.
> Which was complete insanity - lets not continue to make a bad situation worse.

Even if you don't like a feature, you can't just ignore it. We should try to
define reasonable and consistent semantics for these things, and treating them
like aggregates and function calls seems consistent and doesn't introduce a
significant additional implementation burden.  But I realize you don't think it
is worth the effort, so we will have to agree to disagree on this one for now, I
guess.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April  28, 2009  5:46 PM

...
> > Your attempt to talk about "new objects" doesn't help, because that
> > isn't defined by the language, either. You could define the term
> > something like "the master of an expression is the master that would
> > be used for an entity declared by the execution of the expression."
> > Except that I'd probably replace "expression" by "construct" here.
>
> I really think this is misleading.  We talk about function calls and
> aggregates in 7.6.1(13/3), and that is where we need to change the
> rules to say that the master of the associated anonymous object is
> *not* the innermost master in this case, but is rather the master of
> the return object.  As mentioned above, when I say "new objects"
> read "aggregates and function calls" (and if you can stomach it,
> anonymous allocators passed to access parameters).

I sort of see your point, although this seems more complex than necessary,
especially if this doesn't cover everything.

My original idea, after all, was to get rid of the master that is the function
call in this case, and apply these rules to all parameters of such a call. We
gave that up after having to add accessibility checks to the parameters (don't
want to do that for elementary parameters, as it would imply checking the
accessibility of values - yuck).

I suppose an alternative would be to leave the accessibility checks as they are
but eliminate the extra master (the one on the function call) in this case. That
would also give you the "right" master for the anonymous allocators (I don't see
how to do that otherwise; it doesn't fit into the 7.6.1(13/3) wording at all -
and they're not "returnable parameters").

> >>> <      If such a call has an actual parameter that
> >>>>      is an anonymous allocator, then its master is that of the function
> >>>>      call's return object also (I think this partially addresses
> >>>>      concerns about the local scope of anonymous allocators).
> >>>>      If the call's return object is used to initialize an
> >>>>      allocator, and
> >>>>      that object is later unchecked-deallocated, then the implementation may
> >>>>      at that time also finalize and deallocate these "new" objects
> >>>>      associated with the call (including anonymous allocators).
> >>> That sounds worse than co-extensions: it means initializing a
> >>> storage pool with *every* call. Or something. Death to co-extensions!
> >> Can you explain the issue further?  This is just an implementation
> >> permission to *avoid* a storage leak.  That is, if there is an
> >> aggregate or a function call as an actual parameter, and it is
> >> required to live at least as long as the return object, then the
> >> implementation is *permitted* to finalize and reclaim the actual
> >> parameter object when the return object is unchecked-deallocated.
> >
> > I don't think that this semantics could be implemented without
> > amazing handstands. How would you keep track of the source of the
> > part of the parameter which is then returned into an allocated
> > object? If the same function was called passing an existing access
> > value, you surely would not want this behavior. And the call site
> > can have no idea (short of
> > inlining) as to what the function is doing internally -- maybe those
> > pointers came from fresh allocations or from global variables.
> >
> > You'd have to carry source information along with every anonymous
> > access parameter (the accessibility level would not be enough, as
> > passing a locally allocated object would have to be treated
> > differently). And you'd have to carry information about the source
> > of the pointer along with every component that it might be placed into.
>
> The above doesn't mention anonymous allocators.  Here I am merely
> talking about the function calls and aggregates whose lifetime has
> been extended to match that of the return object.  Don't you want
> permission to reclaim those, if the return object is reclaimed?

Umm, the first part of the very first sentence of the quoted text above ends
"...is an anonymous allocator, ...". So I'm mystified as to what you think you
are talking about, because it surely cannot be the quoted text that *I* was
responding to!

Maybe you are reading these two "if"s as completely separate points? I thought
that the second was predicated on the first, because otherwise it is nonsense.

> As far as implementation, when a heap object has access discriminants,
> we automatically associate with it a chain of other objects which are
> to be reclaimed and finalized if the heap object is
> unchecked-deallocated.
> This chain can be used for these extended-life aggregates and function
> calls as well.

If we are talking about new anonymous objects, I don't see how that helps.
They're allocated on the stack, of course, and go away whenever that appropriate
time is.

Oh, I see the problem -- these allocated objects *couldn't* be allocated on the
stack, because they have to live much longer. That's simply not going to work,
that is far too much complexity. That would require *every* function call (that
might have a returnable parameter) to do all of the handstands that a
coextension needs. That is insane, especially in a shared generic context.

The implementation you propose seems like a nightmare, given that there is no
reason to assume that the function actually uses these parameters are part of
the result. I suppose you could build a separate chain for pieces that are
parameters of the call, and then as the last step of the allocator, add them to
the existing chain of objects that is part of the object. But that addition
would have to eliminate duplicates (if a parameter was used as an access
discriminant, it might already be on the list). In any case, it sounds like a
mess. (And it would be very hard to do with our stack-based intermediate code;
we'd have to put any extra such lists in memory, which is a royal pain as the
addresses would have to passed to any routine that might need to use them. We do
that with task masters and rings, and it always seems to lead to a bug where
someone thought that they were not needed.)

I think the sensible thing to do is to ban "accessor" functions from being used
to initialize an allocator. (That would have to be a combination legality and
dynamic check.) That's simpler than trying to make everything a coextension. But
that does mean seem to suggest that we *must* have a syntactic way to
differentiate these functions -- you surely would never want to get one by
accident.

(I know you seem to think there is some wonderous value to this complexity, but
you are clearly the only one of our little group that feels that way. I can't
see any reason to add any more such complexity.)

> > The use of anonymous allocators is just bad. Janus/Ada puts out a
> > warning that "this allocator will leak memory" on every such
> > allocator (since we make no attempt to implement any special storage
> > pools). And I think anyone whose program does not include "pragma
> > Restriction (No_Anonymous_Allocators)" is nuts (since anonymous
> > allocators are not required to be portable - the semantics you are
> > talking about have never been more the Implementation Advice - the
> > most ignorable IA in the entire Ada standard).
>
> Again, the above isn't talking about anonymous allocators.

Which is still news to me.

> >> Since anonymous allocators as actual parameters are treated
> >> essentially like aggregates from the point of view of storage
> >> allocation, it seemed to make sense to give them the same master as
> >> an aggregate and allow the same reclamation.
> >
> > This is not true; that is just IA which ought to be ignored (and
> > surely is in our implementation).
>
> There isn't really much of an implementation burden to treat these as
> aggregates.  I can believe Janus doesn't do that, but you don't need a
> special storage pool, since (with the existing rules) these are just
> being allocated on the
> (secondary) stack like other aggregates.  I also don't think this is
> actually just implementation advice.  The master of an anonymous
> allocator passed to an access parameter is the innermost master of the
> call (see 3.10.2(14/2)), meaning at the very least that finalization
> is required when the other parameters to the call are finalized.  I
> suppose you could leak storage, though I don't see why you would want
> to given the approximate equivalence with aggregates.

Because "new" means allocate from a storage pool! It is just outright misleading
to use it for any other purpose. I don't buy this supposed equivalence with an
aggregate - if you want an aggregate, write an aggregate, not new!

I see you have managed with 3.10.2(14/2) to encode this insanity into the
language. I nearly quit over the entire business of coextensions -- I didn't
mainly because I couldn't *afford* to quit, so I could justify making a stand on
principle.

In the absence of clear customer demand (which does not exist), I will never
implement any part of 3.10.2(14-14.4/2). I think I will just make such
allocators completely illegal (which is what the language should have done;
there is no contract problem because anonymous types are purely syntactic). This
has the advantage of remaining compatible with the language.

In any case, finalization is not the same as storage management. These are
completely separate in the language, and in our compiler, and probably in
everyone else's compiler, too.

> My suggestion is that in the case with referenceable/returnable/
> designatable parameters, they should still be treated like aggregates,
> from a finalization and storage management point of view.  That seems
> pretty straightforward.

Well, they *are* aggregates (or function calls), so now you are talking
something circular. An access parameter is never a returnable parameter. We
could change its master to be the same as that of a returnable parameter
(probably a good idea, in fact), but it still will not be a returnable parameter
(it won't need an accessibility check as they do).

> > Most of this really was an attempt to give the advantages (and
> > problems) of "in out" parameters to functions without doing that.
> > Which was complete insanity - lets not continue to make a bad situation worse.
>
> Even if you don't like a feature, you can't just ignore it.
> We should try to define reasonable and consistent semantics for these
> things, and treating them like aggregates and function calls seems
> consistent and doesn't introduce a significant additional
> implementation burden.  But I realize you don't think it is worth the
> effort, so we will have to agree to disagree on this one for now, I
> guess.

I don't want to ignore them, I want them to be made illegal! They're not
portable or understandable as it is, and Franco pointed out that they are
actively harmful to the use of anonymous access types. (You can never have
Unchecked_Deallocation of anonymous access types because of them, which prevents
any chance of making anonymous access types useful.)

We should have made anonymous allocators illegal in Ada 2005 rather than wasting
days of time (and weeks of implementation time) on coextensions, and that has
not changed. Why you continue to think this is a good thing is beyond my
comprehension.

****************************************************************


Questions? Ask the ACAA Technical Agent