Version 1.3 of ais/ai-00344.txt

Unformatted version of ais/ai-00344.txt version 1.3
Other versions for file ais/ai-00344.txt

!standard 3.09.01 (03)          04-02-29 AI95-00344/01
!class amendment 03-08-07
!status work item 04-02-29
!status received 03-04-21
!priority Medium
!difficulty Hard
!subject Allow nested type extensions
!summary
Type extensions are allowed in scopes more nested than their parent type. Checks are performed on function return and on allocators to ensure that objects of such a type extension do not "escape" to a less nested environment.
!problem
The restriction against nested type extensions makes it harder to instantiate certain generics in a local scope, if those generics internally use extension from some kind of standard library-level type, such as a storage pool, or a controlled type, or a stream. Such restrictions can "snowball," meaning that many generics in a system may only permit library-level instantiations.
!proposal
Repeal the accessibility check in 3.9.1(3). Replace it by something that still allows generic sharing (probably prohibiting extension in generic bodies).
Add accessibility checks on class-wide allocators and return statements.
!wording
Delete the last two sentences of 3.9.1(3).
Add after 4.8(5):
If the designated type of the type of the allocator is tagged, the accessibility level of the designated type shall not be statically deeper than that of the type of the allocator.
Add after 6.5(20):
If the result type is class-wide, a check is made that the accessibility level of the type identified by the tag of the result is not deeper than that of the master that elaborated the function body. If this check fails, Program_Error is raised.
!example
!discussion
[Editor's note: 3.9.1(4) is retained (which prevents extension in generic bodies), because the contract model issues discussed in the AARM remain. You can get similar issues from parameters which depend on formal types. See the e-mail below.]
IMPLEMENTATION ISSUES
When a type is extended in a more nested scope, the dispatching operations may need access to data declared in this nested scope. This will generally require a static link or global display. However, the caller of such a dispatching operation is generally "unaware" that they are calling such a nested dispatching operation. This means that the dispatching operation must deal with the static link or global display internally, using information that the caller passes in.
The simplest approach seems to be as follows:
When defining the record layout for a nested type extension, an additional implicit component is allocated to contain the static link, or a pointer to a saved global display, or in the case of a shared generic, a pointer to an instantiation descriptor in which the static link or global display can be found.
!corrigendum 03.09.01(03)
!ACATS test
!appendix

From: Tucker Taft
Sent: Monday, April 21, 2003  3:44 PM

The current rule that disallows extending a type
at a more nested scope than the parent type has always
been a bit troublesome.  I believe that during the
Ada 9X process we felt quite strongly that this was
the only safe rule.  However, as I have thought
about this rule more recently, it seems to be possible
to avoid problems by simply performing accessibility
checks at allocators and function returns.

The danger from nested extensions is that a global variable
will end up with a tag of a nested type extension.  But this
can only happen in two ways:

   1) A function with a classwide result type returns an object of a
      type extension declared within it.

   2) An allocator for an access-to-classwide type declared at an outer
      level is evaluated for a more nested type extension.

Both of these situations seem eminently checkable.  (1) is very similar
to the current accessibility check on return-by-reference objects.
(2) is similar to a check that is required when the designated
type for an allocator has an access discriminant initialized by
local_obj'access.

Is there some other problem that I have forgotten?

Should we consider an amendment to allow nested extensions?
It would seem to eliminate a number of other annoying problems
where certain generics cannot be instantiated at a nested level.

****************************************************************

From: Randy Brukardt
Sent: Monday, April 21, 2003  4:15 PM

> Should we consider an amendment to allow nested extensions?

No.

> It would seem to eliminate a number of other annoying problems
> where certain generics cannot be instantiated at a nested level.

No, this is the same as saying that we're eliminating all useful generic
sharing. (I define "useful" generic sharing as that done without looking
into the body. If you can look into the body, there is no need or reason for
language rules to be involved, the sharing 'contract' now is essentially the
entire program.)

It is critical that access-to-subprogram types (which include tags) never,
ever 'leak' out of the generic body (because such subprograms have a
'different' profile than the 'apparent' profile used for checking language
rules). The accessibility rules happen to have that effect; but clearly they
were not designed for that. If the accessibility rules are weakened or
removed, you simply have to add them back in some other way. Or, simply drop
generic sharing and the contract model. (I see no reason to keep any of the
generic-sharing rules if any of them are eliminated.)

I started to write up a long LSN/web page on the topic of generic sharing,
because I seem to need to re-explain it every couple of months. Other things
intervened, but it appears that I still need to do so...

****************************************************************

From: Tucker Taft
Sent: Monday, April 21, 2003  5:15 PM

Would it help if you still had to put type extensions
in the spec as opposed to the body (if they are at the
same accessibility level as the spec, that is)?  I thought that
was the thing which simplified generic sharing.  Allowing
extensions inside a subprogram body seems like a different
sort of thing, and I presumed would not break sharing.

****************************************************************

From: Randy Brukardt
Sent: Monday, April 21, 2003  5:37 PM

The only way that it wouldn't break sharing is if there never, ever was a
way to call them from outside of the body (including from calls outside of
the body). That clearly includes dispatching calls, but is not limited to
that. I don't see what such a type (or object) would be useful for.

Indeed, I can't see what your proposed extension is useful for. Your
proposed limitations seem to prevent the type from being used for
dispatching. That presumably includes implicit dispatching (that is,
finalization and storage pools), since that is likely to be implemented with
real dispatching (they certainly are in Janus/Ada). If it's not prohibited
from use in dispatching, I don't see how you expect it to work - you'd have
to include a static link or display in the tag - indeed in ALL tags. (The
'current' one wouldn't work, you could make external calls to the same level
and have the wrong link/display values. That's not a problem for local
variables because they're not accessible, but certainly it would be possible
with a dispatching call where the lack of visibility doesn't matter). That's
clearly a lot of overhead (because you'd have to 'switch' the display or
static link on every dispatching call); there had better be a pretty solid
purpose behind the extension.

****************************************************************

From: Tucker Taft
Sent: Monday, April 21, 2003  6:45 PM

Yes, the static link/display is the issue, I suppose.
It would seem you could implement this by putting
the link/display information in any object of a nested type
extension.  We do something analagous for dealing
with nested protected types, I believe, because we
pass a pointer to the protected object in the same
register we normally use for passing the static
link.  If the protected type is itself nested, the
"real" static link is retrieved from the protected object.

Given that nesting is usually shallow, even a display
would only occupy a few words in the tagged object.

Does that sound feasible?

****************************************************************

From: Randy Brukardt
Sent: Monday, April 21, 2003  5:37 PM

Well, that would work, but its completely non-responsive to my question. In
Janus/Ada, all objects have a static size. Anything dynamically sized is
allocated from a storage pool and stored in a fixed size descriptor (which
is often just a pointer). Thus, there is only the options of including
enough space for everything (8 levels in Janus/Ada) in all objects or
allocating the space off of one of the storage pools (with the associated
runtime overhead).

But this is equivalent to putting it into the tag, just a lot more expensive
in space overhead (unless there is only one object). So I don't see any
difference. Moreover, it does nothing to address the distributed runtime
overhead: every dispatching call has to extract the tag or display from
somewhere (doesn't matter where), and arrange to use it on the call. That's
not too expensive for a static link, but it is very expensive for a display
(because you have to save the old one somewhere, then copy in the new one,
then reverse the process on return). And there is no hope to optimize it out
(unless you're into link-time code generation), because any program can have
a nested type added in the future. The absolute best you can do is check at
runtime whether you have to do it, but even then, there is a gob of code
needed to provide the rarely-to-be-used support (and the extra test and
jump). That's the definition of distributed overhead.

> Does that sound feasible?

Feasible, but not practical as noted above.

Do you have any realistic example of what this would be good for? I still
see no value to it, and you haven't explained how you would use it. You have
to explain what is so important to take on the sort of distributed overhead
that you are discussing here. I'm not going to waste any more time
discussing implementation until I understand what it would be used for.
Otherwise, I'm just answering the same old questions; I have no opportunity
to suggest an alternative that might not be such a problem.

****************************************************************

From: Tucker Taft
Sent: Monday, April 21, 2003  8:38 PM

I'm not sure I understand why this would introduce a dynamic size.  If
inside a subprogram body I extend a type declared outside
the subprogram, I know I am creating a nested extension,
and only objects of that type need a static link/display,
and I "know" exactly how deeply nested I am.
(I wouldn't want to put them in the type descriptor pointed
to by the tag because we allocate and initialize that
statically.)

When dispatching operations of this type are called, they are treated
as though they were declared at the same level as the root type
as far as the caller is concerned.  If they need access to data
at levels more nested than that, they have to go indirect via
the static link/display stored in the tagged object.

I don't think this implies distributed overhead, since only
dispatching operations for nested extensions would have to
use the "funny" static link/display entries.

> ...
>
> Do you have any realistic example of what this would be good for? I still
> see no value to it, and you haven't explained how you would use it. You have
> to explain what is so important to take on the sort of distributed overhead
> that you are discussing here. I'm not going to waste any more time
> discussing implementation until I understand what it would be used for.

I agree that if it implies all of the distributed overhead you
suggest, then it would not be a good idea.  In fact, if it involved
any distributed overhead at all it would be a bad idea.  But
it seems like the overhead could be kept within the place
where a nested extension appears.  Disallowing such
things in generic bodies would seem a reasonably limitation,
since type extensions are not permitted in generic bodies
now anyway.

But I presume the benefits are obvious.  Right now, if
you define a root type like "Finalization.Controlled" or
something else, and then create useful code that deals
with the root-type'Class, this is only usable with types
declared at the same nesting level as the root type.

****************************************************************

From: Randy Brukardt
Sent: Monday, April 21, 2003  9:31 PM

> I'm not sure I understand why this would introduce a dynamic size.  If
> inside a subprogram body I extend a type declared outside
> the subprogram, I know I am creating a nested extension,
> and only objects of that type need a static link/display,
> and I "know" exactly how deeply nested I am.
> (I wouldn't want to put them in the type descriptor pointed
> to by the tag because we allocate and initialize that
> statically.)

OK, but how do you know whether objects of that type have such a thing, and
where it is?? This can't happen by magic: a dispatching call needs exactly
the same layout of components that it is going to deal with in all cases.

> When dispatching operations of this type are called, they are treated
> as though they were declared at the same level as the root type
> as far as the caller is concerned.  If they need access to data
> at levels more nested than that, they have to go indirect via
> the static link/display stored in the tagged object.

You want a wrapper around the wrapper? Yikes - I can't figure out how this
works as it is. That is going to be one messy wrapper, because it has to
have a stack frame of its own (to save the current display in); which means
the parameters will have to be copied.

> I don't think this implies distributed overhead, since only
> dispatching operations for nested extensions would have to
> use the "funny" static link/display entries.

No, I suppose you're right (presuming that such wrappers can be built - I
think there might be problems if the root type is nested. What is the level
of the wrapper in that case? I suppose one can just forget such a case,
'cause it's useless.)

> ...
> Disallowing such
> things in generic bodies would seem a reasonably limitation,
> since type extensions are not permitted in generic bodies now anyway.

But that's a fallacy. The only reason type extensions are prohibited in
generic bodies is because of the assume-the-worst accessibility check.
You're proposing to eliminate the accessibility check; then there is no
longer any legitimate reason for prohibiting type extensions in a generic
body. We'd have to add a specific rule for that - and it would have no real
justification - it would essentially be there only because of me. People
would picket my office if they found out. :-)

> But I presume the benefits are obvious.  Right now, if
> you define a root type like "Finalization.Controlled" or
> something else, and then create useful code that deals
> with the root-type'Class, this is only usable with types
> declared at the same nesting level as the root type.

Maybe to you, but I don't see the "benefits" of declaring a nested type. The
only times I've wanted to do it is in Q&D test programs where declaring a
separate package is a pain. But that 'pain' is a benefit in real, large
systems - it prevents the main subprogram from containing stuff that really
doesn't belong there. Moreover, if you declare a nested type, it can't
(usefully) be inherited from -- which means you're losing some of the
benefits of O-O. So, you almost always want a package for your type
extensions.

If the goal is to get subprogram-level finalization, I'd rather see that
done in a first-class manner (which would be both easier to write and more
obvious that it is unusually expensive). That requires some syntax. But I
see no benefit at all to extending anything other than
Finalization.Limited_Controlled in a subprogram, and that being the case,
I'd rather leave well enough alone and only deal with the case that is
interesting.

****************************************************************

From: Pascal Leroy
Sent: Wednesday, April 23, 2003  5:07 AM

I don't think you have yet provided a compelling justification for this
change.  Other than the specific case of Controlled (which we might want
to address separately) I don't see that nested OO hierarchies are
particularly useful or frequent in practice.  I would not be opposed to
reopening AC-00050, but this is going to irritate Randy.

****************************************************************

From: Tucker Taft
Sent: Thursday, April 24, 2003  10:15 AM

> ... But I
> see no benefit at all to extending anything other than
> Finalization.Limited_Controlled in a subprogram,...

I'm surprised others haven't run into this before.
One of the biggest "privacy" breakages is that
a generic can't be instantiated inside a subprogram
if it happens to implement any of its private types
using type extension of some type declared outside
the generic.

This is particularly annoying for "container"-type
generics, where one would really like to be able to instantiate
and declare a local hash table, for example, but can't do it if
it happens to declare a local storage pool type.

It is often possible with some amount of pain to
move the instantiation, while keeping the local
hash table object (presuming the instantiation declares
a type rather than havings its own internal table object).
But this gets nastier if you are yourself writing
a generic, and the element type is a formal type.

Ultimately you end up with a whole bunch of generics
that can only be instantiated at library level.

****************************************************************

From: Randy Brukardt
Sent: Monday, March 1, 2004   9:12 PM

Tucker wrote in the AI: [Editor's note: Version /01.]

> When defining the record layout for a nested type extension, an
> additional implicit component is allocated to contain the static link,
> or a pointer to a saved global display, or in the case of a shared
> generic, a pointer to an instantiation descriptor in which the
> static link or global display can be found.

Of course, this doesn't work in the generic body for all of the reasons that
previously been discussed.

The accessibility check in generic bodies prevents all kinds of things that
can't be implemented -- and the problems arise from issues that have nothing
to do with accessibility. Every subprogram that can be dispatched to or
accessed via an access-to-subprogram needs a thunk: both to add the hidden
generic parameter (which we've talked about at length), and also to convert
the parameter passing between the implementation for the formal and the
implementation for the actual.

If you were to put the instantiation descriptor into tagged objects that are
extended in generic units, you could declare a wrapper that added that to
the call for dispatching calls. And the tagged object is always passed by
reference, so that's OK. But other parameters might have different passing,
and you couldn't reconcile that in a wrapper (you don't know the actual
types involved).

Which brings up an interesting question: is a operation (potentially)
primitive if it depends on a formal type? We never had to answer this in Ada
95 for bodies; we said that it is true in specs. Consider something like:

    type Root is tagged ...
    procedure Oper (Obj : in out Root; Arg : in Natural);

    generic
       type Priv is private;
    package Gen is
       ...
    end Gen;

    package body Gen is
       type New_Type is new Root with ...;
       procedure Oper (Obj : in out New_Type; Arg : in Priv); --??
       ...
    end Gen;

    package Inst is new Gen (Natural);

Now, does Inst.Oper override the inherited one? If this occurred in the
spec, it would. But if you do this in the body (which is currently illegal),
then sharing without body knowledge is impossible, because you need to know
the actual type(s) in order to determine the contents of the tag. (One also
has to wonder about the contract model implications here.)

****************************************************************

From: Tucker Taft
Sent: Monday, March 1, 2004   9:42 PM

> Of course, this doesn't work in the generic body for all of the reasons that
> previously been discussed.

Sorry, I really don't understand your model enough
to suggest something intelligent.  In any case,
I was presuming that the restriction 3.9.1(4) would remain:

   A type extension shall not be declared in a generic body
   if the parent type is declared outside that body.

Perhaps that restriction makes no sense if we are allowing
type extension in arbitrary non-generic bodies, but I
presumed it was still important for shared generics
for some reason.

...
>     package body Gen is
>        type New_Type is new Root with ...;
>        procedure Oper (Obj : in out New_Type; Arg : in Priv); --??

This would violate 3.9.1(4) I believe.

>        ...
>     end Gen;

****************************************************************

From: Ramdy Brukardt
Sent: Monday, March 1, 2004   9:58 PM

>    A type extension shall not be declared in a generic body
>    if the parent type is declared outside that body.

Sigh. I didn't remember this rule; I've always thought that the accessibilty
check itself preventing body tagged types.

Anyway, the AARM primarily discusses accessibility reasons for this rule.
Those obviously would be gone. But it also discusses problems with abstract
subprograms causing contract problems. Those would still remain, so we'd
still have a reason for the rule (although we'd need to rewrite the
explanation in order for it to make sense). I think you could construct an
example where whether the program was legal or not depended on the actual
type used for another parameter similar to the example I showed in my last
message. So it looks like there are real contract problems with that, so
there isn't a problem.

But I do think a brief discussion of why 3.9.1(4) remains (and that it
exists!) would be valuable in the AI. Otherwise, people might be expecting
more out of this than they'll actually get.

****************************************************************

From: Tucker Taft
Sent: Thursday, April 1, 2004  11:19 AM

Randy, Steve Baird, and I have been having a bit
of a side-bar discussion on paragraph 3.9.1(4) and
how it relates to AI-344 on allowing extending
in nested scopes.  The most recent version of AI-344
did not touch 3.9.1(4) which disallows extending
in a generic body a type defined outside it,
since we would be allowing extension in all
kinds of other contexts.

After a few back-and-forths with Randy, we
came up with the following wording for 3.9.1(4)
which seems to be satisfactory, and
avoids contract model issues associated with extending
types with unknown number of functions that require
overriding (due to a controlling result).  It also
allows slot numbers to be assigned for any new
operations, since the parent is not a formal type.

    Within the body of a generic unit, or the body of
    any of its descendant library units, a tagged type
    shall not be declared as a descendant
    of a formal type of the generic unit.

This is carefully worded to handle child units, as well
as interfaces (hopefully).

The net effect is that you *can* extend a type
declared outside the generic body, so long as it
is not a descendant of a formal.  The anticipated
implementation model is still that the new type
would have a component that pointed to a block
of information which would contain static link/
display/instance descriptor/whatever.  Wrappers
for the dispatching operations would load this
information into appropriate registers, etc.,
and then call/jump to the user-defined primitive
subprograms.  These are essentially the same
thing as "trampolines" which are used by GNU C
and friends to support nested functions.

I have attached below the mail sent by Steve Baird,
Randy, and myself, for the record.

If there are no major objections, I will include
this change in the next version of AI-344.

----------

 From Steve, 3/29/04, 6:54 PM:

I strongly want to be able to instantiate the various container generics
anywhere (e.g. inside of a subprogram which is inside of a generic body),
even if they are implemented using controlled types.

This runs afoul of 3.9.1(4) .

If you've already addressed this problem, then this message can be
ignored.

Given AI-344's runtime checks, the only remaining need for 3.9.1(4) stems,
as I understand it,  from the case described in AARM 3.9.1(4.a-j).

This case can only arise if the parent  type is abstract.

Could 3.9.1(4) be relaxed so that it only applies to abstract types?

I'm not claiming this is elegant, but it would allow an important case.

----------

 From Tuck to Randy and then his reply (3/31/04):

 >> Hi Randy,
 >>     There seems to be some interest in pushing forward AI-344, nested type
 >> extensions.  Paragraph 3.9.1(4) disallowing extending in generic
 >> bodies has been an issue.  It seems somewhat arbitrary, if we allow
 >> extending pretty much anywhere else.  I was thinking that the following
 >> might solve the problem:
 >>
 >> Replace 3.9.1(4) with:
 >>
 >>     A type extension shall not be declared in a generic body if the parent
 >>     type is a formal type of the generic.
 >>
 >> Would this be implementable in your compiler?  Notice that I allow the
 >> parent to be descended from a formal type, but not a direct use of the
 >> parent type.  That avoids the danger that the parent type is class-wide.
 >>
 >> Presumably with AI-251, this would become:
 >>
 >>     A type extension shall not be declared in a generic body if the parent
 >>     type or one of the ancestor interface types is a formal type of the
 >>     generic.
 >>
 >> Can you think about this a bit and let me know?


I thought about this some last week while I was working on the minutes.

One comment is that I never saw any actual reaction to keeping 3.9.1(4). The
only person who seemed to care was you. So I'd suggest not giving this rule
an importance out of proportion to reality. (After all, most types can be
hoisted to the spec.)

First of all, it will be very hard to implement any part of AI-344 in our
compiler. We implicitly add generic descriptor parameters for all
subprograms (declarations and calls) if they are declared in a generic.
Doing this causes the other parameters to be renumbered automatically. The
implementation you've suggested for AI-344 requires this not to be true for
wrappers for tagged types. That will require major restructuring in the
compiler - either the wrappers would have to be created by a totally
separate set of code (meaning the duplication of the 1500 lines of call
generation code - sounds very bad) or the automatic, under the covers
operation of generics would have to be gotten rid of. Neither is likely to
be practical; I'd expect that a likely implementation of AI-344 would
completely disallow it in generics.

In any case, I don't want to stand in the way of progress here (unless it is
clearly going too far, like the generic instantiation junk). So most of the
discussion is more hypothetical than what actually can be implemented. It's
about what could be implemented with heroic efforts to provide a usable
solution, as opposed to what's clearly impossible or outrageous.

Besides implementation problems, that rule is needed to avoid contract model
issues. I hadn't thought about class-wide; the AARM mainly discusses it in
terms of must-be-overridden functions. The example is something like:

      generic
          type Foo is tagged private;
      package Gen is
          ...
      end Gen;

      package body Gen is
          type New_Foo is new Foo with private;
             -- ??
      end Gen;

      type Bar is tagged ...;
      function A_Bar return Bar;

      package New_Gen is new Gen(Bar); --??

Since A_Bar is a primitive function returning an object of Bar, it must be
overridden. But how would New_Foo know to do that? It's clear that the type
cannot depend on a formal.

There is also a practical contract violation here. We have a meta-rule that
tags always can be created statically. We certainly adhere to that rule by
generating the tags for types declared in a generic at instantiation time.
There is no way to do that for a type that depends on a formal that declared
in the body. (And note that here, this is true even if there is an
intermediate type declared in the specification, because that cannot tell us
all of the overriding needed.)

      generic
          type Foo is tagged private;
      package Gen is
          type Foolike is new Foo with null record;
      end Gen;

      package body Gen is
          type New_Foo is new Foolike with private;
          procedure Some_Operation (A : New_Foo); --??
      end Gen;

      type Bar is tagged ...;
      function Some_Operation (A : Bar);

      package New_Gen is new Gen(Bar);

Some_Operation for New_Foo is overriding and must be replaced in the tag for
New_Foo. But how is an implementation supposed to figure that out? It cannot
know whether Foo has a primitive operation Some_Operation.

This could be "solved" by declaring Some_Operation to be not overriding. But
that would violate the principle that a generic works the same as a
non-generic. (We certainly aren't going to declare this as non-overriding
outside of a generic, that would be a gratuitous incompatibility.) Moreover,
there isn't any way I can think of to create this tag at runtime (short of
including the entire Ada symboltable and code in the program, which isn't
remotely practical). Because you'd have to pass the name and parameter and
result profile with every slot in order to figure out any if there is any
overriding. (Usually we assign slot numbers to these, but in this case
that's not possible.)

This gets even more interesting when there is a package spec in the body:

    package body Gen is
       package Nested is
          type New_Foo is new Foolike with private;
          procedure Some_Operation (A : New_Foo); --??
       end Nested;
    end Gen;

Here, Some_Operation is clearly a primitive of New_Foo. How do we assign a
slot number? It might override something that comes from the formal, or it
might not.

So I'm certain that any relaxation of the rule would have to disallow
anything that depended on a formal in any way. (Direct dependence is not
enough.)

I haven't been able to decide if derivations from unrelated types (such as
"Controlled") are a problem or not. Clearly, you could use the same
techniques that you use for non-generic types. The problem (if there is one)
would occur if you need any local data space for the wrappers. This I don't
really know.

At the meeting, we discussed the fact that the wrappers probably would need
a runtime data structure (linked through the dynamic stack, perhaps, or
perhaps on the finalization chain) to find the appropriate display or static
link. Presuming that implementers sign up for that overhead (which is an
open question to me, it seems significant, but at least not distributed),
then that mechanism would provide a mechanism for finding any data needed.
So on that basis, it seems like it *might* be possible to implement. (Of
course, generating the wrappers would be a big problem because of the
generic parameter issue mentioned previously. But the question is that is
would be possible.)

So I'd probably not oppose strenuously a rule like:

      A type extension shall not be declared in a generic body if the parent
      type is descended from a formal type of a generic.

(Note that we have to worry about nested and child cases here, too. It
doesn't matter whose formal it is, we don't want to be dependent on it. But
we don't want this to apply to types declared in instances, so I'm not sure
if the above wording is right. Perhaps we need to say something about "the
generic or its ancestors".)

P.S. I presume that we'll attach this discussion to the AI? I don't want to
have to write this note again!!!

-------------------
 From Tuck to Randy and then his reply again (3/31/04):

...

 >> Since we disallow use throughout the generic body, I think this handles
 >> nested generics and subunits (since they are "plugged" into their stub
 >> prior to legality checks).  Children seem like a problem as you say,
 >> since their bodies are not considered nested within the ancestor.
 >> The "ancestor" of a generic is not always well defined, so
 >> perhaps the following:
 >>
 >>    Within the body of a generic unit, or the body of any of its descendant
 >>    library units, a tagged type shall not be declared as a descendant
 >>    of a formal type of the generic unit.
 >>
 >> This hopefully covers interface types as well.


I think this works.

****************************************************************

From: Randy Brukardt
Sent: Friday, April 2, 2004  9:24 PM

Unfortunately, I've thought of another case where there is trouble creating
a tag in the body without knowing the actual types involved. Consider the
following:

    type Base is tagged ...
    procedure Proc (Obj : in out Base; Arg : in Integer); -- (1)

    subtype Short_String is String (1..10);

    generic
        type Frob is private;
    package Gen2 is
        ...
    end Gen2;

    package body Gen2 is
        type Der is new Base with ...;
        procedure Proc (Obj : in out Der; Arg : in Frob); --(2)
        ...
    end Gen2;

    package Short is new Gen2 (Short_String);
    package Int is new Gen2 (Integer);

In Short (1) and (2) have different profiles, and thus overload each other.
Therefore (2) gets a new slot (which can be calculated at compile-time,
given the above rule).

In Int, however, (2) overrides (1). That means (2) is supposed to get the
slot of (1).

In simple cases like this one, it probably would be possible to figure out
the slots by some sort of run-time tag modification. If there were multiple
such parameters, though, it could get pretty messy.

There also is a practical problem in our implementation. The parameter
passing implementation of Arg in (1) would be copy-in; while for Arg in (2)
it would be by-reference with the copy made for by-copy types at the call
site. Thus, if the wrapper for (2) was just stuck into the slot for (1), it
would have the wrong parameter passing code. Not good.

I'd suggest some sort of rule to fix this problem, but I can't think of
anything off-hand that doesn't throw the baby out with the bathwater. The
easy rule (the tagged type cannot have any primitives which depend on
generic formals) would prevent pretty much everything useful. And for types
like Controlled, it doesn't matter anyway, since the base type doesn't have
any operations that could cause problems.

But a rule that only eliminated problem cases would have to talk about the
intersection of possible profiles of all of the primitives, which seems
awful to describe and implement. Something like "Within the body...., a type
extension is illegal if a primitive operation of the type has one or more
operands which depend on a formal type of a generic, and the primitive could
override another primitive of the type in any possible instantiation of the
generic." (along with the previous restriction, of course).

I think this would still allow containers and Controlled in the bodies of
generics (presuming the containers were derived directly from Controlled),
but would avoid the problem cases.

Alternatively, we could simply say that such "overrides" don't override in a
generic body; but of course, that would mean that making a unit generic
could change its semantics. I don't like that much (and I'd think that
template implementations would like it less).

****************************************************************

From: Robert I. Eachus
Sent: Friday, April 2, 2004  10:08 PM

I guess I see the problem, but I don't see it.  The problem occurs only
if there is a call inside the generic:

Proc(Some_Der, N); -- where N is of type Integer.

In the first instance Short, this call goes to the (derived) Proc
declared inside Gen2.  Your assumption in the Int case is that this call
should go to the  explicitly declared Proc.  But I don't see how it
can.  The only visible Proc inside the generic with an Integer parameter
is the implicitly declared one.

If Der and Proc were declared in the generic package spec, then there
could be external calls where the overriding must be recognized.  But
that happens only if the overriding is in the spec, which is not the
case in question here.

Now if Der was declared in the spec, and there was an explicit Proc in
the body, that could cause a problem. However, I think a rule that said
that such an overriding was not visible outside the (generic) unit would
follow the principle of least surprise.  I think this is the weird case
Randy is worried about.  However, I think that it may be one where a
more general fix is needed.  Notice that a "normal" overriding in the
body of a generic is fine.  It is a visible overriding inside the
generic as well as externally.  The compiler knows that it is an
overriding and can deal with it accordingly.  In this "accidental"
overriding case, only if the author of both the generic body and of the
instantiation is the same  (Ada language lawyer) programmer, would an
overriding visible (only) outside the generic be anything other than
surprising.

****************************************************************

From: Randy Brukardt
Sent: Friday, April 2, 2004  11:27 PM

No, the problem also occurs when there is a dispatching call on Base'Class.
Which Proc gets calls for an object of Der? That call doesn't have to be
inside of the generic; any class-wide call will exhibit the difference in
behavior.

Note that the use of overridding indicators would show the problem, as
neither "overriding" nor "not overriding" would be legal on Proc
(assume-the-worst in a generic body). But they wouldn't help...

In any case, I'm mostly concerned about the implementation effects on an
implementation that tries to share bodies. If there is "conditional"
overriding in the body, sharing is going to be very hard or impossible. And
since none of this is visible in the spec (we're only discussing 3.9.1(4),
which only applies to types completely declared in the body), there can be
no body-agnostic sharing in such a case.

****************************************************************

From: Tucker Taft
Sent: Saturday, April 3, 2004  7:41 AM

> Unfortunately, I've thought of another case where there is trouble creating
> a tag in the body without knowing the actual types involved. ...

I think we would want to treat this analogously to the
situation where a type has a private primitive.  It's slot
is not reused on a type extension even if a primitive with an equivalent
profile is declared outside the scope of the private
primitive.  Instead, the new primitive gets a new slot.

In this case, I would say that in a generic body, if a parameter's
types might be different in some instances, then the primitive
gets a new slot in all instances.  I can't imagine any
value to having overriding vs. overloading depend on the
actuals, since no code outside the instance is going to be
able to name this type.  Yes it could see such an object
at run-time via 'Class, but I don't think that argues
for making the overriding vs. overloading instance-specific.

In the example you give below, the declaration of
"Proc" (2) in the generic body would not override the
inherited primitive "Proc" (1), and in fact wouldn't even
be a primitive.  If Der were declared in a nested
package spec, then the new Proc (2) would be a new primitive
of Der and get a new slot, even if Frob turned out to be
Integer.

> ... Consider the
> following:
>
>     type Base is tagged ...
>     procedure Proc (Obj : in out Base; Arg : in Integer); -- (1)
>
>     subtype Short_String is String (1..10);
>
>     generic
>         type Frob is private;
>     package Gen2 is
>         ...
>     end Gen2;
>
>     package body Gen2 is
>         type Der is new Base with ...;
>         procedure Proc (Obj : in out Der; Arg : in Frob); --(2)
>         ...
>     end Gen2;
>
>     package Short is new Gen2 (Short_String);
>     package Int is new Gen2 (Integer);
>
> In Short (1) and (2) have different profiles, and thus overload each other.
> Therefore (2) gets a new slot (which can be calculated at compile-time,
> given the above rule).
>
> In Int, however, (2) overrides (1). That means (2) is supposed to get the
> slot of (1).

Naaah.  Overriding should be something that is decided before
instantiation for generic bodies.

> In simple cases like this one, it probably would be possible to figure out
> the slots by some sort of run-time tag modification. If there were multiple
> such parameters, though, it could get pretty messy.

Blech.  I can't imagine any value to allowing overriding vs.
overloading to depend on the actual when in a generic body.

> ...
> Alternatively, we could simply say that such "overrides" don't override in a
> generic body; but of course, that would mean that making a unit generic
> could change its semantics.

You aren't just making the unit generic (i.e. wrapping the
word "generic" around it).  You are also taking one of the
types and making it a formal type.  To me that implies you
want the behavior to be "generic" across changes in such
a type.  You don't want it to suddenly have a differnt
behavior if the type happens to match some parameter type
of some primitive of some tagged type that happens to be
used inside the body.  In fact, if you did that, you would
create an interesting capability to do a run-time test for
(untagged) type identity, which is *not* something we have
ever provided in a generic, and I don't think we want to
start now!

> ... I don't like that much (and I'd think that
> template implementations would like it less).

Having a template implementation doesn't change
my view.  Overriding vs. overloading within the body
should be decided when you compile the generic, not
at the time of the instance.

****************************************************************

From: Randy Brukardt
Sent: Saturday, April 3, 2004  4:46 PM

Tucker said, replying to me:

> > ...
> > Alternatively, we could simply say that such "overrides" don't override in a
> > generic body; but of course, that would mean that making a unit generic
> > could change its semantics.
>
> You aren't just making the unit generic (i.e. wrapping the
> word "generic" around it).  You are also taking one of the
> types and making it a formal type.  To me that implies you
> want the behavior to be "generic" across changes in such
> a type.  You don't want it to suddenly have a differnt
> behavior if the type happens to match some parameter type
> of some primitive of some tagged type that happens to be
> used inside the body.  In fact, if you did that, you would
> create an interesting capability to do a run-time test for
> (untagged) type identity, which is *not* something we have
> ever provided in a generic, and I don't think we want to
> start now!

OK, that seems like a good argument.

> > ... I don't like that much (and I'd think that
> > template implementations would like it less).
>
> Having a template implementation doesn't change
> my view.  Overriding vs. overloading within the body
> should be decided when you compile the generic, not
> at the time of the instance.

Of course, this means further futzing with the overriding rules. That
doesn't seem too pleasant, either. But the goal (being able to use
controlled types and containers in generic bodies) seems worthwhile.

****************************************************************

From: Tucker Taft
Sent: Sunday, April 4, 2004  1:26 PM

I'm not sure that's true (though you may have something specific
in mind).  The rules I can find generally say that everything
relating to overloading and overriding is decided when you
compile the generic body.  Visible part of instances are
different, but the body seems pretty uniformly to rely on
overloading/overriding decisions made when the generic body
is compiled.  E.g. 8.3(13), 8.3(26/1), 12.3(14), 12.3(18).

****************************************************************

From: Pascal Leroy
Sent: Monday, April 5, 2004  6:36 AM

But this is not new futzing.  As Tuck pointed out, declarations in the
body should work like declarations in the private part, and you probably
do quite a bit of futzing already to implement 12.3(18).  So you have a
firm foundation on which to build even more futzing.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April 6, 2004  4:22 PM

No, I was thinking about in the Standard, not in our compiler. But you and
Tucker have pointed out 12.3(18), so the Standard is already correct on this
point. No futzing needed. Some implementers might need to futz if they don't
follow 12.3(18) properly, but that's not a problem for this group to worry
about.

****************************************************************

From: Robert I. Eachus
Sent: Monday, April 5, 2004  1:46 AM

>No, the problem also occurs when there is a dispatching call on Base'Class.
>Which Proc gets calls for an object of Der? That call doesn't have to be
>inside of the generic; any class-wide call will exhibit the difference in
>behavior.

My head is starting to hurt.  Right now, Der is declared nested in a
generic package body.  You are proposing to assign it to a "global"
object of type Base'Class, so that it can be operated on outside the
generic.  But there is no way to do that directly, you have to have an
access type declared access (all) Base'Class.  Now you can use an
allocator for this access type inside the generic, and assign its value
to an object of the access type.  See RM 3.10.2(14&15).

So it seems to me that it would be much easier to do any required magic
as a part of that allocator.  An allocator for a class-wide type seems
pretty magic to me anyway.  In fact, in exactly the case we are
discussing, the magic rules in  3.10.2(20) and 3.9.1(3) seem to combine
to allow, for example, type extension in a generic body, the generic
body can be instantiated inside some procedure, and assignment of a
value created by allocator to an access object in the procedure body
would be legal.  In that case, there could be for example a generic
formal subprogram, that gets inherited in the generic, and must be
dispatched to.

So I guess to me it seems silly to distribute overhead to any case other
than allocators in package bodies for class-wide access types which
allocate an object of a tagged type declared in the generic package
body.   Furthermore since the painful cases require that the allocator
be in the sequence of statements for such a generic package, why not
just outlaw it and be done.  (If you have an object of the class-wide
access type in the package body, with an initial value, that object can
only be seen within the package body.  It is assignments in the sequence
of statements at the end of the generic package body that can cause
problems, and it seems to me to be simple enough to treat that scope as
statically deeper than the rest of the generic package.  I have created
packages with a sequence of statements at the end, I have even done it
with generic packages.  But assignments outside the generic--of an
object of a type created in the generic BODY?  Come on...

>Note that the use of overridding indicators would show the problem, as
>neither "overriding" nor "not overriding" would be legal on Proc
>(assume-the-worst in a generic body). But they wouldn't help...
>
>In any case, I'm mostly concerned about the implementation effects on an
>implementation that tries to share bodies. If there is "conditional"
>overriding in the body, sharing is going to be very hard or impossible. And
>since none of this is visible in the spec (we're only discussing 3.9.1(4),
>which only applies to types completely declared in the body), there can be
>no body-agnostic sharing in such a case.
>
I don't know what the "best" way to deal with this is.  But I think
Tucker and I are leaning the same way--allow the useful functionality
even if we have to keep some restrictions.  Allowing derived types in
generic package bodies while not allowing assignment of access values to
objects of the derived type outside the generic seems like a very
livable restriction.

****************************************************************

From: Randy Brukardt
Sent: Monday, April 5, 2004  1:45 PM

> My head is starting to hurt.  Right now, Der is declared nested in a
> generic package body.  You are proposing to assign it to a "global"
> object of type Base'Class, so that it can be operated on outside the
> generic.

Not at all. First of all, 'Der' isn't neccessarily nested; it's in the body,
but could be at library level depending on the instantiation. But in any
case, 'leaking' the object out isn't necessary.

All you need to do is call a class-wide routine on the base class with a
parameter of type Der. If that routine then does a dispatching call on Proc,
you can tell whether it's been overridden or not.

In any case, Tucker's argument seems strong enough to avoid the issue in the
first place, by simply saying that such things are not overriding. *That* we
can implement without trouble in a shared implementation.

****************************************************************


Questions? Ask the ACAA Technical Agent