Version 1.4 of ai05s/ai05-0135-2.txt

Unformatted version of ai05s/ai05-0135-2.txt version 1.4
Other versions for file ai05s/ai05-0135-2.txt

!standard 4.1.3(12)          10-01-15 AI05-0135-2/02
!standard 7.1(2)
!standard 7.1(3)
!standard 7.1(4)
!standard 7.1(5/2)
!standard 7.1(7)
!standard 8.3(25)
!standard 8.4(5/2)
!standard 8.4(16)
!standard 8.5.3(3)
!standard 8.5.3(4)
!standard 10.1.1(12.2/3)
!standard 12.3(2/2)
!standard 12.3(18)
!class Amendment 10-01-07
!status No Action (8-0-1) 10-10-29
!status work item 10-01-07
!status received 09-11-07
!priority Medium
!difficulty Medium
!subject "Integrated" nested packages
!summary
Add "package <P> is ..." to allow integrated packages and package renames.
!problem
One common idiom is to write a derived type to take a type declared within an instantiation (or some other unit) and bring it out to the same level as the instantiation (or into the new unit):
package T_Vecs is new Vectors(Element => T); type T_Vec is new T_Vecs.Container with null record;
This is often done to bring the type and its operations to this location. But often a new, distinct type is not what is wanted. Moreover, some related entities (classwide operations, exceptions, generic operations) don't get exported this way, so additional renaming or declarations are required to completely make the instance transparent.
This use of a derived type is really something of a kludge. What is really wanted here is a way to make all of the declarations declared in the instantiation visible as if they were declared at the same level as the instantiation (as opposed to within it).
Consider a package which needs to export both a private type and an instance of one of the container generics for that type.
Currently, this requires something like
package Pkg is package Inner is type T is private; private ... end Inner;
package I is new Some_Container_Generic (Inner.T); end Pkg;
What is wanted here is a way to make all of the declarations visible in Pkg.Inner (and perhaps Pkg.I; that's less clear) visible as if they were declared immediately within Pkg. This special case of the problem alone has been viewed as being sufficiently important to merit considering major changes in the language (see the 4 alternative versions of AI05-0074).
An unwanted nested package may also be introduced when a user wishes to declare a non-primitive operation for a tagged type, as in
package Pkg is type T is tagged ... ;
package Inner is function Not_Primitive (X : T) return T; end Inner; end Pkg;
As in the previous example, what is wanted here is a way to make all of the declarations visible in Pkg.Inner visible as if they were declared immediately within Pkg.
!proposal
A package or package rename which is declared immediately within the visible part of an enclosing package may be declared with angle brackets enclosing the first occurence of the package identifier. This indicates that the package is to be "integrated" into its parent; this means that clients of the parent have the ability to name the constructs declared within the integrated package as though they were declared within the enclosing package.
One scenario where this construct may be useful is in declaring a package that exports both a private type and a container generic instance for that type without requiring that clients deal with nested packages. This can be accomplished using this new construct as follows:
with Some_Container_Generic; package Pkg is package <Inner> is type T is private; private ...; end Inner;
package <I> is new Some_Container_Generic (T); -- exports type Valise end Pkg;
with Pkg; package Client is X : Pkg.T; Y : Pkg.Valise; use Pkg; Xx : T; Yy : Valise; end Client;
!wording
Modify 4.1.3(12) as follows
The selector_name shall resolve to denote a declaration that occurs immediately within the declarative region of the package or enclosing construct (the declaration shall be visible at the place of the expanded name see 8.3) {, or to denote a declaration that is use-visible by selection at the point of the selector_name (see 8.4)}. The expanded name denotes that declaration.
Replace 7.1(3) to allow optional "< package_id >" syntax
integrated_package_identification ::= < defining_identifier >
package_identification := defining_program_unit_name | integrated_package_identification
package_specification ::= package package_identification is {basic_declarative_item} [private {basic_declarative_item}] end [[parent_unit_name] . identifier ]
Modify 7.1(4):
If an identifier or parent_unit_name.identifier appears at the end of a package_specification, then this sequence of lexical elements shall repeat the defining_program_unit_name {or, in the case of an integrated_package_identification, the defining_identifier}.
Add after 7.1(5/2):
If a package's package_identification consists of an integrated_package_identification, then the package is said to be integrated (see 8.4). The declaration of an integrated package or package renaming (see 8.5.3) shall occur immediately within the visible part of an enclosing package or generic package. The package_identification of a generic_package_declaration shall not consist of an integrated_package_identification. The package_identification of a package_declaration of a library_unit_declaration (or of a package_renaming_declaration of a library_unit_renaming_declaration) shall not consist of an integrated_package_identification.
[We could have moved the legality rules after the static semantic rules and put the definition of "integrated package" into the static semantic section. But that would necessarily change the paragraph number of an unchanged legality rule; since it is very likely that compiler error messages refer to the paragraph numbers of legality rules, we prefer to avoid that. - Editor.]
Add after 8.3(25) (still in the Name Resolution Rules section):
A name shall not resolve to denote an integrated package or package renaming outside of the immediate scope of that declaration.
AARM note: This rule means that an integrated package or package renaming is, in effect, anonymous outside of its immediate scope. However, it remains visible. This peculiar distinction (i.e., visible but unnamable) is important because visibility of the declarations declared within an integrated package may depend on the visibility of the package (see 8.4). Because this is a name resolution rule, the following example is legal:
declare package P1 is package X is end X; end P1;
package P2 is package <X> is end X; end P2;
use P1; use P2;
package Y renames X; -- unambiguously renames P1.X begin null; end;
Add after 8.4(11) (i.e. append to the end of the Static Semantics section)
At any point where an integrated package or package renaming is either potentially use-visible or directly visible, and where an entity declared immediately within the package or renamed package is visible, the entity is potentially use-visible.
At the point of an expanded_name whose prefix denotes a package P (or a rename thereof) which immediately encloses the visible declaration of an integrated package or package renaming, any visible declaration declared immediately within the integrated package (or, in the case of an integrated package renaming, within the renamed package) is said to be "potentially use-visible by selection" at the point of the selector_name. In addition, if the declaration of an integrated package or package rename is "potentially use-visible by selection" at the point of a selector_name, then so are any visible declarations declared immediately within the package or renamed package. A declaration which is "potentially use-visible by selection" at the point of a selector_name is said to be "use-visible by selection" at that point unless
- the defining name of the declaration is not the same as the
selector_name; or
- a visible homograph of the declaration is declared in P; or - more than one such declaration is potentially use-visible
by selection at the point of the selector_name and at least one of them is not overloadable.
Add after 8.4(16) (in the Examples section)
Example of integrated packages:
generic type T is private; package G is function Make return T; end G;
with G; pragma Elaborate (G); package Pkg1 is package <Inner_Pkg_1> is type T is private; private type T is ... ; end Inner_Pkg_1;
package <Inner_Pkg_2> is new G (T); end Pkg1;
with Pkg1; package Pkg2 is X : Pkg1.T := Pkg1.Make; use Pkg1; Y : T := Make; end Pkg2;
Modify 8.5.3(3) to allow optional "< package_id >" syntax:
package_renaming_declaration
::= package_identification renames package_name;
Add after 8.5.3(3.1/2) (Legality Rules):
If a package rename's package_identification consists of an integrated_package_identification, then the package renaming is said to be integrated (see 8.4). The package_name of an integrated package_renaming_declaration shall not denote a limited view of a package. The package_name of an integrated package_renaming_declaration shall not denote (directly or through renaming) a package whose declarative region encloses the package_renaming_declaration.
Modify 10.1.1(12.2/3) (as modified by AI05-0129-1) to disallow limited views of integrated packages:
For each nested package_declaration directly in the visible part {which does not declare an integrated package}, a declaration of the limited view of that package, with the same defining_program_unit_name.
Modify 12.3(2/2) to allow optional "< package_id >" syntax:
generic_instantiation ::= package package_identification is new generic_package_name [generic_actual_part];
Add after 12.3(18) (i.e. append to the end of the Static Semantics section):
If a package instance's package_identification consists of an integrated_package_identification, then the package instance is said to be "integrated" (see 8.4).
!discussion
Just to summarize, adding angle brackets to a package or package rename declaration has four main effects:
1) It is as though an implicit use_clause were added
immediately after the package declaration, so that visible declarations in the integrated package's spec become potentially use-visible within the enclosing package.
This allows:
package P is package <Q> is X : Integer := 0; end Q; -- use Q; Y : Integer := X; end P;
2) It is as though an implicit use_clause were added immediately
after any use_clause that applies to the enclosing package.
This allows:
package P is package <Q> is X : Integer := 0; end Q; end P;
with P; use P; -- use P.Q; package R is Y : Integer := X; end R;
3) It is as though an implicit use_clause were in effect when
resolving the selector_name of a selected name whose prefix denotes the enclosing package.
This allows:
package P is package <Q> is X : Integer := 0; end Q; end P;
with P; package R is Y : Integer := P. -- use P.Q X; end R;
4) The integrated package cannot be named outside of its immediate
scope (and this is a name resolution rule).
Note that this construct is not defined in terms of implicit use_clauses. This is just an alternative way to view it for expository purposes.
----
#2 above states
It is as though an implicit use_clause were added immediately after any use_clause that applies to the enclosing package.
Note that this informal description applies recursively to the implicit use clauses described in #1 - #3.
This allows:
use System; package P is
package <S_Ren> renames System;
end P;
with P; package Q is package <P_Ren> renames P; end Q;
with Q; use Q; package R is A : Address; end R;
----
The usual use_clause name resolution rules apply. This allows:
package P is package <Q> is X, Y : Integer := 0; end Q; X : Float; end P;
with P; package R is Z : Float renames P.X; -- legal end R:
----
If it is decided that integrated package renames are not wanted, it would be easy to remove them from the proposal.
As Erhard has pointed out, there are good reasons to consider this option. It is certainly possible to use integrated package renames to write code that is very difficult to read; this was already true of Ada83 use clauses, but integrated package renames seem to offer substantially greater potential for confusion.
We needed rules to prevent some odd corner cases:
limited with Foo; package P is package <Bar> renames Foo; end P;
We don't want this for the same reasons that we don't allow use clauses of Foo.
Another odd case:
package Outer is package Inner is package <Outer_Rename> renames Outer; end Inner; end Outer;
The declarations of Outer_Rename will always be hidden by those of Outer. So this is useless. It also has a (small) implementation burden, as a simple implementation would be at risk of going into an infinite loop: Outer_Rename would appear inside of itself. Thus the case needs to be detected and handled specially; given that it is useless, that effort might as well be spent toward making it illegal.
Finally, integrated package renames are not needed to solve any of the problems that provided the original motivation for introducing integrated packages.
The wording changes need to delete integrated package renames from this AI would be straightforward. These would include:
- in the additions after 7.1(5/2), delete "or package renaming" and
"(or of a package_renaming_declaration
of a library_unit_renaming_declaration)".
- in the additions after 8.3(25), delete "or package renaming" (twice).
- in the additions after 8.4(11), delete "or package renaming" (three
occurences), "or renamed package" (twice), and "(or, in the case of an integrated package renaming, within the renamed package)" .
- eliminate the changes to 8.5.3(3) and the addition after 8.5.3(3.1/2).
The only occurences of any form of the word "rename" (e.g., renaming, renamed) remaining in the proposed wording changes would then be the rename declaration in the example in the AARM note after 8.3(25) and the text "(or a rename thereof)" added after 8.4(11).
----
An integrated package may not be named outside of its immediate scope. To understand the motivation for this rule, consider the following example (supplied by Tuck):
Text_IO, Sequential_IO, and Direct_IO each include renames of each exception declared in the package IO_Exceptions. It is a no-brainer to replace those renames with
package <Some_Name> renames Ada.IO_Exceptions;
However, what should "Some_Name" be? Well, most natural would be:
package <IO_Exceptions> renames Ada.IO_Exceptions;
But now suppose you have:
with Ada.IO_Exceptions; with Ada.Text_IO; use Ada; use Text_Io; procedure What_Fun is begin ...; exception when IO_Exceptions.Name_Error => -- we want this to compile ...; end What_Fun;
Tuck observes that this rule should "apply to all integrated packages, not just integrated package renames, since the same problem comes up when you declare an integrated subpackage, or an integrated generic instance. You really don't want the names of the integrated packages cluttering up the namespace when you do a "use" of the enclosing package, but you do want them when trying to disambiguate where you would be using a complete expanded name."
----
We simply disallow limited views of integrated packages (10.1.1). They are like instantiations. Now that we don't allow the normal view of an integrated package to be named outside of its immediate scope, why would we want to allow naming of the limited view of an integrated package? If we were going to go down that road, it would be better to incorporate (integrate?) integration into limited views, allowing something like
package P is package <Q> is type T is ... ; end Q; end P;
limited with P; package R is type Ref is access P.T; -- illegal end R;
and we have decided against that. It seems fine to say that an integrated package, like an instance, simply does not have a limited view.
----
When resolving the name P.Q, where P denotes a package, declarations with names other than "Q" may be potentially use-visible by selection at the point of the selector_name, but they will never be use-visible by selection at that point. This is necessary because a visible integrated package declared directly in P must be potentially use-visible by selection regardless of the name of the package.
This allows:
package P is package <P1> is Aaa : Integer; package <Inner> is function F return Integer; end Inner; end P1;
package <P2> is package <Inner> is Bbb : Integer; function F return Float; end Inner; end P2; end P;
with P; package Q is X : Float := P.F; -- legal end Q;
At the point of the selector_name, 8 declarations are potentially use-visible by selection:
P.P1, P.P1.Inner, P.P1.Inner.F, P.P2, P.P2.Inner, P.P2.Inner.F, P.P1.Aaa, and P.P2.Bbb
Only 2 declarations are use-visible by selection:
P.P1.Inner.F and P.P2.Inner.F
This means that the word "such" in "more than one such declaration is potentially use-visible" is important.
In this example,
package P is package <Q> is X, Y : Integer; end Q; end P;
with P; package R is Z : Integer renames P.X; end R;
both X and Y are potentially use-visible at the point of the selector_name, but the word "such" implies that only declarations named "X" are to be considered. Thus, the example is legal.
----
The angle brackets of an integrated package are not part of the package's identifier.
In this example,
package P is package <Q> is end Q; end P;
, the name of the nested package is "Q", not "<Q>".
This is why angle brackets are not repeated at the end of a package spec, at the start or end of a package body, or for a package body stub.
--!corrigendum 8.4(3)
!ACATS test
Add ACATS tests for this new feature.
!appendix

From: Steve Baird
Sent: Thursday, January 7, 2010  6:35 PM

Here's a new version reflecting feedback from the St. Petersburg meeting and
subsequent discussions with Randy. [This is version /01 - Editor.]

This proposal includes:
    - new angle-bracket "package <P> is" syntax instead of
      "use package P".
    - effective anonymity of an integrated package outside of
      its immediate scope
    - support for easily jettisoning integrated package renames
      from the AI if it is decided that this is what we want

****************************************************************

From: Randy Brukardt
Sent: Monday, January 11, 2010  11:29 PM

You still don't have the legality rule to make an integrated renames of a
limited view of a package illegal. Since that is supposed to be equivalent to an
implicit use-clause, and we make use clauses of limited views illegal for good
reasons, we need a similar rule here or we'll reintroduce all of the problems
that we introduced 8.4(5/2) to fix. You call this an "odd corner case" in the
discussion, but there is nothing "odd" about it -- it has to be illegal and have
a rule to that effect.

I suspect that the circular rename you show also ought to be illegal, as it
could never have any useful effect (the names would always be hidden by the
direct names from the outer package).

Adding these rules will make the renames look "heavier" than it currently does,
which is an important data point in whether or not we should be allowing the
renames.

****************************************************************

From: Steve Baird
Sent: Tuesday, January 12, 2010  1:31 PM

> You still don't have the legality rule to make an integrated renames
> of a limited view of a package illegal. Since that is supposed to be
> equivalent to an implicit use-clause, and we make use clauses of
> limited views illegal for good reasons, we need a similar rule here or
> we'll reintroduce all of the problems that we introduced 8.4(5/2) to
> fix. You call this an "odd corner case" in the discussion, but there
> is nothing "odd" about it -- it has to be illegal and have a rule to that effect.
>

Good point. How about swapping the order of the "Legality Rules" and "Static
Semantics" sections of 8.5.3 (so that the definition of an integrated package
rename precedes the legality rules) and then appending the following legality
rule to 4.5.3(3.1/2):
    The package_name of an integrated package_renaming_declaration
    shall not denote a limited view of a package.
?

> I suspect that the circular rename you show also ought to be illegal,
> as it could never have any useful effect (the names would always be
> hidden by the direct names from the outer package).
>

Certainly we should disallow this if it introduces definitional or
implementation problems, but I don't see that it does. Disallowing something
because it is useless seems like overkill. Your next statement is a good
argument for allowing it:
   > Adding these rules will make the renames look "heavier" than it currently
   > does, which is an important data point in whether or not we should be
   > allowing the renames.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, January 12, 2010  4:07 PM

...
> Good point. How about swapping the order of the "Legality Rules" and
> "Static Semantics" sections of 8.5.3 (so that the definition of an
> integrated package rename precedes the legality rules) and then
> appending the following legality rule to 4.5.3(3.1/2):
>     The package_name of an integrated package_renaming_declaration
>     shall not denote a limited view of a package.
> ?

That seems OK, assuming that you meant 8.5.3(3.1/2) here and not the 4.5.3
(Binary adding operators??) that you wrote.

> > I suspect that the circular rename you show also ought to be
> > illegal, as it could never have any useful effect (the names would
> > always be hidden by the direct names from the outer package).
>
> Certainly we should disallow this if it introduces definitional or
> implementation problems, but I don't see that it does.
> Disallowing something because it is useless seems like overkill.

Maybe it is a difference in philosophy, but I think that things that are
*always* useless ought to be illegal. No one intentionally writes (well, other
than ACATS test writers!) anything useless, so if it occurs in a program, it
represents a mistake. And we want the compiler to detect mistakes.

I also worry that having circularly linked symboltable structures could be a
burden for compilers: you'd have to have some special mechanism to detect this
case otherwise you would go into an infinite regress doing lookups (as the
integrated package rename would appear inside of itself). That's probably not
that hard to do, but why spend any effort making some useless case work??

> Your next statement is a good argument for allowing it:
>    > Adding these rules will make the renames look "heavier" than it currently
>    > does, which is an important data point in whether or not we should be
>    > allowing the renames.

Well, the implementation cost of the possibility will exist whether or not a
rule exists to prevent it, so in some sense you are being misleading by hiding
it. Not a big deal, but certainly this is a weak reason for avoiding a rule.

****************************************************************

From: Steve Baird
Sent: Tuesday, January 12, 2010  4:20 PM

>>> I suspect that the circular rename you show also ought to be
>>> illegal,

I don't feel strongly about it.
Would you like wording for this rule?

****************************************************************

From: Randy Brukardt
Sent: Wednesday, January 13, 2010  9:14 PM

Yes, please. It's easier to delete a rule that we decide isn't needed
than to create one out of air (and to remember to consider it).

****************************************************************

From: Steve Baird
Sent: Thursday, January 14, 2010  1:00 PM

We've already swapped the order of the "Legality Rules" and "Static Semantics"
sections of 8.5.3 so that the definition of an integrated package rename
precedes the legality rules.

We're already appending one legality rule after 8.5.3(3.1/2) [not after
4.5.3(3.1/2) - good catch, Randy] to disallow integrated renames of limited
views of packages.

So now we also append:

    The package_name of an integrated package_renaming_declaration
    shall not denote (directly or through renaming) a package
    whose declarative region encloses the package_renaming_declaration.

****************************************************************

From: Randy Brukardt
Sent: Friday, January 15, 2010  9:15 PM

> Good point. How about swapping the order of the "Legality Rules" and
> "Static Semantics" sections of 8.5.3 (so that the definition of an
> integrated package rename precedes the legality rules) and then
> appending the following legality rule to 4.5.3(3.1/2):
>     The package_name of an integrated package_renaming_declaration
>     shall not denote a limited view of a package.
> ?

"Swapping the order" means deleting one or the other sections, because we have
to renumber the paragraphs. And that doesn't seem necessary in this case anyway:
just put the definition of *integrated package* into the legality rules (it's
not used elsewhere here, is it?) There's not a strong boundary between "Static
Semantics" and "Legality Rules" anyway (unlike "Name Resolution Rules" and
"Dynamic Semantics") So we end up with something like:

Add after 8.5.3(3.1/2):

  If a package rename's package_identification consists of an
  integrated_package_identification, then the package renaming
  is said to be *integrated* (see 8.4). The package_name of an
  integrated package_renaming_declaration shall not denote
  a limited view of a package. The package_name of an integrated
  package_renaming_declaration shall not denote (directly or
  through renaming) a package whose declarative region encloses
  the package_renaming_declaration.

I'd suggest the same in 7.1. Note that changing the paragraph numbers of
legality rules is nasty, because it not unusual for compiler error messages to
refer to them; it's much less likely that the paragraph numbers of other things
are referred to. So I'd prefer not to change the paragraph numbers of unmodified
legality rules.

I've modified the draft AI as suggested here. I'm sure someone will have an even
better idea...

****************************************************************

From: Tucker Taft
Sent: Monday, April 26, 2010  8:55 PM

[This thread was split from the thread in AI05-0213-1 - Editor]

> P.S. Are we really giving up on the serious problem of containers of
> private types?? This doesn't work for them.

Time to resurrect the "end private"?

I still think that is the easiest way
to get containers of private types.
Many of the objections seemed to be
more aesthetic than technical.

****************************************************************

From: Randy Brukardt
Sent: Monday, April 26, 2010  9:32 PM

Brad and I would tell you that "private instances" or something like that would
work better. Ed would tell you that piecemeal freezing would be best. Steve
(once at least) would tell you that a special kind of generic formal is the
ticket. ;-)

The problem wasn't a lack of ideas, the problem was that none were natural or
particularly usable. None gathered enough fans to really go forward with them.
With the failure of integrated packages, I think we're pretty much out of
reasonable possibilities.

Luckily, we voted all of those ideas "No Action" at the last meeting. :-)

> > P.S. Are we really giving up on the serious problem of containers of
> > private types?? This doesn't work for them.

I just spent a few moments thinking about the problem based on somehow allowing
allocators and/or components of this new kind of formal in generic bodies. But
even if we could figure out how to define that, it would only solve the problem
for unbounded and indefinite container forms. The bounded forms require
components in the specification -- I don't see any sane way to do that for an
unfrozen type (short of piecemeal freezing, which gives me the willies).

So I'm afraid the programmer is going to have to store pointers (and do their
own memory management) or somehow make the container operations visible
piecemeal, hiding the instance in the private part.

****************************************************************

From: Tucker Taft
Sent: Monday, April 26, 2010  10:32 PM

> The problem wasn't a lack of ideas, the problem was that none were
> natural or particularly usable. None gathered enough fans to really go
> forward with them. With the failure of integrated packages, I think
> we're pretty much out of reasonable possibilities.

I think lumping all of these into one heap is not necessarily helpful.  The "end
private" idea was relatively simple and very stable.  "Popularity" seems like
the wrong way to proceed at this point.  We have worked out many of the details
on almost all of these ideas, and it seems that "end private" was the only one
that remained relatively straightforward to implement while still solving a
significant share of the problems.

If you look at AI05-0074-2 which talks about "end private"
you will see that most of the !appendix is discussing the complexities of
partial instantiations.  There are a few (Baird-ish) questions about the details
of "end private" but they were all relatively easily answered by focusing on
avoiding privacy breaking.

>
> Luckily, we voted all of those ideas "No Action" at the last meeting.
  :-)

So much for "no action."

>>> P.S. Are we really giving up on the serious problem of containers of
>>> private types?? This doesn't work for them.

"end private" solves these pretty easily.
It also supports generic signature packages trivially.

****************************************************************

From: Randy Brukardt
Sent: Monday, April 26, 2010  11:07 PM

> I think lumping all of these into one heap is not necessarily helpful.

Why not? They all solve the same problem, and most weren't very complex in
conception.

> The "end private" idea was relatively simple and very stable.

Give me a break. I don't think anyone ever tried to "vet" the wording you
proposed (indeed, I didn't even remember that you had produced wording).

> "Popularity" seems like the wrong way to proceed at this point.  We
> have worked out many of the details on almost all of these ideas, and
> it seems that "end private" was the only one that remained relatively
> straightforward to implement while still solving a significant share
> of the problems.

This sounds like "my idea is better than everyone else's idea, so let's ignore
everything else and use it".

I don't recall anyone believing this would be "straightforward" to implement.
Adding a new kind of visibility is always hard; I know our compiler has an
invariant that it can stop looking for visible symbols when it reaches the
private marker -- I have no idea how many places would need to change that code.

> If you look at AI05-0074-2 which talks about "end private"
> you will see that most of the !appendix is discussing the complexities
> of partial instantiations.  There are a few
> (Baird-ish) questions about the details of "end private" but they were
> all relatively easily answered by focusing on avoiding privacy
> breaking.

I showed an example that could not be solved by "public" given the rules that
you defined. (The "end private" syntax was roundly hated as I recall.) Look at
my mail of August 13, 2008 (3:17 PM)

I suspect that there are a lot of such examples; the inability to complete
anything later always causes problems with primitiveness and the like.

(I noticed this message because it isn't word-wrapped, but it seems critical to
me.)

There also was a lot of discomfort with the idea of having even more visibility
sections. You can't make that go away by using saying that "popularity" is the
wrong way to proceed. I don't think that jamming in some feature that many
people dislike is going to make selling this Amendment very easy.

****************************************************************

From: Steve Baird
Sent: Monday, April 26, 2010  11:17 PM

> Steve (once at least) would tell you that a special kind of generic
> formal is the ticket. ;-)

Just to recap, the core of the idea I was advocating was to have some mechanism
(perhaps a new kind of formal type, but that's a detail) of getting
    1) fine-grained freezing associated with a package instance spec
    2) dynamically deferred elaboration of a package instance body
    3) statically deferred freezing associated with a package
       instance body (i.e., deferred to the point at which the
       body is elaborated).
so that a not-yet-completed private type could be passed as an actual parameter.

This doesn't help if the generic package spec declares a deferred constant of
the the private type (or of an enclosing type), and folks were unwilling to
consider changing the predefined container specs to eliminate deferred constants
(perhaps by replacing them with parameterless functions).

****************************************************************

From: Randy Brukardt
Sent: Monday, April 26, 2010  11:36 PM

I don't recall this. I thought the problem was less with deferred constants than
with components of the "magic" formal type in the spec. As I noted in a previous
message, that's not required for the unbounded and indefinite forms, but it is
required for the bounded forms. Changing the spec doesn't work, because the
whole point of the bounded forms is elimination of dynamic allocation, and
pushing the components into the body somehow requires dynamic allocation of the
components (since they can't be part of the private type).

(Off hand, I can't think of any deferred constants in the containers specs that
directly depend on the element type. So this description seems wrong to me,
although I can easily believe that there is a problem somewhere -- there always
is!)

****************************************************************

From: Tucker Taft
Sent: Monday, April 26, 2010  11:38 PM

> ...
> This sounds like "my idea is better than everyone else's idea, so
> let's ignore everything else and use it".

I agree that is how it sounds.  But it seems the other proposals mostly
collapsed under the weight of their complexities, while this one seemed to
collapse on the weight of aesthetic concerns.

I had forgotten about the idea of using "public"
instead of "end private" -- that does seem somewhat more "aesthetic."

> I don't recall anyone believing this would be "straightforward" to
> implement. Adding a new kind of visibility is always hard; I know our
> compiler has an invariant that it can stop looking for visible symbols
> when it reaches the private marker -- I have no idea how many places
> would need to change that code.

Implementation difficulties are always hard to gauge, but this one seemed much
more manageable and less likely to produce surprises than the others.

>> If you look at AI05-0074-2 which talks about "end private"
>> you will see that most of the !appendix is discussing the
>> complexities of partial instantiations.  There are a few
>> (Baird-ish) questions about the details of "end private" but they
>> were all relatively easily answered by focusing on avoiding privacy
>> breaking.
>
> I showed an example that could not be solved by "public" given the
> rules that you defined. (The "end private" syntax was roundly hated as
> I recall.) Look at my mail of August 13, 2008 (3:17 PM)

I believe that is solved by allowing an "incomplete" type as a subprogram
parameter or result type.

> I suspect that there are a lot of such examples; the inability to
> complete anything later always causes problems with primitiveness and the like.
>
> (I noticed this message because it isn't word-wrapped, but it seems
> critical to me.)

It certainly didn't solve all of the possible problems, but it solved the basic
ones, including generic signatures and containers of private types, without
delicate surgery on freezing rules or elaboration order.

> There also was a lot of discomfort with the idea of having even more
> visibility sections. You can't make that go away by using saying that
> "popularity" is the wrong way to proceed. I don't think that jamming
> in some feature that many people dislike is going to make selling this
> Amendment very easy.

No argument there.  But not solving this problem would be sad, and at some point
one considers the lesser of various evils.

****************************************************************

From: Steve Baird
Sent: Monday, April 26, 2010  11:59 PM

>> This doesn't help if the generic package spec declares a deferred
>> constant of the the private type (or of an enclosing type), and folks
>> were unwilling to consider changing the predefined container specs to
>> eliminate deferred constants (perhaps by replacing them with
>> parameterless functions).
>
> I don't recall this. I thought the problem was less with deferred
> constants than with components of the "magic" formal type in the spec.

I'd have to go back and review the mail, but offhand I don't see any problem
with components. The idea here is that if no constructs in the instance spec
cause freezing of the actual type (and, of course, using a type as a component
type does not require freezing it) then it is ok to pass in a not-yet-completed
private type as an actual. This could be made part of the contract (perhaps by
inventing a new kind of formal type); this would allow checking for premature
freezing at the point of the generic rather than at the point of the instance.
This is not, however, a necessary and fundamental part of the proposal.

> (Off hand, I can't think of any deferred constants in the containers
> specs that directly depend on the element type. So this description
> seems wrong to me, although I can easily believe that there is a
> problem somewhere -- there always is!)

Wouldn't a bounded map type typically have components of the formal types (in
the private part)?

So a deferred Empty_Map constant causes problems.

****************************************************************

From: Tucker Taft
Sent: Tuesday, April 27, 2010  12:08 AM

This feels awfully fragile to me, where
it sort of works if everything is just
right, but if you make one small change
it all falls apart.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April 27, 2010  12:21 AM

> Wouldn't a bounded map type typically have components of the formal
> types (in the private part)?
>
> So a deferred Empty_Map constant causes problems.

Gee, and you just said that components of formal types didn't cause problems.
:-)

I see your point, but I surely don't recall anyone refusing to change Empty_Map
to a function (could you even tell the difference? I suppose with overloading,
but that seems like an advantage rather than a problem). I didn't think we ever
got anywhere near that point.

I am no fan of fine-grained freezing for generics, because that requires lots of
extra work for handling instances -- and lots of messing around with stuff in
private parts that we share. But I was willing to consider a special kind of
formal type that had fine-grained freezing associated with it (and I suppose
types that depend on it), with everything else being frozen immediately. That
would also have the elaboration changes associated with it. The net effect is
that it wouldn't change easily: a given generic would be one way or the other.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April 27, 2010  12:25 AM

> This feels awfully fragile to me, where it sort of works if everything
> is just right, but if you make one small change it all falls apart.

Why? If it depends on a special kind of generic formal type (which is the only
way I would accept it), the only "small change" that you could make that would
change anything is whether or not to use that formal type. The semantics would
be pretty static otherwise. I do worry about the effect of exactly where the
elaboration of the body happens, but that seems like a necessary evil. (I'd
suggest making it go at the end of the unit in order to avoid it moving easily.)

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April 27, 2010  12:34 AM

> Just to recap, the core of the idea I was advocating ...

If we're going to rehash all of our favorite ideas, I'd like to point out that
my "limited instance" idea works much better now that we allow much more use of
incomplete types in parameters and results. The original complaint about it was
that you had to use the type in access types everywhere; but now you can use it
directly everywhere except to declare direct components.

It also has the advantage that it allows primitive operations that include
containers parameters (as in that e-mail I pointed Tucker at). I suspect that
Steve's suggestion would handle that as well. But neither the current nested
packages nor Tucker's "end private" (or "public" as I called it) can handle that
(which seems to be the point, IMHO). (That means that "integrated packages"
can't do it, either.)

****************************************************************

From: Steve Baird
Sent: Tuesday, April 27, 2010  12:59 AM

> This feels awfully fragile to me, where it sort of works if everything
> is just right, but if you make one small change it all falls apart.

That's the main argument for making this part of the contract so that constructs
which would cause freezing problems at the point of an instantiation can be
rejected at the point of the generic instead.

****************************************************************

From: Tucker Taft
Sent: Tuesday, April 27, 2010  10:09 AM

By "fragile" I meant that small changes to the generic spec might affect, in a
non-intuitive way, whether the special kind of generic formal can be used
successfully.

This may be a case where the complexity of description masks an underlying
simpler idea.  Is the intent to make this formal have the same characteristics
as a private type before its completion?  So that unlike an incomplete type, it
can be used as a component type (that seems like about the only distinction
left).  It would mean that the composite type with this type as a component
cannot be frozen in the generic spec itself. And the instance body needs to be
postponed until after the completion of the actual private type.

If my mental model is correct, this seems to bring back several of the
body-elaboration-time issues we struggled with before, and adds some complexity
to freezing rules enforcement.  And both body elaboration and freezing rules are
somewhat "fragile" from the user point of view, I would say.

****************************************************************

From: Steve Baird
Sent: Tuesday, April 27, 2010  10:43 AM

I think your mental model is correct - as you said, the big difference between
this and a formal incomplete is that one of these guys can be used as a
component type (in the generic spec) and can be used arbitrarily (including, for
example, object declarations) in the generic body.

The freezing and body-elaboration issues that you allude to are what I was
talking about when I mentioned (dynamically) deferred instance body elaboration
and (statically) deferred freezing for such an instance body.

> And both body
> elaboration and freezing rules are somewhat "fragile" from the user
> point of view, I would say.

This still seems to me like a much less fundamental change for users than "end
private".

****************************************************************

From: Edmond Schonberg
Sent: Tuesday, April 27, 2010  11:07 AM

I agree that this seems to have fewer complex consequences than "end private" .
But if this does not solve the containers of private types issue it is also much
less interesting, and not clearly worth the trouble.  It seems to me that at
least for unbounded containers a minimal rewriting should make it possible to
use this to implement containers of private types.

****************************************************************

From: Tucker Taft
Sent: Tuesday, April 27, 2010  11:07 AM

> This still seems to me like a much less fundamental change for users
> than "end private".

Interesting.  I guess our mileage will vary.

Folks seem pretty able to handle nested packages or protected types with nested
private declarations. From a visibility point of view, that is fairly similar.
Certainly programmers familiar with Java or C++ are used to private and public
things being somewhat interspersed.

I think Bob Duff long ago identified the awkward position Ada is in, where they
physically separate out public and private within a package spec, but
nevertheless have an executable semantics for the spec.  If the specs were only
declarations with no run-time elaboration, the separation would be trivial.  But
since the declarations involve run-time elaboration, the separation is awkward
and creates all kinds of complexity.

Allowing some "public" declarations to follow the completion of a private type
eliminates some of that awkwardness.

****************************************************************

From: Tucker Taft
Sent: Tuesday, April 27, 2010  11:34 AM

> I agree that this seems to have fewer complex consequences than "end
> private" ...

I am surprised by the sense that there are complex consequences.  Proposals like
Steve's involve subtly new dynamic semantics of existing constructs.  The
dynamic semantics of "end private" are essentially identical to what you would
have if you ignored the "end private."  That seems like a significant advantage.

****************************************************************

From: Edmond Schonberg
Sent: Tuesday, April 27, 2010  12:20 PM

It's the static semantics I'm concerned about, but given that the real prize is
containers of private types, I'm willing to review the "end private" proposal.
We are speaking about a single private part, not an unlimited number as in those
other languages, right?

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April 27, 2010  12:43 PM

> Folks seem pretty able to handle nested packages or protected types
> with nested private declarations.
>  From a visibility point of view, that is fairly similar.
> Certainly programmers familiar with Java or C++ are used to private
> and public things being somewhat interspersed.

Maybe, but I've never used a nested package with a private part outside of the
examples we've constructed during this debate. It's not something I would
intentionally do. The whole idea of mixing private and public declarations makes
me ill. (I tolerate protected types because I have no choice and rationalize it
that they belong to the type. But they're almost always in a body (I've only
used one visible protected type ever, the one described in the queue e-mail), so
it's not relevant.

I think breaking the current invariant that you can stop reading at "private"
(which is surely one I depend on) is really a very important change to many
people (including me). It makes this feel like a much larger change than it
actually is. For those of us familiar with reading Ada, it could take a very
long time to get used to this change -- it would be easier to banish the private
part altogether.

> I think Bob Duff long ago identified the awkward position Ada is in,
> where they physically separate out public and private within a package
> spec, but nevertheless have an executable semantics for the spec.  If
> the specs were only declarations with no run-time elaboration, the
> separation would be trivial.  But since the declarations involve
> run-time elaboration, the separation is awkward and creates all kinds
> of complexity.
>
> Allowing some "public" declarations to follow the completion of a
> private type eliminates some of that awkwardness.

One other issue is that you continue to insist on the worst possible syntax.
"end private" is just wrong: either you leave out the semicolon and break the
invariant that a semicolon follows all "end"s, or you put the semicolon in and
have what appears to be a terminator for the construct in the middle of a
construct. There currently is no circumstance where "end" occurs in the middle
of a construct in Ada, and this is one thing we're not going to change.

It would be much better to introduce this section with "public" (or something
similar). Of course, that requires a new keyword. But the alternative of using a
terrible syntax from a consistency basis just to avoid a new keyword is just
unacceptable.

****************************************************************

From: Randy Brukardt
Sent: Tuesday, April 27, 2010  1:07 PM

> It's the static semantics I'm concerned about, but given that the real
> prize is containers of private types, I'm willing to review the "end
> private" proposal.  We are speaking about a single private part, not
> an unlimited number as in those other languages, right?

Tucker had made a fairly complete write-up in AI05-0074-2. It's worth reading
again if you want to re-discuss this. It's restricted to one such part, and as I
recall there are good reasons for such a restriction.

But:

(1) The syntax is unacceptable. There has to be a new keyword or some other
    syntax. "end private" will never get my vote, no matter what the semantics:
    "end" must never appear in the middle of a construct.

(2) Once you have this, people are going to wonder why only one is allowed. Even
    if there are semantic problems with more than one, it is going to feel
    strange to allow exactly 2 of something. It brings up the question of which
    visible part a declaration ought to go -- many of them can only go in one or
    the other. (For instance, primitive operations of a private type have to go
    in the first visible part if something freezing (like an instance) is used
    in the second part, as they can't follow any freezing.)

(3) The loss of the invariant that you stop reading at the keyword "private" is
    a BIG DEAL. Both from readability and implementation standpoints.

----

I'd rather use an incomplete instantiation in the visible part (like deferred
constants and the like). But I realize that people didn't like having an extra
instance declaration, and I never found a syntax that really reflected how it
works (it's the inverse of the normal: clients have more visiblity than the
package itself does).

(BTW, the concern about extra declarations is bogus. All of the solutions
suggested need some sort of extra declarations. The extra visible part needs a
passel of incomplete types in the first visible part to have any hope of being
usuable. Steve's solutions need modification of the contracts of the generics
involved, and have by far the worst changes semantically.)

Brad has shown that the incomplete instantiation can solve problems that the
other methods can't (although he went over the top with some of his ideas).

----

Point is, continuing to talk about this (in the absence of truly new ideas) is a
waste of time. Hardly anyone is going to change their position, even though it
is clearly wrong. ;-)

Keep in mind that the only important thing that you cannot do with nested
packages is declare primitive operations that use the container type. (We don't
have decent visibility control, but that problem is so pervasive in Ada that
worrying about it in this case is silly.) So the only thing that really matters
is comparing what you cannot already do with restructuring: in this case,
"public" doesn't fare very well. You still need a lot of restructuring and extra
declarations. Incomplete instances solve the same problem without blowing up the
reading model of Ada code or any dynamic semantics changes or any need to
restructure. They're the least invasive of all of the solutions.

****************************************************************

From: Steve Baird
Sent: Wednesday, April 28, 2010  11:42 AM

> I agree that this seems to have fewer complex consequences than "end
> private" .  But if this does not solve the containers of private types
> issue it is also much less interesting, and not clearly worth the
> trouble.  It seems to me that at least for unbounded containers a
> minimal rewriting should make it possible to use this to implement
> containers of private types.

I would hope that this solution would work for all predefined container generics
(after minimal rewriting to eliminate, e.g., any problematic deferred
constants). That is the whole point of this proposal and if it doesn't
accomplish that, then I certainly agree that it is a bad idea.

****************************************************************

From: Edmond Schonberg
Sent: Wednesday, April 28, 2010  12:01 PM

I looked at some of the current container packages and the rewriting does not
look too onerous, but Randy claims that it will be much harder for bounded
containers.

****************************************************************

From: Randy Brukardt
Sent: Wednesday, April 28, 2010  12:16 PM

Steve seemed to think that what I was concerned about was not a problem.
Assuming he's right, there wouldn't be any especial difficulty.

I'm at least as concerned about restructuring of clients. My thought experiments
have suggested that all of these schemes (including this one) will require
restructuring of clients to some extent. Effectively, using nested packages is
not significantly worse than what any of these proposals would require. Meaning
it probably isn't worth the effort to dream up some complex feature to handle
it. (That insight was the driving idea behind "integrated packages", which was
to get rid of the naming overhead of nested packages, and *only* the naming
overhead.)

I'll try to write up some examples of what I mean this evening (I need to do
that to verify the thought experiment).

P.S. I'm still convinced that incomplete instances is the best idea, because it
gives compile-time indication of where restructuring is needed. The
Baird-Schonberg plan would only give run-time indication (elaboration failures)
-- although I suppose GNAT would find some way to do these at compile-time.

****************************************************************

From: Bob Duff
Sent: Wednesday, April 28, 2010  7:06 PM

> > ...
> > This sounds like "my idea is better than everyone else's idea, so
> > let's ignore everything else and use it".

I don't see it that way.  To me, it sounds like "let's get together and solve a
problem we all agree is important to solve."

> No argument there.  But not solving this problem would be sad, and at
> some point one considers the lesser of various evils.

I agree.

ARG has a tendency to let "best" be the enemy of "good enough", which means that
serious language deficiencies don't get solved due to aesthetic or other
more-minor concerns.

(in another message):

> If my mental model is correct, this seems to bring back several of the
> body-elaboration-time issues we struggled with before, and adds some
> complexity to freezing rules enforcement.  And both body elaboration
> and freezing rules are somewhat "fragile" from the user point of view,
> I would say.

I agree about freezing, but not about body elab.
Generic instance body elaboration usually doesn't do anything interesting, and
even when it does, one normally doesn't care too much when it happens.

****************************************************************

From: Randy Brukardt
Sent: Wednesday, April 28, 2010  7:32 PM

> I don't see it that way.  To me, it sounds like "let's get together
> and solve a problem we all agree is important to solve."

That's not what Tucker was saying. He was saying let's go back and use an idea
that was previously rejected because it breaks what many consider a primary
invariant of Ada source code: reading can stop at "private".

I'd be happy to solve the problem, except: (a) none of the proposed solutions
are significantly better than doing nothing (see the message I hope to write
tonight to see that); (b) no one is proposing a new idea; (c) we talked the old
ideas to death and no one is changing their opinions.

The latter factor wouldn't matter if we didn't have a hundred other things to
do, but we do and a lot of them are more important than this problem (given (a)
and (b)).

> > No argument there.  But not solving this problem would be sad, and
> > at some point one considers the lesser of various evils.
>
> I agree.
>
> ARG has a tendency to let "best" be the enemy of "good enough", which
> means that serious language deficiencies don't get solved due to
> aesthetic or other more-minor concerns.

The problem is that we don't even agree what the real problem is. Most of what
has been claimed to be impossible is in fact legal now using nested packages and
the enhancements to incomplete types that we've already agreed to. There mainly
remains a naming and organization issue. It's not at all clear that a massively
expensive feature (and I think it is safe to say all of the proposals are in
that category) is worth the small gain.

> (in another message):
>
> > If my mental model is correct, this seems to bring back several of
> > the body-elaboration-time issues we struggled with before, and adds
> > some complexity to freezing rules enforcement.  And both body
> > elaboration and freezing rules are somewhat "fragile" from the user
> > point of view, I would say.
>
> I agree about freezing, but not about body elab.
> Generic instance body elaboration usually doesn't do anything
> interesting, and even when it does, one normally doesn't care too much
> when it happens.

You've said this before, and you're wrong:
(1) A package instantiation is the only way to get a procedure call into the
    elaboration of a specification, and it is not unusual to see it used that
    way. In that use, the *only* issue is where the body elaborates; moving it
    somewhere else would be fatal (since it wouldn't have needed to be in the
    specification in the first place if the elaboration didn't have some
    significant side-effect).

(2) Where body elab happens matters because if it is too late, you would get
    random Program_Errors from access-before-elaboration errors. (Surely we
    aren't going to abandon those checks.)

(1) can be mitigated by Steve's plan to make this semantics somehow part of the
    contract. That would prevent existing elaborations from running into the
    problem. It could be mitigated further by restricting what could be done in
    the body of such a generic (although that probably would be overkill).

(2) is much harder to deal with. It could be done by defining the elaboration to
    occur at a freezing point, except that itself doesn't work, since
    elaboration might be inside of a conditional expression at that point.
    Meaning that the elaboration code would also be inside of a conditional
    expression and thus it might not actually get run. So it has to happen at
    some point earlier than the freezing point, which is trouble. (I tried a
    scheme like that for generating thunks in Janus/Ada, and it didn't work for
    this sort of reason. In that case, I was able to delay all of the code to
    the end of the declarative part or specification since none of it needs to
    be run directly; but that won't work for elaboration code that has to
    execute).

P.S. "you're wrong" is probably a little strong for this last topic; I don't
want to waste time thinking up appropriate weasel-words tonight; so I apologize
for any offense.

****************************************************************

From: Tucker Taft
Sent: Wednesday, April 28, 2010  8:00 PM

>> I don't see it that way.  To me, it sounds like "let's get together
>> and solve a problem we all agree is important to solve."
>
> That's not what Tucker was saying. He was saying let's go back and use
> an idea that was previously rejected because it breaks what many
> consider a primary invariant of Ada source code: reading can stop at "private".

It was rejected, but I believe it was largely on what I would consider
"philosophical" grounds.  Now that we have more fully investigated many other
approaches, I am bringing this one up again because, at least in my view, it is
significantly simpler to explain, less likely to result in unexpected "gotchas,"
invents no new kinds of formal types or generic instances, and involves no
changes to the dynamic semantics of any existing constructs.

> I'd be happy to solve the problem, except: (a) none of the proposed
> solutions are significantly better than doing nothing (see the message
> I hope to write tonight to see that); (b) no one is proposing a new
> idea; (c) we talked the old ideas to death and no one is changing their opinions.

I look forward to your promised note this evening and how it applies to
end-private/public...

****************************************************************

From: Edmond Schonberg
Sent: Wednesday, April 28, 2010  10:19 PM

> It was rejected, but I believe it was largely on what I would consider
> "philosophical" grounds.  Now that we have more fully investigated
> many other approaches, I am bringing this one up again because, at
> least in my view, it is significantly simpler to explain, less likely
> to result in unexpected "gotchas," invents no new kinds of formal
> types or generic instances, and involves no changes to the dynamic
> semantics of any existing constructs.

I'm beginning to be sympathetic to this solution. I know that Randy finds the
trailing part (does it have a name?) unacceptable, but it is the simplest way to
fix the container-of-private-type problem, it does not present any complex
implementation issues, and it's easy to describe. Casual readers from other
languages might wonder why only one private part is allowed,  but this approach
has fewer problems than the limited instantiation or the formal incomplete type
proposal, both of which have complex (to my mind) freezing implications.

For what it's worth... I know that Randy feels that all approaches have been
discussed to death (or at least to deadlock).  I just want to indicate that
there might be shifts in the vote :-)!

****************************************************************

From: Randy Brukardt
Sent: Thursday, April 29, 2010  1:20 AM

> I'll try to write up some examples of what I mean this
> evening (I need to do that to verify the thought experiment).

OK, here goes. I'll use the example from an old mail of mine, updated to use
various approved new features, and to show how the three options work. (I'm
going to try to play fair, but it always possible that I'll forget some
important possibility.)

Our example is based on a Claw abstraction:

    package Menus is
        type Menu_Item_Type is tagged private;
        type Menu_Item_List_Type is array (Positive range <>) of Menu_Item;
        Empty_Menu_List : constant Menu_Item_List_Type;

        procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
                                New_Submenu : in Menu_Item_List_Type);
            -- Make Menu_Item into a submenu with the given items.

        function Get_Submenu (Menu_Item : in Menu_Item_Type) return
               Menu_Item_List_Type;

        -- Many other operations here.
    private
        type Menu_Item_Type is ...;
        Empty_Menu_List : constant Menu_Item_List_Type := (1..0 => <>);
    end Menus;

Make_Submenu and Get_Submenu are primitive, dispatching operations of
Menu_Item_Type. We're going to assume for the purposes of this discussion
that they must be such for the design to work (so restructuring that does
not keep them as primitive operations is out of bounds).

Now, imagine that you want to have a version of Make_Submenu for vectors of
menu items. (This seems a very reasonable thing to want to do, especially
given the difficulty of handling arrays with unknown bounds.)

The basic conversion would look like:

    with Ada.Containers.Vectors;
    package Menus is
        type Menu_Item_Type is tagged private;
        package Menu_Vector is new Ada.Containers.Vectors (Menu_Item_Type);
            -- Illegal!
        Empty_Menu_List : constant Menu_Vector.Vector;

        procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
                                New_Submenu : in Menu_Vector.Vector'Class);
            -- Make Menu_Item into a submenu with the given items.

        function Get_Submenu (Menu_Item : in Menu_Item_Type) return
               Menu_Vector.Vector'Class;

        -- Many other operations here.
    private
        type Menu_Item_Type is ...;
        Empty_Menu_List : constant Menu_Vector.Vector := Menu_Vector.Empty_Vector;
    end Menus;

but as noted, this is illegal because Menu_Item_Type is not completed at the
point of the instantiation (and thus freezing it is illegal). [I added the
'Class because a Vector is a tagged type, and non-primitive parameters of a
tagged specific type are dubious: they would not work with extensions and
that defeats the purpose of taggedness.]

First, let's see what we can do with the existing features of Ada (including
the new incomplete type rules):

    with Ada.Containers.Vectors;
    package Menus is
        type Menu_Item_List_Type is tagged;
        package Inner is
           type Menu_Item_Type is tagged private;
           procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
                                   New_Submenu : in Menu_Item_List_Type'Class);
               -- Make Menu_Item into a submenu with the given items.

           function Get_Submenu (Menu_Item : in Menu_Item_Type) return
                  Menu_Item_List_Type'Class; -- Newly allowed by AI-151.

           -- Many other operations here.
        private
           type Menu_Item_Type is ...;
        end Inner;
        -- (1)

        package Menu_Vector is new Ada.Containers.Vectors
(Inner.Menu_Item_Type);
        type Menu_Item_List_Type is new Menu_Vector.Vector with null record;
        -- (2)
        Empty_Menu_List : constant Menu_Item_List_Type := Menu_Item_List_Type(Menu_Vector.Empty_Vector);
    end Menus;

This required a lot of restructuring, and it required deriving the vector
type.

This also has the disadvantage of requiring extra naming (for package
Inner). We could help that by add derivations and renamings at (1). We need
both because derivation only inherits primitive operations, leaving
class-wide operations, exceptions, objects, and the like behind. We have the
same problem at (2): we also need to derive cursors and rename objects if we
want to use them here.

This is clearly a maintenance hazard. But it should be clear that we already
are 90% of the way there. It isn't that this cannot be done, it is that it
requires a lot of convolution to get there. So any solution needs to be able
to do this with a lot less convolution; preferably none.

Integrated packages (AI05-0135-2) was supposed to address the naming
problem, which would reduce the need for derivations. But we'd still need
the derivation of Menu_Item_List_Type (to complete the incomplete type), and
it turns out that we'd move the maintenance hazard to the clients. Ugh.

OK, let's look at the other three solutions. First up, Tucker's baby, the
"public" part:

    with Ada.Containers.Vectors;
    package Menus is
        type Menu_Item_Type is tagged private;
        type Menu_Item_List_Type is tagged; -- (1)
        procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
                                New_Submenu : in Menu_Item_List_Type'Class);
           -- Make Menu_Item into a submenu with the given items.

        function Get_Submenu (Menu_Item : in Menu_Item_Type) return
           Menu_Item_List_Type'Class; -- Newly allowed by AI-151.

        -- Many other operations here.
    private
       type Menu_Item_Type is ...;
    public
        package Menu_Vector is new Ada.Containers.Vectors (Inner.Menu_Item_Type);
        type Menu_Item_List_Type is new Menu_Vector.Vector with null record;
        -- (2)
        Empty_Menu_List : constant Menu_Item_List_Type :=
		Menu_Item_List_Type(Menu_Vector.Empty_Vector);
    end Menus;

Note that we still need the incomplete type at (1), and that means that we
have to do a derivation at (2) to complete it. And that then brings up the
question of whether to derive Cursors and generics and the like.

As with the nested package case, we need to rearrange everything, which
isn't that pleasant.

The *only* thing this has solved is the naming (no need to mention package
Inner). It's essentially the same as integrated packages in that way, but
more limited (so the maintenance problems don't exist).

I realize that there are cases where something declared in the instance
won't be needed in primitive operations (signature packages is the obvious
case). OTOH, it's also been noted that you can imagine cases where this is
not enough and you'll still need a nested package.

---

Moving on to my baby (now taken over by Brad), the limited instance. Recall
that within the package spec it provides effectively a limited view of the
instance (just incomplete types - yes I know that a regular limited view
doesn't see anything in an instance, so it's not quite the same, but the
effect is the same as limited with of a visible package), but the clients
see the regular instance that is hidden in the private part:

    with Ada.Containers.Vectors;
    package Menus is
        type Menu_Item_Type is tagged private;
        package Menu_Vector is new limited Ada.Containers.Vectors (Menu_Item_Type);
        Empty_Menu_List : constant Menu_Vector.Vector; -- (1)

        procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
                                New_Submenu : in Menu_Vector.Vector'Class);
            -- Make Menu_Item into a submenu with the given items.

        function Get_Submenu (Menu_Item : in Menu_Item_Type) return
               Menu_Vector.Vector'Class;

        -- Many other operations here.
    private
        type Menu_Item_Type is ...;
        package Menu_Vector is new Ada.Containers.Vectors (Menu_Item_Type); -- (2)
        Empty_Menu_List : constant Menu_Vector.Vector := Menu_Vector.Empty_Vector;
    end Menus;

The only changes here are to add "limited" and put the completing instance
at (2). This is much less change from the "ideal" than "public" would
require (no extra derivations, for instance).

There is also no dynamic change (the limited instance isn't elaborated), and
almost no freezing changes. The big issue is dealing with the bifucated
visibility of the limited instance. It makes logical sense, but it might be
hard to implement.

There is also another minor issue: the deferred constant at (1) is illegal.
There are two ways to deal with this:

(1) Allow deferred constants of incomplete types so long as the incomplete
type is completed in the same specification. Much like subprograms, there
really is no reason to require this type to be not incomplete at the point
of the deferred constant declaration (which is just a "link" to the real
constant in the private part) -- so long as the completion is legal, there
is no real problem with the deferred constant.

(2) Leave the language alone and just change it to a function call. In this
case, we could use a renames-as-body in the private part to provide the
implementation, so we wouldn't even need to have a real body.

To summarize, this requires much less restructuring, no changes to the
generic unit, and doesn't require derivations. The downsides are a new
feature that is a bit difficult to grok, and the inability to derive from
the types of Menu_Vector in this package. Unlike the first two schemes, we
don't have a real visible type to derive from within the package Menus: you
can't derive from an incomplete type. And we can't rename anything from the
limited instance, either (that's not even exported).

---

Moving on to Steve's idea of some sort of magic contract that would allow
fine-grained freezing and deferred body elaboration.

Let's assume that Vectors has been treated with this magic. What would this
look like?

    with Ada.Containers.Vectors;
    package Menus is
        type Menu_Item_Type is tagged private;
        package Menu_Vector is new Ada.Containers.Vectors (Menu_Item_Type);
        Empty_Menu_List : constant Menu_Vector.Vector;

        procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
                                New_Submenu : in Menu_Vector.Vector'Class);
            -- Make Menu_Item into a submenu with the given items.

        function Get_Submenu (Menu_Item : in Menu_Item_Type) return
               Menu_Vector.Vector'Class;

        -- Many other operations here.
    private
        type Menu_Item_Type is ...;
        -- (1)
        Empty_Menu_List : constant Menu_Vector.Vector :=
	    Menu_Vector.Empty_Vector; -- (2)
        -- (3)
    end Menus;

This has allowed us to write almost what we wanted directly. But it is not
quite as rosey as it appears.

First of all, where the instance body elaborates is important: if it
elaborates at (1), then the call at (2) is fine, but if it elaborates at
(3), the call at (2) raises Program_Error and we're hosed. It might be
possible to adjust the rules to prevent this particular problem: we could
require it to elaborate at some specific freezing point. But that seems like
a problem, because the fine-grained freezing (everything had better be
frozen before we elaborate the body!), and because of the possibility that
that freezing point is in a conditional expression or the like (in which
case, the body might not elaborate at all if the condition is wrong - ugh).

Second, like the limited instance, we probably can't visibly derive from the
types exported from Menu_Vector. That would fail if they depend on the
private type in some way. We'd need to be careful with the rules to avoid
making the results depend on the contents of the private part, too. Mr.
Private would not be happy if Cursors (a private type) could be derived
because they don't actually use a component of the private type.

Unlike the limited instance, we could do some renaming, but of course
renaming is even more of a maintenance hazard than derivation (at least
derivation automatically picks up primitive operations). So I'm not
convinced that is much of an advantage.

Of course, this scheme also requires modifications to generics to allow them
to use it (not too big of a deal), some modifications to the dynamic
semantics of instances (I don't think that is a huge deal, either, if we can
work out a sane rule of where to do it), but a boatload of messing with
freezing rules.

----

Biased summary:

So we come to the end of our tour. What we've learned is even if we just
tried to "just make it work like people expect, dammit", we really would
still have a somewhat limiting solution. We can't break the privacy of the
generic unit (I surely hope no one is thinking of that!), and that means
that clients are going to be limited in what they can do with the instance.
Indeed, the result would probably be a huge ball of rules close to Brad's
private instantiation -- and that would also have the risk of making new
kinds of partial views available.

A more limited instance works just as well without so many rules. But it
*is* pretty weird to have a construct that has *more* visibility for clients
than for the package itself. (The closest analog is the deferred constant,
which acts like a real constant to clients -- but that is just one minor
aspect.) And that extra visibility for clients is the key to the whole idea:
they get to see the instance in its full glory (since they don't have
elaboration and freezing issues - it's all done by the time they can see
it).

Tucker's extra "public" part actually doesn't buy all that much. It reduces
the extra naming needed for nested packages, but otherwise requires all the
same restructuring, extra incomplete types, and derivations that nested
packages do. But of course nested packages already exist and don't require
anything doing anything at all: you can use them right now in your programs.

Thus I come to the same conclusion as I have since before the Atlanta
meeting more than 5 years ago: the only solution worth pursuing is the
limited instance (and it's baby brother, the formal incomplete type). It's
reasonably simple and offers the ability to write most things as you would
expect to (not everything, of course); it doesn't break privacy or have any
bizarre freezing requirements. Elaboration happens exactly where the full
instance is written, so no surprises nor messy rules. It doesn't require
introducing extra incomplete types and derivations (it in fact prevents
derivations, a definite downside).

But I have to wonder if the advantage is enough to bother with the effort of
defining it. For me personally, I'd rather use my scarce political capital
on making containers better (with decent accessors and iterators) and on
making contracts better (with one of the solutions from my AI05-0186-1
paper). My guess is that most of you feel the same way, if put to the test.
So I think we should drop the whole idea (or just go ahead with formal
incomplete, which seems really simple).

****************************************************************

From: Tucker Taft
Sent: Thursday, April 29, 2010  2:04 AM

> The basic conversion would look like:
>
>     with Ada.Containers.Vectors;
>     package Menus is
>         type Menu_Item_Type is tagged private;
>         package Menu_Vector is new Ada.Containers.Vectors (Menu_Item_Type);

In "today's" Ada, it seems pretty standard after an instantiation of a container
generic to export the container type using a type derivation:

           type Menu_Vector_Type is
             new Menu_Vector.Vector with null record;

In other words, this is an existing idiom that is used fairly frequently.  (The
ASIS Semantic Subsystem provides an example of this with all the Holders.)

>             -- Illegal!
>         Empty_Menu_List : constant Menu_Vector.Vector;
>
>         procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
>                                 New_Submenu : in Menu_Vector.Vector'Class);
>             -- Make Menu_Item into a submenu with the given items.
>
>         function Get_Submenu (Menu_Item : in Menu_Item_Type) return
>                Menu_Vector.Vector'Class;
>
>         -- Many other operations here.
>     private
>         type Menu_Item_Type is ...;
>         Empty_Menu_List : constant Menu_Vector.Vector :=
>		Menu_Vector.Empty_Vector;
>     end Menus;
> ...
> OK, let's look at the other three solutions. First up, Tucker's baby,
> the "public" part:
>
>     with Ada.Containers.Vectors;
>     package Menus is
>         type Menu_Item_Type is tagged private;
>         type Menu_Item_List_Type is tagged; -- (1)
>         procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
>                                 New_Submenu : in Menu_Item_List_Type'Class);
>            -- Make Menu_Item into a submenu with the given items.
>
>         function Get_Submenu (Menu_Item : in Menu_Item_Type) return
>            Menu_Item_List_Type'Class; -- Newly allowed by AI-151.
>
>         -- Many other operations here.
>     private
>        type Menu_Item_Type is ...;
>     public
>         package Menu_Vector is new Ada.Containers.Vectors (Inner.Menu_Item_Type);
>         type Menu_Item_List_Type is new Menu_Vector.Vector with null record;
>         -- (2)
>         Empty_Menu_List : constant Menu_Item_List_Type :=
>		Menu_Item_List_Type(Menu_Vector.Empty_Vector);
>     end Menus;
>
> Note that we still need the incomplete type at (1), and that means
> that we have to do a derivation at (2) to complete it. And that then
> bring up the question of whether to derive Cursors and generics and the like.
>
> As with the nested package case, we need to rearrange everything,
> which isn't that pleasant.

I don't see that we are "rearranging" everything.  We have moved the
instantiation down without otherwise changing it. We haven't created any new
subpackages.  If we typically do a type derivation anyway to export the
container type, then we have exactly the same visible interface from the
client's point of view as the "ideal" (but illegal) solution.

In current Ada, if we were willing to make the "Menu_Vector_Type"
into a private type, then we would end up doing essentially exactly this, namely
moving the instantiation to a point where it is legal, and then using a type
derivation to provide the completion for Menu_Vector_Type.  I.e.:

    type Menu_Vector_Type is tagged private;

   ...

   private
    type Menu_Item_Type is ...

    package Menu_Vectors is new ...

    type Menu_Vector_Type is new Menu_Vectors.Vector with null record;

So this idiom is pretty familiar.

Really the only difference is that we can now use an incomplete type rather than
a private type, and we end up with a type that is visibly derived from
Menu_Vectors.Vector, rather than privately derived, which simplifies the users
job fairly dramatically since they don't have to define all the relevant
operations on the private Menu_Vector_Type.

From my perspective, the public/end-private solution allows the Ada programmer
to continue to do things they were already doing in pretty much the same way,
using existing idioms, but they can get a bit more visibility, which is critical
for the generic signature concept, and quite nice to have for containers of
private types.

And no need to worry about spec instantiation and body instantiation happening
at different points, no new freezing rules, and no need to "tweak" any existing
generics to make them work with this new feature.

****************************************************************

From: Bob Duff
Sent: Thursday, April 29, 2010  8:36 AM

> This is clearly a maintenance hazard.

What is the maintenance hazard you see here?

> A more limited instance works just as well without so many rules. But
> it
> *is* pretty weird to have a construct that has *more* visibility for
> clients than for the package itself.

Well, "pretty weird" is not a strong technical argument -- it's a weak aesthetic
concern.

Anyway, why is it "pretty weird"?  You can't declare a variable of a private
type in the visible part, but clients can declare variables of that same type.
So we already have ways in which clients can do more.

****************************************************************

From: Randy Brukardt
Sent: Thursday, April 29, 2010  1:41 PM

> > This is clearly a maintenance hazard.
>
> What is the maintenance hazard you see here?

Mostly with the necessary renames: if something changes in the generic package,
it won't be reflected at the site of use, leading to errors or missing
functionality. See also my answer to Tucker.

> > A more limited instance works just as well without so many rules.
> > But it *is* pretty weird to have a construct that has *more*
> > visibility for clients than for the package itself.
>
> Well, "pretty weird" is not a strong technical argument -- it's a weak
> aesthetic concern.
>
> Anyway, why is it "pretty weird"?  You can't declare a variable of a
> private type in the visible part, but clients can declare variables of
> that same type.
> So we already have ways in which clients can do more.

I was trying to explain others discomfort with this idea. I'm 100% on board
(obviously), but many others aren't. There are only two sane complaints to my
mind: (1) the fact that clients can do more; and (2) the extra declaration. The
latter seems pretty lame to me, given the Ada has a long tradition of multiple
part declarations: private types, incomplete types, deferred constants. The
former I give more weight to, in particular because the syntax doesn't seem to
invoke the right semantics (and all of the existing cases are far more limited
in scope).

The implementation ought to be easier than the other proposals (everything is
new, existing stuff hardly changes at all, no dynamic changes at all), and it
fits in well with existing Ada capabilities (it doesn't cause the complete
destruction of one of the basic meta-rules of reading Ada source, one that
implementations make a lot of use of in their code as well).

****************************************************************

From: Randy Brukardt
Sent: Thursday, April 29, 2010  2:10 PM

> > The basic conversion would look like:
> >
> >     with Ada.Containers.Vectors;
> >     package Menus is
> >         type Menu_Item_Type is tagged private;
> >         package Menu_Vector is new Ada.Containers.Vectors
> > (Menu_Item_Type);
>
> In "today's" Ada, it seems pretty standard after an instantiation of a
> container generic to export the container type using a type
> derivation:
>
>            type Menu_Vector_Type is
>              new Menu_Vector.Vector with null record;
>
> In other words, this is an existing idiom that is used fairly
> frequently.  (The ASIS Semantic Subsystem provides an example of this
> with all the Holders.)

I agree this is a common idiom, but it does not work well with (most of) the
containers. Recall that almost all containers packages export *two* types:
cursors and the container. Only Holders (which have no cursors) and vectors
(where you don't have to use the cursors, using indexes instead) could use this
idiom without major headaches.

You can't derive *both* types, and if you don't, there are a lot of reading
operations that don't get inherited (they only take cursors, not the containers,
so they're not primitive for the container). So for lists, maps, sets, and
trees, this sort of derivation would make a worse mess than doing nothing.

On top of that, you also don't get generic units (like the sorting for vectors
and lists) and the various constants. Even if you rename the generic unit, you'd
still have to use a batch of type conversions to use it (because it isn't
inherited and thus would have the original type for operands).

Finally, in the not unlikely case that the instance is passed to another generic
as the formal package, you again will have to use a passel of type conversions
to do anything.

(The derivation also didn't work in Ada 95, but that we fixed.)

Truth is, derivation is a lousy idiom to use with rich packages. It common
simply because it's all we have. The primary reason behind integrated packages
is to get rid of this counter-productive derivation. It's a secondary goal for
all of the other solutions as well. I don't think a solution that still requires
this derivation is acceptable.

...
> > As with the nested package case, we need to rearrange everything,
> > which isn't that pleasant.
>
> I don't see that we are "rearranging" everything.  We have moved the
> instantiation down without otherwise changing it.
> We haven't created any new subpackages.  If we typically do a type
> derivation anyway to export the container type, then we have exactly
> the same visible interface from the client's point of view as the
> "ideal" (but illegal) solution.

As above, no one *wants* to do the derivation, especially with the containers.
It's a terrible solution.

The need to shuffle declarations around to get things to be legal is what I mean
by "rearranging". There isn't a clear rule to give as to why to put things in
one place or another. Ordinary programmers are not going to understand the
reasons well, and will rather randomly move things until the compiler is happy.
That could lead to a lot of stuff in the following part that doesn't need to be
there, or more likely a lot of frustration.

> In current Ada, if we were willing to make the "Menu_Vector_Type"
> into a private type, then we would end up doing essentially exactly
> this, namely moving the instantiation to a point where it is legal,
> and then using a type derivation to provide the completion for
> Menu_Vector_Type.  I.e.:
>
>     type Menu_Vector_Type is tagged private;
>
>    ...
>
>    private
>     type Menu_Item_Type is ...
>
>     package Menu_Vectors is new ...
>
>     type Menu_Vector_Type is new Menu_Vectors.Vector with null record;
>
> So this idiom is pretty familiar.

Of course if you do that, you have manually reexport every vector operation that
you are going to use (the derived ones being private). That's likely to be very
unpleasant.

> Really the only difference is that we can now use an incomplete type
> rather than a private type, and we end up with a type that is visibly
> derived from Menu_Vectors.Vector, rather than privately derived, which
> simplifies the users job fairly dramatically since they don't have to
> define all the relevant operations on the private Menu_Vector_Type.
>
>  From my perspective, the public/end-private solution allows the Ada
> programmer to continue to do things they were already doing in pretty
> much the same way, using existing idioms, but they can get a bit more
> visibility, which is critical for the generic signature concept, and
> quite nice to have for containers of private types.

Derivation and especially type extension is evil when it used to get visibility,
because you get a new type and only a subset of the operations. This is a
terrible idiom (even if common), and we surely should not encourage it.
(Especially as it is actively harmful for most of the containers, as outlined
above).

> And no need to worry about spec instantiation and body instantiation
> happening at different points, no new freezing rules, and no need to
> "tweak" any existing generics to make them work with this new feature.

I agree with all of that. Limited instances do none of those things (they're a
compile-time construct only, have no effect on freezing, and don't require any
changes to generics), and as a pleasant side-effect kill the awful derivation
idiom dead.

But you are using this justify a feature that destroys an important invariant of
reading Ada code, and requires the use of an actively harmful idiom to even
work.

Honestly, if you truly think that derivation here is a good thing (or even
something to be encouraged), then it is clear to me that we really have no
chance whatsoever on any sort of agreement about this feature. Which again
suggests that doing nothing is preferable, because we don't even agree on the
goals. (The current situation is "good enough" if you are willing to do these
derivations and renames. It is those of us that disagree with doing those that
need better solutions.)

****************************************************************

From: Tucker Taft
Sent: Friday, April 30, 2010  7:37 AM

Can you visibly derive from a type defined by such a "limited" instance
(presumably with a private extension if tagged), immediately following it? Can
you do any renamings immediately following it?

****************************************************************

From: Edmond Schonberg
Sent: Friday, April 30, 2010  7:42 AM

By "it" you mean the instance whose actual is not fully defined yet?
Then the answer should be no, just as for any incomplete type.

****************************************************************

From: Tucker Taft
Sent: Friday, April 30, 2010  7:53 AM

Sorry for the confusion.  This question was in the context of limited instances,
not formal incomplete types.

My point is that if you can't use standard idioms with them such as renamings or
type derivations, they become less useful and more confusing.

For an instance of a generic with a formal incomplete type, I presume you can do
anything with the types defined inside the generic, because they are "normal"
types.  But of course it would not be easy to change a container generic to take
its "Element" type as a formal incomplete type!

****************************************************************

From: Randy Brukardt
Sent: Friday, April 30, 2010  1:44 PM

> My point is that if you can't use standard idioms with them such as
> renamings or type derivations, they become less useful and more
> confusing.

These are "standard idioms" that need to be abandoned ASAP, so I have a hard
time caring.

As I mentioned yesterday, derivation from the containers packages is actively
harmful. And you can get the same (incomplete) effect by using prefix notation
in your calls without introducing any junk types. Indeed, at best (which is
rare), derivation is neutral, but it almost always introduces complications. It
should be reserved for cases where you really need a new type (which is never
the case for this idiom).

Renames aren't quite as bad, but they take an amazing amount of work to be
useful. And they are error-prone and a long-term maintenance headache (every
change in the generic has to be reflected in all of the renames).

The point of integrated packages is to be able to avoid this idiom and get the
same effect without the downsides (it has other issues, of course). If you
really believe in the value of this idiom, you should be in favor of integrated
packages no matter what other features exist. But I suspect hardly anyone really
cares to that point.

In any case, this is the fundamental root of our disagreements on this topic. I
think the idiom is so distasteful that being forced to use it in any reasonable
case automatically disqualifies a proposed feature from consideration. You think
this idiom is so important that being unable to use it in any reasonable case
automatically disqualifies a proposed feature from consideration. These two
positions cannot be reconciled, so unless we can find evidence that one or the
other positions is held by no one else, there really is no point in continuing
to discuss this topic. And I don't believe this is the case; Dan Eilers has been
lobbying for years for some way to eliminate that derivation, and I would guess
that he is just the loudest voice. I would be surprised if you were the only one
that misguidely thinks that idiom is important.

The point here is that to support this in some reasonable way, we're going to
have to drastically change the way people read and write Ada programs. Either
we're going to invalidate some common (but dubious) idioms, or we're going to
have to make a sea change in how we read specifications. Can this level of
disruption be worth solving this problem (even if we could agree on which one to
do)?? I seriously doubt it.

So let's stop wasting our time on this one and work on something useful (say,
the wording for the magic accessors, referencing, and iterators...).

****************************************************************

From: Tucker Taft
Sent: Friday, April 30, 2010  2:21 PM

This idiom is used heavily in much of the Ada code I have seen that uses
generics to define container-like things.  I agree the cursors add complexity,
but there are plenty of container packages that don't have cursors, but instead
rely on generic instantiation.

I agree the integrated packages would have advantages, and this is one of them.
But I don't see as much value in other proposals that make type derivation
and/or renaming impossible, meaning that all types defined by generic
instantiation are at a nested level.

Especially when you use a generic instantiation to "mix-in" something, say add a
linked-list aspect, you really want to bring the resulting type back out to the
outer level.

Lacking something like integrated pacakges, I see the need to continue to
support the derivation/renaming idiom.  And for private types, that is exactly
what you want, so it is an idiom that will continue to be used in that context.

The end-private/public approach allows this idiom to be used, but doesn't
*require* it.

Interestingly, the limited instance and the end-private/public approach would
work well *together,* because after the end of the private part, presumably the
limited instance would become unlimited, and you could derive and/or rename
entities declared inside it, and generally make full use of it.

In fact, one nice feature of the "end-private/public"
is that it works fine with almost any existing or proposed feature of the
language, since it is just allowing part of the visible part to come after all
private types, deferred constants, and, potentially, limited instances, have
been completed.

Maybe there is a combined approach here that beats either one alone...

****************************************************************

From: Tucker Taft
Sent: Friday, April 30, 2010  2:32 PM

One way of making the end-private/public idea more palatable might be to require
"begin private" if you want to have an "end private".  That way the reader is
warned right from the beginning that they should not stop reading.  E.g.:

package Foo is
    type T is private;

    ...

   begin private
      type T is ...;
   end private;

    package T_Signature is new Set_Signature(T);
    ...

end Foo;

This also eliminates the need for the "public" reserved word, while keeping a
balance between "begin"s and "end"s.

****************************************************************

From: Bob Duff
Sent: Friday, April 30, 2010  2:43 PM

> One way of making the end-private/public idea more palatable might be
> to require "begin private" if you want to have an "end private".  That
> way the reader is warned right from the beginning that they should not
> stop reading.  E.g.:

Good idea.  This should calm Randy's rabid opposition to the "end private"
syntax.

****************************************************************

From: Steve Baird
Sent: Friday, April 30, 2010  3:02 PM

I like this idea. Now I need to go back and review the old discussions of this
proposal to look for visibility-related issues.

IIRC, the text after "end private" does not have visibility into the private
part, but it also must not declare homographs, nor can it declare completions of
things declared in the private part (e.g., if the private part contains "type
T;" and does not complete it, then the completion must come in the body, not in
the post-private part of the spec).

****************************************************************

From: Randy Brukardt
Sent: Friday, April 30, 2010  6:02 PM

> This also eliminates the need for the "public" reserved word, while
> keeping a balance between "begin"s and "end"s.

This is better, but it still puts an end and a semicolon in the middle of a
construct.

And its irrelevant, because no version of this is going to solve the problem for
most of the containers.

This solution (like the current Ada solution) requires an extra incomplete type
(private doesn't work because you can't complete that in any visible part) and a
derivation to complete that type. But that derivation is necessarily going to
only partially inherit operations from the instance (especially for list, map,
set, tree containers that use cursors exclusively). Thus all it can do is make a
mess.

Aside: The definition of the containers (specifically, that container read
operations do not take a container parameter) is a disaster for many reasons:

* you can't use prefix calls for reading operations using cursors, forcing reads
  and writes to look different, and generally forcing a Hobson's choice between
  a package use clause for the instance or lots and lots of dot notation;

* reading operations using cursors aren't primitives of the containers, so that
  derivations don't bring them along. This means that it's generally impractible
  to extend the containers usefully: you have to manually recreate all of the
  reading operations. It also destroys the derivation idiom that Tucker is
  mysteriously fond of;

* Not having the container as an explicit parameter makes it impossible to write
  pre/post conditions based on the container object (this could be fixed if we
  provided an operation to extract a container from a cursor, but that operation
  would be unsafe [since it wouldn't reflect the container lifetime]);

* In addition, since the container is not explicit, the "global in" has to
  include all containers of the appropriate type. That would kill much
  optimization unnecessarily.

I disliked this organization from the beginning, but didn't realize just how
terrible it is until much later. I wish I had been more forceful against it. Now
we're stuck with it (we're not going to start over with the containers).

End aside.

Presuming that we are not redesigning the containers, this will only work
adequately for the vector and holder containers. The others could be written
this way, but the results would be a total mess (requiring dozens of explicit
subprogram definitions to straighten out, if it is even possible). I don't see
any point in pursuing a solution that only works for a few containers, or still
requires dozens of renames to make a tolerable solution.

So, unless you can eliminate any requirement to derive a type (and still allow
use as parameters in primitive operations) or you are willing redesign the
containers to make this derivation less of a mess, there is really no point in
pursuing any solution. (If you're willing to have a mess, you can do that
trivially in Ada 2005 already, and Ada 2012 makes it even easier).

****************************************************************

From: Bob Duff
Sent: Friday, April 30, 2010  6:33 PM

> > This also eliminates the need for the "public" reserved word, while
> > keeping a balance between "begin"s and "end"s.
>
> This is better, but it still puts an end and a semicolon in the middle
> of a construct.

Please think of the "private part" as a "construct", syntactically.
Then I think you'll like Tucker's idea, as far as syntax.  There's no "in the
middle" then, other than the usual nesting of blahbity-blah...end blahbity-blah
syntax we already have.

As to semantics:

> And its irrelevant, because no version of this is going to solve the
> problem for most of the containers.

I'm lost.  I thought the "begin private ... end private" thing belongs in
_clients_ of the containers, so I don't see how that would affect the containers
themselves.  Please enlighten me.

****************************************************************

From: Randy Brukardt
Sent: Friday, April 30, 2010  7:32 PM

>Please think of the "private part" as a "construct", syntactically.
>Then I think you'll like Tucker's idea, as far as syntax.  There's no
>"in the middle" then, other than the usual nesting of blahbity-blah
>...end blahbity-blah syntax we already have.

A private part cannot semantically be separated from the rest of a package
specification. It's just a portion; it shouldn't *look* separate in any
important way.

After all, if we add this, it is a very short step to saying that these should
be separately compilable. That way lies madness (at least mine... :-) [I am
strongly opposed to such things, as they can only be implemented in a
source-based compiler -- our compiler has no notion of source dependencies and
there is no current need for such dependencies.]

...
> > And its irrelevant, because no version of this is going to solve the
> > problem for most of the containers.
>
> I'm lost.  I thought the "begin private ... end private"
> thing belongs in _clients_ of the containers, so I don't see how that
> would affect the containers themselves.  Please enlighten me.

Maybe I should have said "And its irrelevant, because no version of this is
going to solve the problem {when using}[for] most of the containers."

But I suppose I need to explain this problem more clearly.

Let's look at a selected part of the specification of the list container (I
stripped out a lot of irrelevant stuff):

generic
   type Element_Type is private;
   with function "=" (Left, Right : Element_Type)
      return Boolean is <>;
package Ada.Containers.Doubly_Linked_Lists is
   type List is tagged private;

   type Cursor is private;

   Empty_List : constant List;

   No_Element : constant Cursor;

   function Is_Empty (Container : List) return Boolean;

   function Element (Position : Cursor) return Element_Type;

   procedure Replace_Element (Container : in out List;
                              Position  : in     Cursor;
                              New_Item  : in     Element_Type);

   procedure Append (Container : in out List;
                     New_Item  : in     Element_Type;
                     Count     : in     Count_Type := 1);

   function First (Container : List) return Cursor;

   function Last (Container : List) return Cursor;

   function Next (Position : Cursor) return Cursor;

   function Previous (Position : Cursor) return Cursor;

   ...

The problem here is that there are a number of operations that operate on an
implicit container. They are container operations, but they don't have a
container parameter. That means that these operations ("Element", "Next",
"Previous" above) aren't primitive operations of type List.

That means that when you derive from type List, you don't get those operations.
So if you are using derivation to make operations visible, you won't get those
operations at all. Moreover, you can't use those operations in prefix notation
(which greatly reduces the need for either the derivation or prefixing by the
instance name: it's only necessary for the type name when declaring objects).

In addition, since the derivation creates a new type, it's possible that those
operations will require type conversions to work, or worse fail internally
(raising Constraint_Error because of a type conversion tag check).

Deriving from type Cursor would make things worse, since then the cursors and
the lists would have different types (we'd have operations with a new cursor and
an old list, and a old cursor and a new list, but never both being new!).

This doesn't mention that fact that the constants Empty_List and No_Element
aren't inherited, nor is Generic_Sorting. Making sense of this will require a
bunch more renames and the like.

(As I noted in my last message, this also causes problems for extensions of type
List, since there is no way to override the cursor operations like Element to
use the extension. They have to be rebuilt if needed - yuck.)

Derivation of a type from a generic instance almost always has these problems.
(It's similar to the reason that you voted against "use all type"; it doesn't
get everything.) It's possible to create a generic unit without these issues,
but its rarely done.

Thus, I find "solutions" that require derivation to be distasteful. The current
Ada solution for this problem requires that sort of derivation in many common
cases -- but it already exists and we're surely not removing it so that's OK.
But I don't see any value to adding a solution that continues to perpetrate this
problem.

To show an example of the problem, recall my example of the other day,
rejiggered to use lists and this latest syntax proposal:

    with Ada.Containers.Doubly_Linked_Lists;
    package Menus is
        type Menu_Item_Type is tagged private;
        type Menu_Item_List_Type is tagged; -- (1)
        procedure Make_Submenu (Menu_Item : in out Menu_Item_Type;
                                New_Submenu : in Menu_Item_List_Type'Class);
           -- Make Menu_Item into a submenu with the given items.

        function Get_Submenu (Menu_Item : in Menu_Item_Type) return
           Menu_Item_List_Type'Class; -- Newly allowed by AI-151.

        -- Many other operations here.
    begin private
       type Menu_Item_Type is ...;
    end private;
        package Menu_List is new Ada.Containers.Doubly_Linked_Lists (Inner.Menu_Item_Type);
        type Menu_Item_List_Type is new Menu_List.List with null record; --(2)
        -- (3)
        Empty_Menu_List : constant Menu_Item_List_Type := Menu_Item_List_Type(Menu_List.Empty_Vector);
    end Menus;

The incomplete type at (1) is necessary in order that the Make and Get
operations are primitive for Menu_Item_Type. That forces the use of the
derivation at (2).

But in order to have a complete set of operations, we need to rename/redeclare
the ones based on cursors at (3), or we have to use type conversions to use the
original operations within Menu_List. This is most acute for the sorting generic
in Menu_List (it's not possible to instantiate it for Menu_Item_List_Type, only
for Menu_List.List, so lots of type conversions are needed to use it).

We don't have that confusion if we use the operations directly declared within
Menu_List, even if that forces use of "use Menu_List" or dot notation.

So I don't believe that any solution that requires restructuring that package
such that a derivation like (2) is required is any real benefit. (Recall that
you can write the above already by putting all of it up to "end private" except
for (1) into a nested package.) The big win is avoiding the need to introduce
extra, unnecessary types. If we could get rid of the extra dot notation, that
would be a bonus. Integrated packages was intended to do the second, but we
never really fixed the first, more important problem.

Either of the instantiation-based solutions do fix the extra type problem, but
they both have the effect of preventing the use of this derivation idiom. While
I think this is a benefit, I can see why some (named Tucker) think that this is
a deficit!

BTW, Brad's major objection to these solutions also was the need to do type
derivation. His reason was different, however: the types he wanted to define in
the generic are not allowed by the language to be derived (protected types). So
the above organization still leaves Brad unable to use his reusable task-safe
containers. (Note that this also applies to the Queue containers that we're
defining as part of the language!!)

Anyway, if we could eliminate the need for the type derivation, then I'd just be
nauseous rather than violently ill. :-) One thought would be to allow completion
of incomplete types by some sort of type renaming declaration. That could be a
subtype declaration, but I suspect that the subtype would have to be limited to
a subtype_mark representing a first subtype. (I believe that we previously tried
the general case and it blew up.)

****************************************************************

From: Randy Brukardt
Sent: Monday, May 3, 2010  8:49 PM

For some reason, I didn't see this reply on Friday, but I wanted to answer one
point...

Tucker Taft writes:
> This idiom is used heavily in much of the Ada code I have seen that
> uses generics to define container-like things.  I agree the cursors
> add complexity, but there are plenty of container packages that don't
> have cursors, but instead rely on generic instantiation.

True, but only because nothing better is available. No one wants to introduce
extra types here.

> I agree the integrated packages would have advantages, and this is one
> of them. But I don't see as much value in other proposals that make
> type derivation and/or renaming impossible, meaning that all types
> defined by generic instantiation are at a nested level.
>
> Especially when you use a generic instantiation to "mix-in"
> something, say add a linked-list aspect, you really want to bring the
> resulting type back out to the outer level.
>
> Lacking something like integrated pacakges, I see the need to continue
> to support the derivation/renaming idiom.  And for private types, that
> is exactly what you want, so it is an idiom that will continue to be
> used in that context.

No, this is *not* what you want for private types. What you want is to be able
to complete the type with whatever is exported from the generic instance without
introducing additional junk types. If the generic instance is used as a formal
package, or if it exports generic units of its won (like Sorting), or if it has
something like Cursors, those junk types make things messy at best (forcing the
insertion of many, many type conversions). I'm acutely aware of this, because
there were lots of problems like this in the code generated by the Claw Builder
-- which attempted to use this idiom for visibility.

That's true in *all* of these cases: you want to bring out/complete the type
with the type exported from the instance - there is no need to introduce an
extra type which always causes problems (sometimes minor, sometimes major).

> The end-private/public approach allows this idiom to be used, but
> doesn't *require* it.

Yes it does, if there is any need to use the container exported type in visible
primitive operations. That seems pretty likely (indeed, if you *don't* need
this, why in the world are you putting the container in the same specification
in the first place? Just to give yourself extra pain???)

My point is that you are forced to use this idiom because there is no better way
to deal with creating the needed view of the type early enough to be useful. The
incomplete instance gives you that, but it does mean that the client view is
compromised. Probably the *best* solution would be a combination of the
incomplete instance and the integrated package.

> Interestingly, the limited instance and the end-private/public
> approach would work well
> *together,* because after the end of the private part, presumably the
> limited instance would become unlimited, and you could derive and/or
> rename entities declared inside it, and generally make full use of it.
>
> In fact, one nice feature of the "end-private/public"
> is that it works fine with almost any existing or proposed feature of
> the language, since it is just allowing part of the visible part to
> come after all private types, deferred constants, and, potentially,
> limited instances, have been completed.
>
> Maybe there is a combined approach here that beats either one alone...

The important issue is to find a way to eliminate the derivation idiom, because
it works very poorly with the existing containers, and in any case introduces
extra junk types that are not needed for strong type enforcement (so they are
going to get in the way as outlined above).

If we can do that, we can make any of the solutions work (probably even
including the existing nested packages). If not, we're wasting our time (at
least for Ada.Containers, which I thought was the primary goal).

****************************************************************


Questions? Ask the ACAA Technical Agent