Version 1.3 of ai05s/ai05-0074-1.txt

Unformatted version of ai05s/ai05-0074-1.txt version 1.3
Other versions for file ai05s/ai05-0074-1.txt

!standard 12.3(1)          07-10-24 AI05-0074-1/01
!class Amendment 07-10-24
!status work item 07-10-24
!status received 07-09-27
!priority Medium
!difficulty Hard
!subject Limited view of generic instantiations
!summary
(See proposal.)
!problem
Ada 95 provides formal package parameters. One way of using these parameters is to define a "signature" for a class of abstractions, such as all set abstractions, or all physical unit abstractions, and then build a new generic abstraction on top of the original class of abstractions using this signature as the template for a formal package parameter.
Unfortunately, it is difficult to use signatures because of the fact that an instantiation freezes all of its actual parameters.
For example:
Given the signature for a set abstraction:
generic type Element is private; type Set is private; with function Size(Of_Set : Set) return Natural is <>; with function Union(Left, Right : Set) return Set is <>; with function Intersection(Left, Right : Set) return Set is <>; with function Empty return Set is <>; with function Unit_Set(With_Item : Element) return Set is <>; with function Nth_Element(Of_Set : Set) return Element is <>; package Set_Signature is end;
... we could define a generic that required some set abstraction, but it didn't care which one so long as it implemented the above signature:
generic with package Base_Set is new Set_Signature(<>); package Layered_Abstraction is type Cool_Type(Set : access Base_Set.Set) is limited_private; procedure Massage(CT : in out Cool_Type; New_Set : Base_Set.Set); ...
end Layered_Abstraction;
Now if we want to define a set type and provide the pre-instantiated signature for it, we run into trouble:
generic type Elem is private; with function Hash(El : Elem) return Integer; package Hashed_Sets is type Set is private; function Union(Left, Right : Set) return Set; ...
package Signature is new Set_Signature(Elem, Set); private type Set is record ... end record; end Hashed_Sets;
The problem is that we can't do the instantiation of Set_Signature where we would want to do so, because the instantiation freezes the type "Set" prematurely.
A similar problem occurs when a type wants to include a pointer to a container based on the type being defined. For example:
package Expressions is type Expr_Ref is private;
package Expr_Sequences is new Sequences(Expr_Ref); -- Freezing trouble here! type Seq_Of_Expr is access Expr_Sequences.Sequence;
function Operands(Expr : Expr_Ref) return Seq_Of_Expr;
...
private type Expression; -- completion deferred to body type Expr_Ref is access Expression; end Expressions;
Here we have a case where we want to instantiate a generic using a private type, and use the results of the instantiation as the designated type of an access type, which is in turn used as a parameter or result type of an operation on the private type.
Unfortunately, we can't instantiate the "Sequences" generic with Expr_Ref, since it is private.
!proposal
Introduce a "limited instantiation" as a "specification" for an instance given later.
!wording
[Editor's note: This wording isn't all up to AARM quality; some was borrowed from the last version of AI95-359-4.]
limited_generic_instantiation ::= package defining_identifier is limited new generic_package_name generic_actual_part;
A limited_generic_instantiation shall be given in the visible part of a package specification. A limited_generic_instantiation requires a completion, which shall be a full generic_instantiation in the private part of the same package specification. The actual parameters of the completion shall match the corresponding actual parameter of the limited package instantiation, whether the actual parameter is given explicitly or by default.
The rules for matching of actual parameters between the completion and the limited instantiation are as follows:
* For a formal object of mode IN the actuals match if they are static expressions with the same value, of if they statically denote the same constant, or if they are both the literal NULL.
* For a formal subtype, the actuals match if they denote statically matching subtypes.
* For other kinds of formals, the actuals match if they statically denote the same entity.
AARM NOTE: We considered using full conformance rules here instead of
formal-package-ish matching. However, we wanted to use rules consistent with formal packages, and it seemed simpler to just define the particular matching rules needed between instantiations. Also we wanted to ensure that no evaluations would take place at the point of the limited package instantiation.
[I borrowed this from the last draft of AI95-0359-4 - ED.]
The elaboration of an limited_generic_instantiation has no effect.
For a reference to the reference to the defining_program_unit_name of an limited_generic_instantiation:
* If the reference is within the package specification that contains the limited_generic_instantiation (and before the completing generic_instantiation):
The reference denotes the limited view of the package created by expanding the generic unit [Unfortunately, we can't just say the "limited view of the instantiation", because that is empty. Better
wording will be needed].
* Otherwise, the reference denotes the completing generic_instantiation. (The fact that that instantiation is in the private part is ignored.) [I know this is not quite right, based on the discussion in Atlanta. The view cannot include any information derived from the private part, in particular additional operations declared based on the full declaration of any private types. The AI95-359-4 wording was "The declarations within a limited generic instance provide views of the entities declared by the corresponding declaration within the full instance. The view is static only if the entity was static in the generic package." But that doesn't quite work with the formulation I gave above.]
Change 13.14(5):
* The occurrence of a generic_instantiation {other than a limited_generic_instantiation} causes freezing; ...
AARM NOTE: The rules on the use of incomplete types prevents trouble
using entities declared in a limited_generic_instantiation.
Change 13.14(8/1):
... An object name or nonstatic expression causes freezing where it occurs, unless the name or expression is part of a default_expression, a default_name, {a limited_generic_instantiation,} or a per-object expression of a components's constraint, in which case[,] the freezing occurs later as part of another construct.
!discussion
This is limited to packages because you can use renaming-as-body to get this effect for subprogram instantiations. Moreover, you cannot export a type from a generic subprogram, so there is no possibility of recursive use. [Note that we could lift this limitation if it if felt to be more regular.]
!example
Here's an example of making a Container available in the visible part of a package:
package Something is type Priv is tagged private; package Priv_Lists is limited new Ada.Containers.Doubly_Linked_Lists (Priv); ... private type Priv is ... package Priv_Lists is new Ada.Containers.Doubly_Linked_Lists (Priv); end Something;
with Something; procedure Do_It is PList : Something.Priv_Lists.Container; -- This is allowed. begin ... end Do_It;
And here's an example of using a recursive data structure as part of the definition of a private type:
package Something_Else is type Node is tagged private; package Node_Lists is limited new Ada.Containers.Doubly_Linked_Lists (Node); ... private type Node is tagged record Parent : access Node; Children : access Node_Lists.Container; -- "Recursive" definition. Siblings : access Node_Lists.Container; ... end record; package Node_Lists is new Ada.Containers.Doubly_Linked_Lists (Node); end Something_Else;
!ACATS test
ACATS B and C-Test(s) are necessary.
!appendix

From: Tucker Taft
Date: Thursday, September 27, 2007 12:33 PM

Robert made the mistake of asking me before his
talk at AdaUK on Tuesday what was on my "wish
list" for Ada 2015.  I mentioned three things:

   1) Instantiation of generics with private types
   2) Region-based storage management
   3) Units checking

This of course got me to thinking some more about these
things.  As far as instantiation of generics with
private types, in the same package where the private
type is declared, I am currently enamored of two
different solutions, both of which might be interesting
in their own right:

   A) Allow the notion of "end private;" where the private
      part ends before the end of the package spec, and then
      you can return to visible declarations.

   B) Allow the ability to have child units of a
      package or generic package that are effectively
      "pre-withed" by all dependents.

With (A), you simply move the instantiations to a point
after the "end private;" and then you can instantiate
any generic with the now completed private types
declared in the package.  Robert immediately asked whether
it made sense to have multiple private parts in a
single package.  I didn't have an immediate response.

With (B), you make your instantiations be child units,
and then "pre-with" them on behalf of your clients,
and then you are able to use private types in
the instantiations, and your clients think of them as
sub-packages.  I believe GNAT may at one point have
played a trick roughly like this for Text_IO, by making
Integer_IO and friends into children of Text_IO, but
"pre-with"ing them so clients see them as subpackages.
If a generic unit "pre-withs" a child, then there is
no particular reason the child needs to be limited to
being itself a separate generic, since the instantiator of the
parent unit "knows" to instantiate the pre-with'ed
children as well.

Syntactically, I thought that given "with" creating
a dependence starting at the beginning of the
compilation unit, and "private with" creating
a dependence starting at the word "private,"
then "end with <child1>, <child2>;" might be
a natural way to "pre-with" the specified children,
with it taking effect at the end of the
compilation unit.

E.g. for Text_IO, one might have:


     private with System.RTS.Files;

     end with Ada.Text_IO.Integer_IO,
              Ada.Text_IO.Enumeration_IO,
              Ada.Text_IO.Float_IO, ...

     package Ada.Text_IO is
         type File_Type is private;
         ...
     private
         type File_Type is new
           System.RTS.Files.File_Type with ...
     end Ada.Text_IO;
        -- conceptually, the "with"s of children
        -- happen here, and *are* inherited by all
        -- clients of Ada.Text_IO.


Comments, flames, etc. as usual.

****************************************************************

From: Randy Brukardt
Date: Thursday, September 27, 2007  1:41 PM

> Comments, flames, etc. as usual.

How's your homework coming? ;-)

Oh, you wanted comments on your ideas??

The second one looks more appealing (not the syntax, though!), because it
doesn't require changing the visibility model of the language. However, the
"forward" reference in the "end with" as you have it would be problematical,
because it requires referencing something that by definition could not have
yet been created. I know that when I tried to use such an idea in one of the
early proposals for what became limited with, it was not well received by
users. I'm not sure if that is a real problem in this context.

Syntactically, I'd suggest something like "include with", which would of
course require another keyword.

Going back one more step, here's my "wish list" for Ada 2015:

   1) In out parameters for functions (with expressions with function calls for
      which order could change the result being made illegal);
   2) More exposure of concurrency for multi-core systems (combined with more
      checking to reduce the possibility of deadlocks/indeterminate results);
   3) Instantiation of generics with private types.

****************************************************************

From: Robert A. Duff
Date: Thursday, September 27, 2007  6:00 PM

>   1) Instantiation of generics with private types

It seems to me that the most important case is where there is some mutual
recursion going on, such as:

    with Ada.Containers.Vectors; use Ada.Containers;
    package Trees is
        type Tree is private;
        package Tree_Vectors is new Vectors (Tree,...); -- Illegal!
        ...
    private
        type Tree is
            record
                ...
                Subtrees : access Tree_Vectors.Vector;
            end record;
    end Trees;

Otherwise, you can just put the instance in a child.  That's an annoyance,
because clients have to 'with' it, but it doesn't prevent you from doing
anything.  The mutually recursive case, on the other hand, requires massive
restructuring to work around the restriction.

****************************************************************

From: Tucker Taft
Date: Thursday, September 27, 2007 11:16 PM

> It seems to me that the most important case is where there is some mutual
> recursion going on, such as:
> 
>     with Ada.Containers.Vectors; use Ada.Containers;
>     package Trees is
>         type Tree is private;
>         package Tree_Vectors is new Vectors (Tree,...); -- Illegal!
>         ...
>     private
>         type Tree is
>             record
>                 ...
>                 Subtrees : access Tree_Vectors.Vector;
>             end record;
>     end Trees;

Couldn't you just use an incomplete type here:

      with Ada.Containers.Vectors; use Ada.Containers;
      package Trees is
          type Tree is private;

          ...
      private
          type Tree_Vector;
          type Tree is
              record
                  ...
                  Subtrees : access Tree_Vector;
              end record;

          package Tree_Vectors is new Vectors (Tree,...);
          type Tree_Vector is new Tree_Vectors.Vector;
      end Trees;

Presuming you are going to use access values, I don't
see a problem.  You could still use the "end private"
to make the instantiation to be *visible*, if that
were desirable.

> Otherwise, you can just put the instance in a child.  That's an annoyance,
> because clients have to 'with' it, but it doesn't prevent you from doing
> anything.  The mutually recursive case, on the other hand, requires massive
> restructuring to work around the restriction.

The one place where putting the instantiation in the child
doesn't really work is when you are trying to include
the instantiation of, say, a signature package within a generic.
A generic can't have a child that is an instantiation.
Either the "end private" or the "end with" would
solve this.

****************************************************************

From: Robert A. Duff
Date: Friday, September 28, 2007  7:00 AM

> Presuming you are going to use access values, I don't
> see a problem.  You could still use the "end private"
> to make the instantiation to be *visible*, if that
> were desirable.

OK, that's reasonable, so long as there's a way to export Tree_Vector (and its
ops).

To have mutual recursion, Ada requires an access type to break the chain.
I would prefer any solution allow the programmer maximum flexibility in
where to put that access type (e.g. as above, versus in the generic, etc).
Otherwise, there will be cases where you are annoyingly forced to use
TWO access types.

> > Otherwise, you can just put the instance in a child.  That's an annoyance,
> > because clients have to 'with' it, but it doesn't prevent you from doing
> > anything.  The mutually recursive case, on the other hand, requires massive
> > restructuring to work around the restriction.
> 
> The one place where putting the instantiation in the child
> doesn't really work is when you are trying to include
> the instantiation of, say, a signature package within a generic.
> A generic can't have a child that is an instantiation.
> Either the "end private" or the "end with" would
> solve this.

OK.  But the "end private" idea has the advantage that you can refer to the
instance from within the package itself, right?

I'm pretty sure that if you want two visible parts, you're going to want
several visible parts and several private parts.  Somebody famously said,
a programming language should have zero, one, or infinite of any thing.

By the way, the "end with" syntax is, well..., it will take some "getting used"
to.  If I see "end" (especially at the beginning of a line) I think "this is
the end of something".

****************************************************************

From: Jean-Pierre Rosen
Date: Friday, September 28, 2007  9:05 AM

> By the way, the "end with" syntax is, well..., it will take some "getting used"
> to.  If I see "end" (especially at the beginning of a line) I think "this is
> the end of something".
> 
Hmmm... since the effect is to automatically with the children, what 
about "with all" ?

(Designing new syntax is always great fun :-)

****************************************************************

From: Randy Brukardt
Date: Friday, September 28, 2007  1:11 PM

> By the way, the "end with" syntax is, well..., it will take some "getting used"
> to.  If I see "end" (especially at the beginning of a line) I think "this is
> the end of something".

As I said yesterday, the idea doesn't bother me, but the syntax is horrible.
It doesn't convey the meaning of the construct at all (which is that the
with is "included" in any uses of this package). I suggested "include with",
because that conveys the meaning, and I would much rather that we referred
to Tucker's idea that way, so that it is intuitively obvious what it is
we're talking about. ("end private" has that effect, too, as well as
complete revulsion. :-). Maybe someone else has a syntax suggestion that
actually conveys the meaning (sorry, J-P, but "with all" is just as bad),
but in the absence of that, I strongly suggest that we use "include with"
since it does.

After all, my generic default type proposal was defeated primarily because
of syntax concerns. I would think Tucker would not want that to happen to
this idea.

****************************************************************

From: Tucker Taft
Date: Friday, September 28, 2007  2:27 PM

Or "all with Ada.Text_IO.Integer_IO;"
    -- That is "all [clients implicitly] with Ada.Text_IO.Integer_IO"

Another approach is to use "is separate" which has
been bandied about in the past, and treat them
as package "spec subunits."  E.g.:

   package Ada.Text_IO is
     ...
     generic package Integer_IO is separate;
     generic package Float_IO is separate;
   private
     ...
   end Ada.Text_IO;

****************************************************************

From: Jean-Pierre Rosen
Date: Monday, October 1, 2007  3:23 AM

And how would it be different from a child package, apart from not 
seeing the private part of the parent?

****************************************************************

From: Randy Brukardt
Date: Monday, October 1, 2007  2:30 PM

Because it would be automatically included whenever the parent is withed,
whereas a child unit has to be withed separately. Tucker is trying to make
it easier to separate specifications into separate units when the logical
view is a single unified specification.

****************************************************************

From: Tucker Taft
Date: Monday, October 1, 2007  2:53 PM

Bob makes the point that the "end private" idea
is more powerful, in that you can use items from
the instantiation to visibly complete incomplete types
declared before the instantiation.  When trying
to create mutually recursive types, that seems
pretty important.

From an implementation point of view, multiple
visible parts also seems simpler than "spec subunits"
or "inclusive with" of children.  Any comments
on the "end private" syntax, and/or whether a semicolon
is desirable after "end private"?  I suspect a different
indenting would be used for "private/end private,"
such as:

     package Trees is
         type Tree is private
         ...
       private
         ...
       end private
         ...
     end Trees;

****************************************************************

From: Edmond Schonberg
Date: Monday, October 1, 2007  3:22 PM

A smaller syntactic change would be to have both views of the type in  
a single declaration:

private type Tree is record ...

You can have as many of those as you want, which answers the "zero,  
one, infinity" rule.   Full "private sections" would sit heavily on  
my stomach (and Randy's too, I recall).

****************************************************************

From: Randy Brukardt
Date: Monday, October 1, 2007  3:45 PM

...
>  From an implementation point of view, multiple
> visible parts also seems simpler than "spec subunits"
> or "inclusive with" of children.

Gee, Tucker, what are you smoking these days? I want some! :-)

The implementation of "inclusive with" is trivial: just jam an extra with
into the list of withed units when the appropriate thing is encountered in
the loaded unit. (But I do agree that it is not an ideal solution to the
mutual recursion problem.)

OTOH, the implementation of a new kind of visibility would have a lot of
effects throughout the compiler, because visibility is such a pervasive
idea. (And in Janus/Ada it is somewhat distributed, which makes things even
worse.) I also worry about information "leakage" out of the private part; we
currently have a number of rules intended to ensure that you cannot depend
on the contents of a private part, and it seems likely that there would be
effects visible in an "end private" part that would effectively render items
in the private part visible.

I still think this is a bad idea; I still think that two part instantiations
can work if sufficiently limited (we never really finished working on that
idea - it was only the broadly defined version that didn't work). Ed's idea
also would work (although it has the issue of exposing the implementation in
practice even if not semantically).

****************************************************************

From: Robert A. Duff
Date: Monday, October 1, 2007  5:15 PM

>...Any comments
> on the "end private" syntax, and/or whether a semicolon
> is desirable after "end private"?

From a purely syntactic point of view, I don't mind "end private".
It was "end with" that rubbed me the wrong way.

I suppose you can have multiple private parts, as in:

    package P is
        ... -- public stuff
    private
        ... -- private stuff
    end private;
        ... -- public stuff
    private
        ... -- private stuff
    end private;
        ... -- public stuff
    end P;    

?

A semicolon?  Yeah, I guess so.  Shrug.

Syntax is important, but the semantics is where trouble might arise.

****************************************************************

From: Tucker Taft
Date: Monday, October 1, 2007  5:35 PM

I should know better than to guess at relative
implementation difficulties.  My presumption
was that multiple visible parts were not
significantly different from single visible
parts.  There isn't really a "new" kind of
visibility, is there?  Some declarations
in the spec are private, and some are visible,
which is how it is now.

But I guess the universal rule in this area is:
Your Mileage May Vary.

I would be curious whether how other vendors
see the relative implementation difficulty.

One major problem I have with the "private type ..."
suggestion is that you don't have any way to
declare types that are not visible at all
using that notation.  It is quite common that
some of the types of components of a private
type are not visible at all.  I also worry
that the "private type ..." proposal is
a completely different approach to visibility
from a conceptual point of view, creating
a dramatic shift in perception of how things
are done.

****************************************************************

From: Randy Brukardt
Date: Monday, October 1, 2007  6:07 PM

> I should know better than to guess at relative
> implementation difficulties.  My presumption
> was that multiple visible parts were not
> significantly different from single visible
> parts.  There isn't really a "new" kind of
> visibility, is there?  Some declarations
> in the spec are private, and some are visible,
> which is how it is now.

Well, each declaration is either visible or it is not at a particular point
of the program. But that changes for each different unit/scope that you go
through. There are already multiple kinds of private parts (package,
protected), and the rules for what is visible are complex.

Our implementation changes a visibility flag on each declaration as we enter
or leave different program units; how that works exactly is different for
packages, subprograms, protected types, task types, child units (and public
units are different than private units). [At least I *think* this is how it
works; I didn't design or write this code so I'm somewhat hazy on the
details.) For instance, for packages, there is a pointer that separates the
private declarations from the public ones. Obviously, if there are
additional parts, there are going to be more "declaration ranges" as well,
and that will make additional complications.

There are currently about 96 references to the "private_list" pointer in
Janus/Ada; all of those would need to be looked at. Obviously, some are just
initializations, but many seem to have important effects. 96 things to look
at seems like a massive job.

> But I guess the universal rule in this area is: Your Mileage May Vary.

Right; it's usually not possible to guess the implementation effort for some
other compiler. (And it's not that easy even for your own!)

I'd be especially be wary of comparing implementation effort between two
ideas unless one is absurdly simple (like just a syntax change).

> One major problem I have with the "private type ..."
> suggestion is that you don't have any way to
> declare types that are not visible at all
> using that notation.  It is quite common that
> some of the types of components of a private
> type are not visible at all.  I also worry
> that the "private type ..." proposal is
> a completely different approach to visibility
> from a conceptual point of view, creating
> a dramatic shift in perception of how things
> are done.

I agree with this, which is why I still think that the two part
instantiation (which matches up well with a two part type declaration) is
the way to go. The main problem I see is that the syntax choices we looked
at did not really properly convey what is going on (and that is important in
order to make the use intuitive). [I realize that there were semantic
problems, but I still think those could be worked out by limiting the
functionality of the "private" instantiation -- we never really tried that
seriously to see, as Pascal removed it from the Amendment agenda at that
point.]

****************************************************************

From: Robert I. Eachus
Date: Monday, October 1, 2007  6:59 PM

>I still think this is a bad idea; I still think that two part instantiations
>can work if sufficiently limited (we never really finished working on that
>idea - it was only the broadly defined version that didn't work). Ed's idea
>also would work (although it has the issue of exposing the implementation in
>practice even if not semantically).
  

I think that the end private (with or without semicolon) is conceptually 
the easiest for users to master.  I do think though that a restricted 
version of that proposal would be a lot easier to both implement and 
understand.  First yes, it would be possible to have multiple private 
parts, but is that game really worth the candle?  If you have ONE 
private part, then private types have to be declared before the private 
part, along with deferred constants and so on.  If we hang the 
elaboration of child units then the package body right at the end of the 
private part--or the package declaration if there isn't one--then 
instantiations of children there are fine.  As for making private 
details visible, no reason it should.  What you care about is that 
private types, and for that matter subprograms in the (first) public 
part are now elaborated.

Would there be some confusion from the fact that declarations in the 
second public part would not be visible in the body?  Not that much.  
Would it be a problem?  I'd have to do some experimentation, but I don't 
see it as a problem.  You might have to put some declarations in a child 
package then use renaming in the second public part to make them visible 
as part of the main package API.  I have to think that in most cases 
that will be what you want to do anyway.

Now to explain some magic handwaving above.  How do you tell that a 
reference in the second private part is to a child package?  Either a 
with clause, a use clause, or both in the second public part.  Allowing 
with clauses there is much more like the two part package declarations, 
but a bit easier to maintain.  I'd like to limit it to use clauses for 
child units for methodological reasons, but such units could have their 
own with clauses for anything, so that would not limit much in reality.

****************************************************************

From: John Barnes
Date: Monday, October 1, 2007  1:44 AM

Of all the suggestions, I like the end private idea. In fact I sent the
following a few days ago but it seemed to go down a black hole. I said in a
reply to Tuck:

>    A) Allow the notion of "end private;" where the private
>       part ends before the end of the package spec, and then
>       you can return to visible declarations.

I suggested that to Jean in a letter in 1983 or thereabouts but he didn't
reply! I always thought that the Ada all-in-one-lump approach to visibility,
although elegant, was not flexible enough in practice. Another case of
needing truth before beauty.

PS And of course dammit - you need a semicolon!

****************************************************************

From: Jean-Pierre Rosen
Date: Monday, October 1, 2007  3:16 AM

> A smaller syntactic change would be to have both views of the type in a 
> single declaration:
> 
> private type Tree is record ...

Except of course that it would be ambiguous:
    private
        type Tree is ...

The more I think about the "end private" idea, the more uncomfortable I 
feel. I'm afraid there is a huge can of worms sleeping under it...

For example:
Child units currently see *the* private part. Would they see all of them?

If I put a "use" clause in a private part, does it extend into the 
following non-private parts?

How would the "private with" work? It should apply to all private parts, 
therefore we'll have visibility coming in and out...

****************************************************************

From: Pascal Leroy
Date: Tuesday, October 2, 2007  8:00 PM

> Bob makes the point that the "end private" idea is more 
> powerful, in that you can use items from the instantiation to 
> visibly complete incomplete types declared before the 
> instantiation.  When trying to create mutually recursive 
> types, that seems pretty important.
> 
>      package Trees is
>          type Tree is private
>          ...
>        private
>          ...
>        end private
>          ...
>      end Trees;

My first reaction is that this reeks of C++.

This is one of the (many) things that I hate in C++: the declaration of a
class can include any number of public/protected/private sections in any
order.  True, this is more flexible than the Ada model, but then it sooo
easy to misuse.  In Ada, as soon as you see the word private, you know that
you are done with the exported declarations.  In C++ you have to read the
entire class declaration, keeping track of all the visibility changes, just
to find out which declarations are exported.  It is extremely easy to miss
one of the little words public/protected/private, and be just totally
confused about what a class does.

At any rate, it would be good to explain what exactly is visible after "end
private" and how you ensure that no private information is leaked out of the
package.

****************************************************************

From: Pascal Leroy
Date: Tuesday, October 2, 2007  8:01 AM

> I still think this is a bad idea; I still think that two part 
> instantiations can work if sufficiently limited (we never 
> really finished working on that idea - it was only the 
> broadly defined version that didn't work).

I agree with this.  I always thought that we were on the right track, but we
just didn't have enough time to find a model that would avoid all these
Baird pathologies.

****************************************************************

From: Robert A. Duff
Date: Tuesday, October 2, 2007  8:06 PM

> My first reaction is that this reeks of C++.

...says the guy who is leaving the Ada world for greener pastures.  ;-) ;-)

And if the C++ standard says "the sky is blue", should we automatically dispute
that?  ;-)

Note smileys -- I'm just teasing you, Pascal.

> This is one of the (many) things that I hate in C++: the declaration of a
> class can include any number of public/protected/private sections in any
> order.  True, this is more flexible than the Ada model, but then it sooo
> easy to misuse.  In Ada, as soon as you see the word private, you know that
> you are done with the exported declarations.  In C++ you have to read the
> entire class declaration, keeping track of all the visibility changes, just
> to find out which declarations are exported.  It is extremely easy to miss
> one of the little words public/protected/private, and be just totally
> confused about what a class does.

If it's properly indented, I don't find it hard to see the private/public
structure.

> At any rate, it would be good to explain what exactly is visible after "end
> private" and how you ensure that no private information is leaked out of the
> package.

Indeed.  Demons lurk, here, in the details.

****************************************************************

From: Randy Brukardt
Date: Wednesday, October 24, 2007  11:32 PM

...
> At any rate, it would be good to explain what exactly is visible after "end
> private" and how you ensure that no private information is leaked out of the
> package.

Let's just look at a few cases that would need to be defined:

* What if is there is a use clause in the private part? Does it extend into the
  following non-private part(s)? [This was from Jean-Pierre.] I think the answer
  has to be no, but that surely complicates things (and implementations).

* How about "additional operations" of a type? That is, in something like:

    package P is
       type Q is limited private;
    private
       type Q is record
          A : Integer := 10;
       end record;
    end private;
       Obj : Q;
       B : Boolean := (Obj = Obj);
    end P;

   Again, probably not. More generally, declarations in a private part should not be
   visible in a following visible part, so we'd have visibility going out at "end private;"
   and then returning when we reached the body.

    with Func;
    package P is
       type Q is limited private;
    private
       type Q is access Integer;
       Obj : Q;
    end private;
       I : Integer := Obj.all;
    end P;

   This reduced visibility worries me, as we generally don't lose visibility in a single
   package: everything is additive (visible part, private part, body). It seems to offer
   the opportunity for more anomalies.

* Do public children see all of the visible declarations but none of the private ones
  of their parent? (Seems necessary.)

I'm sure there are more issues to consider. I haven't seen any posting by Steve Baird
on this yet. ;-) The whole thing sounds like fun. :-(

****************************************************************

From: Tucker Taft
Date: Thursday, October 25, 2007  7:25 AM

Randy Brukardt wrote:
> Pascal notes:
> 
> ...
>> At any rate, it would be good to explain what exactly is visible after
> "end
>> private" and how you ensure that no private information is leaked out of
> the
>> package.
> 
> Let's just look at a few cases that would need to be defined:
> 
> * What if is there is a use clause in the private part? Does it extend into
> the following non-private part(s)? [This was from Jean-Pierre.] I think the
> answer has to be no,  ...

Agreed, clearly no.

> 
> * How about "additional operations" of a type? That is, in something like:
> 
>     package P is
>        type Q is limited private;
>     private
>        type Q is record
>           A : Integer := 10;
>        end record;
>     end private;
>        Obj : Q;
>        B : Boolean := (Obj = Obj);
>     end P;
> 
>    Again, probably not. More generally, declarations in a private part
> should not be visible in a following visible part, so we'd have visibility
> going out at "end private;" and then returning when we reached the body.

Agreed.

> 
>     with Func;
>     package P is
>        type Q is limited private;
>     private
>        type Q is access Integer;
>        Obj : Q;
>     end private;
>        I : Integer := Obj.all;
>     end P;
> 
>    This reduced visibility worries me, as we generally don't lose visibility
> in a single package: everything is additive (visible part, private part,
> body). It seems to offer the opportunity for more anomalies.

Perhaps, though I think nested packages with private
parts are pretty similar.  With nested packages,
visibility comes and goes.

Child packages are perhaps even closer to this.  When
you start a child package, although they logically "follow"
the private part of their ancestor packages,
they don't "see" the private part.  When you hit
the private part of the child package, suddenly
the private declarations of all the ancestors become
visible.

So from an implementation and anomaly point of view,
at the "end private;" you might imagine you are starting a
child package, which cannot take advantage of visibility
on the parent's private part until it hits its own private
part or body.

> * Do public children see all of the visible declarations but none of the
> private ones of their parent? (Seems necessary.)

They don't see the private declarations until they hit their
own private part or body.  No real surprise here.

> 
> I'm sure there are more issues to consider. I haven't seen any posting by
> Steve Baird on this yet. ;-)The whole thing sounds like fun. :-(

Clearly there will be some implementation effects, but I
don't see many semantic anomalies on the visibility side,
since I believe a child package experiences much the same
"coming and going" of visibility.

The anomalies I could imagine would have to do with "completion"
and overriding, since these kinds of things can't be
simulated by a child package (though a child subprogram
can "simulate" overriding in certain non-tagged cases).
And I agree we will need to work them through if we decide
to pursue this proposal. I am happy to do so, after
I have finished my other homework ... ;-).

****************************************************************

From: Robert A. Duff
Date: Thursday, October 25, 2007  9:15 AM

> Let's just look at a few cases that would need to be defined:
>...

I'm surprised by Tucker's answer.  My answer is that "obviously", the
visibility in the second visible part includes everything that came before,
just as if "end private;" were erased.  (Now Tuck can tell me why that's
stupid.  ;-))

The purpose of privacy is to hide things from OTHER packages, not from the
package itself.  This should be legal:

    package P is
    private
        type Base_Type is range 1..2**31-1;
    end private
        subtype Exported_Type is Base_Type'Base;
    end P;

to get a type that has at least 32 bits, but matches the hardware bounds.  I
don't want to pester the client with a useless name -- Base_Type is visible in
the visible part, but not in clients.

By the way, C++ has a similar feature.  It might help to look at the details,
which I don't remember.  Of course C++ visibility is quite different from Ada,
so it might be irrelevant.  And C++ visibility is broken in some ways, so we
don't necessarily want to mimic it -- but it couldn't hurt to look.

> * Do public children see all of the visible declarations but none of the
> private ones of their parent? (Seems necessary.)

Yes.

> I'm sure there are more issues to consider. I haven't seen any posting by
> Steve Baird on this yet. ;-)The whole thing sounds like fun. :-(

Indeed.  I'm not at all sure the whole idea will work.

****************************************************************

From: Jean-Pierre Rosen
Date: Thursday, October 25, 2007  7:34 AM

> Let's just look at a few cases that would need to be defined:
> 
And:

I assumed when I read the proposal that there could be several private 
parts, but I didn't see any mention about that. Is it the intent? That 
would certainly seem logical from a user point of view:

package Pack is
   -- visible
private
    -- hidden
end private;

   -- visible again

private
   -- hidden again
end Pack;

Have great fun with visibility going in and out!

****************************************************************

From: Randy Brukardt
Date: Thursday, October 25, 2007  7:56 PM

...
> >     with Func;
> >     package P is
> >        type Q is limited private;
> >     private
> >        type Q is access Integer;
> >        Obj : Q;
> >     end private;
> >        I : Integer := Obj.all;
> >     end P;
> >
> >    This reduced visibility worries me, as we generally don't
> lose visibility
> > in a single package: everything is additive (visible part, private part,
> > body). It seems to offer the opportunity for more anomalies.
>
> Perhaps, though I think nested packages with private
> parts are pretty similar.  With nested packages,
> visibility comes and goes.
>
> Child packages are perhaps even closer to this.  When
> you start a child package, although they logically "follow"
> the private part of their ancestor packages,
> they don't "see" the private part.  When you hit
> the private part of the child package, suddenly
> the private declarations of all the ancestors become
> visible.

Not really: within a single unit, all that can happen is the *addition* of
new things. Once you can see the additional operations, they stay visible
until the end of the unit. (The situation in outer units may be different,
but I don't see that as relevant -- we know that visibility can change in
external units, but it doesn't reduce *directly within* a unit.)

> So from an implementation and anomaly point of view,
> at the "end private;" you might imagine you are starting a
> child package, which cannot take advantage of visibility
> on the parent's private part until it hits its own private
> part or body.

A child package starts with a "clean" visibility slate, into which the
parent and other "withs" are loaded. Handling the private part at that point
is (relatively) easy. A "end private" has a complex visibility state,
already existing, where things would have to be subtracted piece-meal.
(Reloading is not an option: you haven't stored it out yet!) This seems
sustantially more complex. (Of course, I haven't tried implementing this, so
it might prove easier than I think now.)

> > * Do public children see all of the visible declarations but none of the
> > private ones of their parent? (Seems necessary.)
>
> They don't see the private declarations until they hit their
> own private part or body.  No real surprise here.
>
> > I'm sure there are more issues to consider. I haven't seen any posting
by
> > Steve Baird on this yet. ;-)The whole thing sounds like fun. :-(
>
> Clearly there will be some implementation effects, but I
> don't see many semantic anomalies on the visibility side,
> since I believe a child package experiences much the same
> "coming and going" of visibility.

No, only "coming". There's no "going": once the private part of a parent is
visible, it stays that way. It's the "going", for declarations in the same
unit, which I think doesn't exist currently and has the potential to add
complications.

Now, I realize that other models of child compilations are possible, but for
us, each one starts with a clean slate which is built up with whatever is
needed (a parent is just a hidden "with" of a unit). The only magic is
making the private part visible when the child "private" is reached, and
that is a pure addition to visibility.

> The anomalies I could imagine would have to do with "completion"
> and overriding, since these kinds of things can't be
> simulated by a child package (though a child subprogram
> can "simulate" overriding in certain non-tagged cases).
> And I agree we will need to work them through if we decide
> to pursue this proposal. I am happy to do so, after
> I have finished my other homework ... ;-).

Sounds good, especially the part about finishing your other homework. :-)

This issue and the possible solutions are on the agenda for Washington,
although I'd expect any brainstorming on it to come after we get our other
work done. (You'll note it is right on the bottom of the list... ;-)

BTW, you are welcome to give the same treatment to the write-up of my
proposal for this problem that you'll find in AI05-0074-1 (another AI number
that I fear will live in infamy, like AI-217 [limited with] and AI-359 [the
4(!) previous versions of this problem]). I hope I can take it as well as I
dish it out. ;-)

****************************************************************

From: Tucker Taft
Date: Thursday, October 25, 2007 11:13 PM

> I'm surprised by Tucker's answer.  My answer is that "obviously", the
> visibility in the second visible part includes everything that came before,
> just as if "end private;" were erased.  (Now Tuck can tell me why that's
> stupid.  ;-))

I would think that would be a recipe for privacy "leakage."
If you, for example, declare a subtype of a private
type in a visible part that follows the private part
that defined its full type, is the subtype a partial
view or a full view of the type?  E.g.:

    type P is private;
   private
    type P is new Integer;
   end private;
    subtype S is P;

What are the properties of subtype S?  I
really think it needs to still be private.

I think from a *visibility* point of view,
it should be as though the private declarations
were not there at all.  The only effect of
the private declarations should be to complete
certain earlier declarations, so they can be used
in contexts where a completely defined
entity is required.

> The purpose of privacy is to hide things from OTHER packages, not from the
> package itself.  

If that were true, then why wouldn't the visible part of
a child package have visibility on the private part of
its parent?  It comes physically after it, it is inside
the declarative region of the parent, but it doesn't
have visibility on the private declarations until you
reach its own private part.  One reason is to avoid
privacy leakage.  The visible part of a child package
*can* take advantage of the fact that the private types
of the parent have been completely defined, so they
can be used in contexts where a completely defined
private type is required.

Basically, I think the visibility on the parent from
the visible part of a child is the best model for
the visibility after "end private;".

> ...  This should be legal:
> 
>     package P is
>     private
>         type Base_Type is range 1..2**31-1;
>     end private
>         subtype Exported_Type is Base_Type'Base;
>     end P;
> 
> to get a type that has at least 32 bits, but matches the hardware bounds.  I
> don't want to pester the client with a useless name -- Base_Type is visible in
> the visible part, but not in clients.

I can see how that might be useful, but it would just
introduce too many anomalies, I suspect.

> By the way, C++ has a similar feature.  It might help to look at the details,
> which I don't remember.  Of course C++ visibility is quite different from Ada,
> so it might be irrelevant.  And C++ visibility is broken in some ways, so we
> don't necessarily want to mimic it -- but it couldn't hurt to look.

The basic C++ "protection" model is quite different from
Ada's.  "Private" declarations are just as "visible" as
"public" declarations, but you get an error if you try
to use them in a place that isn't supposed to have access
to private declarations.  So this doesn't really address
the issue, because the question simply becomes whether
you are allowed to "use" (as opposed to "see") the
declarations from a private part in a following visible
part.

The other big difference in C++ is that you don't
have "two part" type declarations in C++, in the
way you do in Ada for private types.

****************************************************************

From: Robert A. Duff
Date: Friday, October 26, 2007  9:01 AM

...
> What are the properties of subtype S?  I
> really think it needs to still be private.

You may well be right, but I'd like to understand why.  What if S is an integer
type?  Would that cause the sky to fall?

Or, what if S is an integer type within this package, but when viewed from
outside, it is private?  There's a "+" implicitly declared in the private part;
what if it's visible in the second visible part, but not visible in clients?

...
> > The purpose of privacy is to hide things from OTHER packages, not from the
> > package itself.
> 
> If that were true, then why wouldn't the visible part of
> a child package have visibility on the private part of
> its parent?

I must admit, I've never quite understood the high-level rationale for that.
If the so-called "leakage" is explicit in the program text, what's the harm?
I suppose the devil's in the details...

...

> I can see how that might be useful, but it would just
> introduce too many anomalies, I suspect.

Quite likely true.  But the opposite approach seems to have some difficulties,
too.

> > By the way, C++ has a similar feature.  It might help to look at the details,
> > which I don't remember.  Of course C++ visibility is quite different from Ada,
> > so it might be irrelevant.  And C++ visibility is broken in some ways, so we
> > don't necessarily want to mimic it -- but it couldn't hurt to look.
> 
> The basic C++ "protection" model is quite different from
> Ada's.  "Private" declarations are just as "visible" as
> "public" declarations, but you get an error if you try
> to use them in a place that isn't supposed to have access
> to private declarations.

Ah, yes, thanks for the reminder.  I (now) agree that the C++ rules are
irrelevant to this discussion.

****************************************************************

From: Tucker Taft
Date: Friday, October 26, 2007  11:56 AM

>> What are the properties of subtype S?  I
>> really think it needs to still be private.
> 
> You may well be right, but I'd like to understand why.  What if S is an integer
> type?  Would that cause the sky to fall?

No, but it changes the rules significantly.  In many
places in the reference manual, we indicate that a private
part of a language-defined package is "not specified."
It is nice to be able to do so (not just for the language
designers, but for anyone specifying an abstraction
via a package spec).  I would hope that this property
of packages remains true in general, namely you can look
just at the visible part(s) of a package and make
sense of it without having to look at the private
part at all.  Furthermore, a maintainer can change
the private part of the package and not have to
worry about the semantic effects on clients of
the package.
 
> Or, what if S is an integer type within this package, but when viewed from
> outside, it is private?  There's a "+" implicitly declared in the private part;
> what if it's visible in the second visible part, but not visible in clients?

Again, that means you can't really understand what is going
on in the visible part without studying the private part.

> ...
>>> The purpose of privacy is to hide things from OTHER packages, not from the
>>> package itself.
>> If that were true, then why wouldn't the visible part of
>> a child package have visibility on the private part of
>> its parent?
> 
> I must admit, I've never quite understood the high-level rationale for that.
> If the so-called "leakage" is explicit in the program text, what's the harm?
> I suppose the devil's in the details...

I believe the rationale is as above, namely that you
don't need to study the private part of the parent
to understand the visible parts of the child packages.

Interestingly, this point gives a rationale for only
allowing one private part in a package.  If you
allow more than one, then to understand what is
going on in the visible parts you need to know
which private types are completed in which private
part.  I don't think that is a feature.  In these
discussions, I have harborred a personal dislike for the
idea of multiple private parts in a single package,
but I haven't had a good reason other than aesthetic.
But this to me seems like a good reason.  If you
write a package spec and leave the private part(s)
unspecified, then you don't want to have to specify
which private part completes what type (or
deferred constant).

So I would go further and say that the
"end private;" proposal should only allow one private
part, that in the "post-private" visible
part you can't introduce any more private types
or deferred constants, and that all declarations
prior to the "end private;" are frozen at the
point of the "end private;", except for incomplete
types.

With this approach, you have no doubt what has
to happen in *the* private part (the completion
of all partial views and deferred constants, plus
any relevant rep-clauses), and you know that
everything is frozen (except incomplete types)
at the "end private;", whether or not there is
a freezing occurrence that happens to appear
in the private part.

Aesthetically, this also feels better.  The
private part can be "sandwiched" between a
pre-private and a post-private visible part,
but you don't have to worry about or deal
with private parts "sprinkled" throughout
the package spec at random points.

Finally, the nature of the pre-private and
the post-private visible parts are pretty
different, since the private types
and deferred constants need to precede
the private part, and the instantiations
and incomplete type completions can follow
it.

There can be some advantages to having
a "standard" usage model implied by the
constructs of the language, rather than
having so many different equally good ways
of solving the same problem that no standard style
emerges, and programming styles within
the same language vary dramatically.

One last point.  One reason why I like the
notion of a "post-private" visible part
to handle instantiations is that it is
common for instantiations to be followed by
one or more derived type declarations,
bringing some of the types defined by
the instance out into the surrounding
scope.  E.g.:

     package T_Vectors is
       new Containers.Vectors(T, ...);
     type T_Vector is new T_Vectors.Container
       with null record;

The proposal of an "internally limited" instantiation
that provides internally only a "limited view" where
all exported types are incomplete, wouldn't
support this standard paradigm (since you can't
derive from an incomplete type).  That could
be a significant downside of that proposal,
in my view.

The post-private visible part can easily accommodate
this approach.  If necessary, an incomplete
type declaration for T_Vector could precede it
in the pre-private visible part (or in the
private part).  E.g.:

   package P is
     type T is private;

     type T_Vector is tagged;
     type T_Vec_Ptr is access T_Vector;

     function Inner_List(X : T) return T_Vec_Ptr;
     procedure Set_Inner_List(
       X : in out T; List : in T_Vec_Ptr);

     ...

    private
       type T is record
          Inner_List : T_Vec_Ptr;
          ...
       end record;
       ...  -- rep-clauses for T, deferred constants, etc.
    end private;

     package T_Vectors is ...
     type T_Vector is new T_Vectors.Container
       with null record;

   end P;

This feels pretty natural to me, with the post-private
part doing a straightforward and useful job.

It is interesting that when you do a "limited with" on
a package you don't see any nested instances, but you
do see derived type declarations.  So using this
paradigm of deriving from a type declared in a nested
instance has the useful effect of making the type
available (as an incomplete type) via a "limited with."
The "internally limited" instantiation will always
be completely invisible via a "limited with."

Finally, because there is no "leakage" from the private part,
we could reasonably write:

  package Ada.Neat_Standard_Package is

    type Container is private;
    ...

    private
      -- not specified by the language
    end private;

    package Useful_Instance is
      new Great_Generic(Container, ...)
     ...

  end Ada.Neat_Standard_Package;

in a specification of some future language-defined
package, where we wanted to include some visible
instantiations (such as signatures), and there
would be no danger in saying the private part is
"not specified" since there is no doubt as to what
would have to happen in the private part.

Note also that the post-private part acts somewhat
like the visible part of an "anonymous" child
that you want all clients to "with" and "use."
Semantically I think it can follow many
of the same rules as a child visible part, and
could even be implemented in a somewhat
similar fashion, by starting from an empty
slate, and then "loading up" the visibility
stack with the with and use clauses, the pre-private
visible part, etc.  This would avoid having
to remove declarations from the visibility
stack, which I could imagine would be a burden
for some implementations.

****************************************************************

From: Robert A. Duff
Date: Friday, October 26, 2007  1:48 PM

> Interestingly, this point gives a rationale for only
> allowing one private part in a package.  If you
> allow more than one, then to understand what is
> going on in the visible parts you need to know
> which private types are completed in which private
> part.  I don't think that is a feature.

OK, that makes sense.

>...In these
> discussions, I have harborred a personal dislike for the
> idea of multiple private parts in a single package,
> but I haven't had a good reason other than aesthetic.
> But this to me seems like a good reason.

Agreed.  I find vague aesthetics much more convincing when backed up by good
reasons!

Of course, allowing just one _visible_ part would be a further
simplification.  ;-)

****************************************************************

From: Tucker Taft
Date: Friday, October 26, 2007  2:21 PM

> Of course, allowing just one _visible_ part would be a further
> simplification.  ;-)

Uhhhh, true, but how does that solve our original problem
with instantiations using private types?

****************************************************************

From: Robert A. Duff
Date: Friday, October 26, 2007  2:47 PM

It doesn't.  Hence the smiley.  I was just attempting to wryly point out that
this idea might not fly at all (and maybe in fact the whole problem can't be
solved reasonably).  Sorry for being unclear.  Anyway, I guess we have several
years to think about it...

****************************************************************

*** End of discussion of "end private" idea ***

****************************************************************

From: Tucker Taft
Date: Friday, October 19, 2007 11:16 PM

Randy and others indicated that they thought we came
close on allowing for instantiation with private types
by having a special kind of "partial" instantiation
preceding the "full" instantiation.

Here are some recent thoughts I had on this.  They may be
redundant or overlapping with our earlier discussions,
but I'm not sure.  The basic idea is that we have
a "partial" instantiation, where we instantiate only
the spec of the generic, and have no special pre-freezing
of the actuals.  Then we have the "full" instantiation
where the actuals are frozen, and we instantiate
the body.  For example:

    type Priv is private;

    package Inst is new Gen(Priv) with private;
       -- "with private" indicates that
       -- private types are permitted, and no
       -- automatic freezing of actuals occurs.
       -- Only the *spec* of the generic Gen
       -- is instantiated at this point.
       -- (Of course some other syntax is possible.)

  private

    type Priv is record
        X : Inst.Something;
    end record;

    package Inst is new Gen(Priv);
       -- at this point, we instantiate the
       -- body of Gen, and do the usual pre-freezing
       -- of the actual parameters.


This two-phase instantiation of a generic package is
almost identical to a hand expansion as a nested package
spec, followed by a later hand expansion for the
corresponding body.  This means that pretty much anything
you can do with a nested package you can do with a
generic.

An important caveat: this means that you have instantiation-
time checks that may reveal certain aspects of the private
part of the generic.  For example, if in the private
part of the generic there is an object declaration
of the formal type, then that would freeze the actual type
at that point.  In addition, type circularity checks
could reveal whether the formal type is used directly or
only with a level of indirection in a type declared in
the generic (in the example above, we are assuming that
type "Something" in the generic does not include a
subcomponent of the formal type).

These instantiation-time checks don't seem significantly
worse than the ones we already have (every place where
we say a given rule "applies also in the private part
of an instance of a generic unit.").  We have somewhat
gotten used to the fact that the private part of a generic
forms part of its contract.

The main thing I like about this approach is that it
really doesn't involve a bunch of special rules, such
as making types automatically incomplete, etc.  It
just separates the instantiation of the spec from the
body, and associates the "pre-freezing" of actuals with
the instantiation of the body.

I am almost certainly missing some subtlety, but at
the moment it feels pretty straightforward.

****************************************************************

From: Tucker Taft
Date: Friday, October 19, 2007 11:41 PM

I should add, that if there are nested instantiations
inside the generic, then during the "partial"
instantiation, *probably* only spec instantiations should
take place, with the body instantiations being
delayed until the enclosing generic's body is
instantiated.  I knew I'd find at least one
subtlety...

****************************************************************

From: Randy Brukardt
Date: Friday, October 19, 2007 11:58 PM

> Randy and others indicated that they thought we came
> close on allowing for instantiation with private types
> by having a special kind of "partial" instantiation
> preceding the "full" instantiation.
>
> Here are some recent thoughts I had on this.  They may be
> redundant or overlapping with our earlier discussions,
> but I'm not sure.

I recommend going back and reading the minutes on the various versions of
AI-359 in order to see the problems.

> The basic idea is that we have
> a "partial" instantiation, where we instantiate only
> the spec of the generic, and have no special pre-freezing
> of the actuals.  Then we have the "full" instantiation
> where the actuals are frozen, and we instantiate
> the body.

...
> The main thing I like about this approach is that it
> really doesn't involve a bunch of special rules, such
> as making types automatically incomplete, etc.  It
> just separates the instantiation of the spec from the
> body, and associates the "pre-freezing" of actuals with
> the instantiation of the body.
>
> I am almost certainly missing some subtlety, but at
> the moment it feels pretty straightforward.

I think that there were problems with elaboration of the package
specification. You can't actually do anything that would depend on the
private type (because you don't yet know its declaration), so what you could
put in the specification would be quite limited.

For Janus/Ada, we generate thunks at the point of the instantiation which
are called by the elaboration. Those obviously assume that the full type
definition is available. (We share the elaboration part of the specification
in the same way that we share the body; if we were able to do real template
expansion, we probably would have used that everywhere...)

The privacy breaking is uncomfortable, mainly because it could easily mean
that generics like the containers may not be usable in specifications
depending on their implementation. That suggests that we'd have to add a
boatload of restrictions on how the private part is implemented (not
allowing instantiation of the containers with a private type would be a
non-starter, given that it is one of the problems that we need to fix). And
I have to wonder what it would do to the IBM implementation.

The solution still seems to me to be a limited instantiation that exports a
limited view of the generic within the specification (but exports the full
view of the generic to clients of the package). The problem we had in
Atlanta was trying to export too much to the package; that led to all kinds
of anomolies. Exporting nothing would work great, but wouldn't solve Bob's
recursion problem. My main concern is that I never came up with a syntax
that really explained what is happening. Maybe something like:

    package Inst is new Gen(Priv) for export;

which makes it clear that Inst is only for export and for only limited local
use.

****************************************************************

From: Randy Brukardt
Date: Saturday, October 20, 2007 12:00 AM

> I should add, that if there are nested instantiations
> inside the generic, then during the "partial"
> instantiation, *probably* only spec instantiations should
> take place, with the body instantiations being
> delayed until the enclosing generic's body is
> instantiated.  I knew I'd find at least one
> subtlety...

Mr. Code sharing is about to throw up. :-)

****************************************************************

From: Randy Brukardt
Date: Saturday, October 20, 2007 12:07 AM

...
> My main concern is that I never came up with a
> syntax that really explained what is happening. Maybe something like:
>
>     package Inst is new Gen(Priv) for export;
>
> which makes it clear that Inst is only for export and for only
> limited local use.

Indeed, this suggests that the partial instantiation is not visible at all
inside the package specification (until it is completed, of course).

If we want a limited view, too, we could add syntax for that:

     package Inst is new Gen(Priv) for export with limited;

There can be no anomolies with the first form, because it has no semantic
effect on the package: it simply makes a declaration in the private part
visible from the outside of the package.

The second form adds a limited view so that recursive data types can be
created. I'd expect that to work roughly the same as regular limited view,
so I think we've already covered the anomolies. (Famous last words... ;-) I
realize this construct is a bit more limiting for Bob than some of the other
proposals, but at least with it you can write the sort of things he wants to
(as opposed to the current situation where it is nearly impossible).

****************************************************************

From: Tucker Taft
Date: Saturday, October 20, 2007 10:58 AM

> I think that there were problems with elaboration of the package
> specification. You can't actually do anything that would depend on the
> private type (because you don't yet know its declaration), so what you could
> put in the specification would be quite limited.

Can you elaborate on this?  When you say "you can't actually do
anything" are you talking about the compiler or the
programmer?  How is this different from a nested package at
the point of the partial instantiation, that refers to a
private type from the enclosing package?  It is true that in
both cases, you can't lay out any types declared in the nested
package that depend on the outer private types, but there is
no requirement to do so until you hit a freezing point.  By
postponing the instantiation of the body, we hopefully postpone
the freezing points.

> For Janus/Ada, we generate thunks at the point of the instantiation which
> are called by the elaboration. Those obviously assume that the full type
> definition is available. (We share the elaboration part of the specification
> in the same way that we share the body; if we were able to do real template
> expansion, we probably would have used that everywhere...)

I think you clearly would need to postpone generating some of the thunks,
and separate the elaboration into two routines, one for the spec
which would use only the thunks it really needed for the elaboration
of the spec, and one for the body, which would use the remaining thunks.

> 
> The privacy breaking is uncomfortable, mainly because it could easily mean
> that generics like the containers may not be usable in specifications
> depending on their implementation. That suggests that we'd have to add a
> boatload of restrictions on how the private part is implemented (not
> allowing instantiation of the containers with a private type would be a
> non-starter, given that it is one of the problems that we need to fix). And
> I have to wonder what it would do to the IBM implementation.

I agree we might have to specify additional implementation requirements.
The most important would be that the generic must support partial
instantiation.

Secondarily, we would have to decide whether we want
to require that certain exported types not include the formal
types as subcomponents.  My sense would be that we probably *don't*
want to impose this latter requirement on the "definite" versions,
so that "bounded" versions can be created that use no levels of
indirection.  On the other hand, imposing this requirement on
the "indefinite" versions would seem reasonable, and actually
makes the "indefinite" versions that much more flexible.

I suppose this brings up another possibility, where we only
defer freezing an actual type if the corresponding formal type
has unknown discriminants.  Since we know that a formal type
with unknown discriminants could *never* be a subcomponent of
an exported "definite" type, we eliminate the privacy breaking.
That has a nice ring to it...

I can even imagine we consider eliminating the need for two-part
instantiations, and make them "implicit" when there is a formal
type with unknown discriminants.  That is, if a formal type
has unknown discriminants, then the freezing of the corresponding
actual type, and the body instantiation, is postponed
until the next "general" freezing point.  That has some
upward compatibility issues, of course, but they might be
acceptable in trade for avoiding explicit two-part
instantiations.

****************************************************************

From: Tucker Taft
Date: Saturday, October 20, 2007 11:01 AM

Once you accept having two separate elaboration routines,
one for the spec, and one for the body, I don't see this
makes sharing more difficult.  Furthermore, if we
limit this capability to generics that have at least
one formal type having unknown discriminants (which is
related to what I suggested in my most recent response),
this reduces the distributed overhead.

****************************************************************

From: Robert I. Eachus
Date: Saturday, October 20, 2007 10:49 PM

>If we want a limited view, too, we could add syntax for that:
>
>     package Inst is new Gen(Priv) for export with limited;
>
>There can be no anomolies with the first form, because it has no semantic
>effect on the package: it simply makes a declaration in the private part
>visible from the outside of the package.
>
>The second form adds a limited view so that recursive data types can be
>created. I'd expect that to work roughly the same as regular limited view,
>so I think we've already covered the anomolies. (Famous last words... ;-) I
>realize this construct is a bit more limiting for Bob than some of the other
>proposals, but at least with it you can write the sort of things he wants to
>(as opposed to the current situation where it is nearly impossible).

With my Norm Cohen Halloween costume on, and tongue very firmly in 
cheek, might I suggest:

     package Inst is not private new Gen(Priv) with limited;

or if you prefer:

     not private package Inst is new Gen(Priv) with limited;

What about making the limited view the default?  Then you could say:

         not private package Inst is not limited new Gen(Priv);

Am I being silly here?  A bit.  But the first case seems to map almost 
directly to the multiple private part model.  As I see it, the problems 
there come from exporting types and declarations in the private part, by 
a renaming, derived type, or subtype in the generic package 
specification. In one sense we are trying to poke holes in the privacy 
screen, so it seems unfair to talk about unintended holes that a 
programmer can avoid.  On the other hand, the Ada philosophy is to make 
such things explicit.

So maybe we are looking in the wrong place.  We know what we want: to be 
able to export generic instamces which are not child packages but can 
see into the private part.  If we flag the generic for export in some 
way, it may be that we want to flag any otherwise private types, 
subprograms, or whatever that such a generic can make visible outside 
the private part.

Taking off the Norm Cohen costume, something like:

package Foo is


   type Bar is private;

   ...

private

   export type Bar is....;

   export type FooBar is Bar with...;

   type Foob is ...;

   export package FuBar is new ...;
   -- can have Bar or FooBar as generic formals, but not Foob;

end Foo;


Seems to me we avoid most of the potential issues this way.  Yes, you 
can use this mechanism to break privateness.  But then again, unless 
someone is holding a gun, or the software equivalent, to the 
programmer's head, there is no requirement to make any particular type 
private, or to even have any private types at all.

For the package FuBar, the export keyword (or whatever syntax is chosen) 
has  semantic consequences:  The visibility of the unit is increased.  
For non-generic objects, types, or subprograms, all the notation does is 
flag that they can be used in such quasi-public instantiations.  Or you 
could take the view that it also expands the visibility: into the 
generic_actual_part of an exported generic instantiation.

****************************************************************

From: Randy Brukardt
Date: Sunday, October 21, 2007 10:12 PM

Tuck, we already have two elaboration routines. That's not the point; the
point is that you can't elaborate the body at the point of the instance;
you'd have to define some later point that occurred implicitly. But that
idea was roundly disliked last time, and I can't imagine what's changed
about that.

Essentially, you are rehashing all of the ideas that we previously
discarded. They didn't work two years ago and I don't know of anything
that's changed since. For instance, there was a strong dislike for "implicit
elaboration" at some later point, because a small change in a generic spec
or in an instantiation can cause the semantics to change (and possibly
break).

Can we possibly go back to where we left off (which is what I was trying do)
and not go through all of these discarded ideas??

****************************************************************

From: Randy Brukardt
Date: Sunday, October 21, 2007 10:22 PM

> > I think that there were problems with elaboration of the package
> > specification. You can't actually do anything that would depend on the
> > private type (because you don't yet know its declaration), so what you
could
> > put in the specification would be quite limited.
>
> Can you elaborate on this?  When you say "you can't actually do
> anything" are you talking about the compiler or the
> programmer?

The programmer, mainly. There are a lot of restrictions on the usage of
private types before the full definition, and none of those can be allowed
in a generic that will be used in your scheme.

> How is this different from a nested package at
> the point of the partial instantiation, that refers to a
> private type from the enclosing package?

When you write a nested package, you do that while you design the rest of
the package, and you can avoid (well, you have to avoid) doing anything that
would be illegal with the private type.

But with a generic, you are talking about a package that was designed
separately, and may very well already exist. I don't think people are going
to want to redesign all of their generics to work in this context.

> It is true that in
> both cases, you can't lay out any types declared in the nested
> package that depend on the outer private types, but there is
> no requirement to do so until you hit a freezing point.  By
> postponing the instantiation of the body, we hopefully postpone
> the freezing points.

Piece-meal freezing of generic units was rejected last time -- too much of
an implementation earthquake.

> > For Janus/Ada, we generate thunks at the point of the instantiation which
> > are called by the elaboration. Those obviously assume that the full type
> > definition is available. (We share the elaboration part of the specification
> > in the same way that we share the body; if we were able to do real template
> > expansion, we probably would have used that everywhere...)
>
> I think you clearly would need to postpone generating some of the thunks,
> and separate the elaboration into two routines, one for the spec
> which would use only the thunks it really needed for the elaboration
> of the spec, and one for the body, which would use the remaining thunks.

There is only one generic descriptor, and I don't see any reason for more.
The problem really is that there would be no way to prevent "early" calls to
thunks that don't exist, so there would be many impossible-to-find bugs to
deal with. (Our generics already have that problem, and I surely don't want
to make it worse.)

Remember that you can't make any calls (including those for thunks) until
you've elaborated the generic. It's hard to imagine what you could do
between the partial instantiation and the full one. (Yes, nested packages
have this problem, too, Program_Error is raised by doing almost anything.
That makes them not very useful.)

> > The privacy breaking is uncomfortable, mainly because it could easily mean
> > that generics like the containers may not be usable in specifications
> > depending on their implementation. That suggests that we'd have to add a
> > boatload of restrictions on how the private part is implemented (not
> > allowing instantiation of the containers with a private type would be a
> > non-starter, given that it is one of the problems that we need to fix). And
> > I have to wonder what it would do to the IBM implementation.
>
> I agree we might have to specify additional implementation requirements.
> The most important would be that the generic must support partial
> instantiation.
>
> Secondarily, we would have to decide whether we want
> to require that certain exported types not include the formal
> types as subcomponents.  My sense would be that we probably *don't*
> want to impose this latter requirement on the "definite" versions,
> so that "bounded" versions can be created that use no levels of
> indirection.  On the other hand, imposing this requirement on
> the "indefinite" versions would seem reasonable, and actually
> makes the "indefinite" versions that much more flexible.

The scheme I was proposing needs none of this. The only thing that changes
is that we have a new kind of partial instantiation.

> I suppose this brings up another possibility, where we only
> defer freezing an actual type if the corresponding formal type
> has unknown discriminants.  Since we know that a formal type
> with unknown discriminants could *never* be a subcomponent of
> an exported "definite" type, we eliminate the privacy breaking.
> That has a nice ring to it...
>
> I can even imagine we consider eliminating the need for two-part
> instantiations, and make them "implicit" when there is a formal
> type with unknown discriminants.  That is, if a formal type
> has unknown discriminants, then the freezing of the corresponding
> actual type, and the body instantiation, is postponed
> until the next "general" freezing point.  That has some
> upward compatibility issues, of course, but they might be
> acceptable in trade for avoiding explicit two-part
> instantiations.

The implicit idea was roundly rejected in the past. Let's not keep rehashing
the old stuff.

****************************************************************

From: Tucker Taft
Date: Sunday, October 21, 2007 10:26 PM

The proposal I have been talking about
presumes there is a partial instantiation
(I suggested the "with private" syntax),
followed by a "full" instantiation.
The spec elaboration(s) happen at the
point of the partial instantiation,
and the body elaboration(s) happen at the
point of the full instantiation.

Sorry if that wasn't clear.

****************************************************************

From: Randy Brukardt
Date: Sunday, October 21, 2007 10:41 PM

It was clear, but then you started talking about implicit instantiations and
having different behavior for different kinds of formals. That certainly was
discredited before.

My recollection is that we talked about a model similar to the one you are
proposing now and rejected it for some reason. But I don't recall the
reason, and I'm too far behind on my work after the disk failure to spend
time looking it up. (So I may just be spreading FUD, I hope not.)

****************************************************************

From: Tucker Taft
Date: Sunday, October 21, 2007 11:03 PM

Sorry if it feels like I am dragging you through
the past.  Sometimes new ideas turn up when you
visit old ground again.  Of the various
musings I have uttered, the one that I think that
*is* new and might be worth further investigation
is as follows:

    1) Allow "partial" instantiations *only* if
       at least one of the formal types is
       a formal type with unknown discriminants.

    2) At the partial instantiation, freeze all
       actuals *except* the actuals corresponding
       to formals with unknown discriminants.
       Instantiate the spec, and only partially
       instantiate nested instantiations of
       generics with formals with unknown discrims.

    3) At the full instantiation, freeze the actuals
       corresponding to formals with unknown discrims,
       then instantiate the bodies of any nested
       instantiations only partially instantiated
       thus far, and then instantiate the body of
       the "main" generic.

This has the advantage that implementations need
support partial instantiations only for a subset
of generics, presuming there might be a higher distributed
overhead to support partial instantiations on all
generics.  It also has the advantage that all actual
types in a generic instance may still be presumed
to be "pre-frozen," *unless* the formal has unknown
discriminants.  Furthermore, because formals with
unknown discriminants cannot be used as components
of other types, it is safe to use any type exported
by the generic instance in the full definition of
a private type passed as the actual, since it can't
create a circularity.

****************************************************************

From: Robert A. Duff
Date: Monday, October 22, 2007  8:25 AM

>...  Furthermore, because formals with
> unknown discriminants cannot be used as components
> of other types, it is safe to use any type exported
> by the generic instance in the full definition of
> a private type passed as the actual, since it can't
> create a circularity.

Well, that seems to solve the problem, but it has some flaws.
I'm not sure they're fatal flaws.

It means I can create a Thing record containing a Vector of pointers to Things,
either by passing a pointer to Vectors or by passing Thing to
Indefinite_Vectors.  But I can't create a Thing containing a pointer to a
Vector of Things.  It's bad enough that we force the programmer to introduce a
level of indirection in order to have recursive types -- now we're telling them
which place has to have the indirection.

It's overkill -- you're forced to use a level of indirection even when there is
no cycle, or else do the old-fashioned thing (make the instantiation a child).
The usual: you can't do X because if you also did Y, then there would be
trouble -- it's pretty annoying if you don't want to do Y.

Maybe it's good enough.  OTOH, I suppose we have several years to search for
better solutions.  ;-)

****************************************************************

From: Tucker Taft
Date: Monday, October 22, 2007  9:00 AM

If you are willing to insert the level of indirection
explicitly using an access type, then you can create
a thing that contains a pointer to a vector
of things using an incomplete type:


    type Vector_Of_Things;

    type Thing is record
        Vec : access Vector_Of_Things;
    end record;

    package Thing_Vectors is new Vectors(Thing, ...);
    type Vector_Of_Things is new Thing_Vectors.Vector
      with null record;

Is that adequate?

****************************************************************

From: Robert A. Duff
Date: Monday, October 22, 2007  9:40 AM

I'm confused.  I thought we were talking about the case where Thing is private,
and Thing_Vectors is exported from the package.  So "new Vectors(Thing)" is
illegal, since the formal does not have unknown discrims.

****************************************************************

From: Tucker Taft
Date: Monday, October 22, 2007 10:19 AM

Clearly the above can be in the private part, with a visible
partial view of  "Thing" declared in the visible part.
But you are right that you can't also export Thing_Vectors
in the above, except via the hypothetical "end private;"
construct, which would immediately precede the instantiation.

So you are right that the partial instantiation approach
allows for indirection buried in the generic, whereas
the "end private;" allows for indirection in the user's private
type.  You need both to allow for both.

One could argue that it wouldn't be all bad for the language
to make a choice, favoring one over the other.  Or one
could argue that the language should support both equally.
Right now it supports neither, and if you want to create
a mutually recursive combination approximating type and
vector-of-type, then you are limited to type and vector
of ptr-to-type.

Randy is a big fan for burying access types inside abstractions,
and the partial instantiation idea seems to be the only
one so far that allows all uses of access types to be buried
inside the container abstractions.

The "end private;" seems more general, but it is roughly
equivalent to using a child.  The partial instantiation
provides a capability that cannot be reproduced using a child.

Seems like some tough choices.

****************************************************************

From: Randy Brukardt
Date: Monday, October 22, 2007  6:56 PM

> Sorry if it feels like I am dragging you through
> the past.  Sometimes new ideas turn up when you
> visit old ground again.  Of the various
> musings I have uttered, the one that I think that
> *is* new and might be worth further investigation
> is as follows:
>
>     1) Allow "partial" instantiations *only* if
>        at least one of the formal types is
>        a formal type with unknown discriminants.

Yes, this is new.

>     2) At the partial instantiation, freeze all
>        actuals *except* the actuals corresponding
>        to formals with unknown discriminants.
>        Instantiate the spec, and only partially
>        instantiate nested instantiations of
>        generics with formals with unknown discrims.

But this is a variation of ideas that we could not get to work in the past.
The problem was two-fold: implementation earthquakes for various models, and
privacy violations. It's actually three-fold for this one, because partially
instantiating nested instantiations means that where the body of those
nested instantiations are elaborated would differ depending on how the
instantiation is written. That seems bad: it's not uncommon to use an
instantiation to get a body elaborated in a spec (Ada provides no other way
to do that). So, given a generic like:

     generic
         type Foo (<>) is ...
     package Bar is
         package Nested is new <some generic> (Foo);
         subtype Bounds is Integer range 1 .. Nested.Func;
     end Bar;

A partial instantiation of Bar would raise Program_Error on the call of
Nested.Func (because the body is not elaborated) but would work correctly
for a normal instantiation. That's unpleasant.

>     3) At the full instantiation, freeze the actuals
>        corresponding to formals with unknown discrims,
>        then instantiate the bodies of any nested
>        instantiations only partially instantiated
>        thus far, and then instantiate the body of
>        the "main" generic.
>
> This has the advantage that implementations need
> support partial instantiations only for a subset
> of generics, presuming there might be a higher distributed
> overhead to support partial instantiations on all
> generics.

I don't think that this would be very useful in simplifying the
implementation, but of course I can't say how it would work for other
implementers.

> It also has the advantage that all actual
> types in a generic instance may still be presumed
> to be "pre-frozen," *unless* the formal has unknown
> discriminants.  Furthermore, because formals with
> unknown discriminants cannot be used as components
> of other types, it is safe to use any type exported
> by the generic instance in the full definition of
> a private type passed as the actual, since it can't
> create a circularity.

But the real problem is that you haven't addressed the anomalies that Steve
and Erhard pointed out the last time we discussed this (and that effectively
killed it). The problem was that a lot of partially defined stuff
potentially gets exported from a partial view, and this goes beyond the
sorts of things that we currently have to deal with.

My personal opinion is that the "partial instantiation" (versus a "limited
instantiation") is never going to work. I've been trying to make the
"limited instantiation" more acceptable, but I haven't gotten any real
feedback from you.

I realize that some people don't like the idea for some reason; what I'd
like to find out is what about the proposal is disliked so that it can be
enhanced. (I'm convinced that it would be best to start with a mechanism
that works and would not clobber implementers, and then see if we can safely
extend it to be more powerful.)

---

To summarize the latest idea again:

    exported_generic_instantiation ::=
        generic_instantiation [for export [with limited view]];

An exported_generic_instantiation shall be given in the visible part of a
package specification. An exported_generic_instantiation has to have a
completion in the private part of the same package specification. The
completion has to conform [I believe we have worked out rules for that in
the past].

The elaboration of an exported_generic_instantiation has no effect.

For a reference to the reference to the defining_program_unit_name of an
exported_generic_instantiation:

* If the reference is within the package specification that contains the
exported_generic_instantiation (and before the completing
generic_instantiation):
    If the reserved words "with limited" do not appear, then the reference
is illegal;
    If the reserved words "with limited" do appear, then the reference
denotes the limited view of the package created by expanding the generic
unit [Unfortunately, we can't just say the "limited view of the
instantiation", because that is empty. Better wording will be needed].

* Otherwise, the reference denotes the completing generic_instantiation.
(The fact that that instantiation is in the private part is ignored.)

---

This formulation has no elaboration or freezing issues, which were the major
stumbling blocks in the past attempts. (Uses outside the package are
allowed, because all of the elaboration and freezing has to already have
happened.) The only flaw that I can see is that it isn't as intuitive as it
could be, and I blame that mostly on the syntax and terminology. Perhaps
there is a way to make it more intuitive without bringing in all of the
problem areas.

Here's an example:

    package Something is
        type Priv is tagged private;
        package Priv_Lists is new Ada.Containers.Doubly_Linked_Lists (Priv)
for export;
        ... -- Use of Priv_Lists here is illegal.
    private
        type Priv is ...
        package Priv_Lists is new Ada.Containers.Doubly_Linked_Lists (Priv);
    end Something;

    with Something;
    procedure Do_It is
        PList : Something.Priv_Lists.Container; -- This is allowed.
    begin
        ...
    end Do_It;

And an example for Bob:

    package Something_Else is
        type Node is tagged private;
        package Node_Lists is new
            Ada.Containers.Doubly_Linked_Lists (Node) for export with
limited view;
        ...
    private
        type Node is tagged record
            Parent : access Node;
            Children : access Node_Lists.Container; -- "Recursive" definition.
            Siblings : access Node_Lists.Container;
            ...
        end record;
        package Node_Lists is new Ada.Containers.Doubly_Linked_Lists (Node);
    end Something_Else;

****************************************************************

From: Randy Brukardt
Date: Monday, October 22, 2007  7:09 PM

...
> It means I can create a Thing record containing a Vector of pointers to Things,
> either by passing a pointer to Vectors or by passing Thing to
> Indefinite_Vectors.  But I can't create a Thing containing a pointer to a
> Vector of Things.  It's bad enough that we force the programmer to introduce a
> level of indirection in order to have recursive types -- now we're telling them
> which place has to have the indirection.

Careful here: by wanting too much the last time, you caused us to look for
other solutions; and the net effect is that you got none. (That may have
been because of a mis-interpretation of your comments, but it surely
happened.)

I think the most important thing is that you can write a private type that
uses the containers recursively. You can't currently do that in any way at
all in Ada, and that is clearly bad. But the most important thing is that
there is a way; obsessing over having the perfect way probably will lead
back to where we are now. (The only way that won't happen is if several
implementers drop out of the ARG, removing their objections to proposals
that have split freezing or elaboration. But then there probably won't be a
need for future Ada standards...)

Of course, we should try to make this as usable as possible, but we have to
be careful to avoid making "best" the enemy of "better" to the point where
we end up with "not-at-all".

****************************************************************

From: Randy Brukardt
Date: Monday, October 22, 2007  7:17 PM

> I'm confused.  I thought we were talking about the case where Thing is private,
> and Thing_Vectors is exported from the package.  So "new Vectors(Thing)" is
> illegal, since the formal does not have unknown discrims.

Humm, I think my current proposal solves this problem:

    package Bob is
        type Thing is private;
        package Thing_Vectors is new Vectors(Thing, ...) for export with limited view;
    private
        type Thing is record
            Vec : access Thing_Vectors.Vector;
        end record;
    end Bob;

There are only two problems worth solving here:
  (1) The "Duff" problem of using a container of a type within the type itself;
  (2) The "Signature" problem of exporting a generic instantiation of private types.

My proposal has solved both of these, and introduces no new dynamic
semantics [which is where the problem was]. (Other than possibly a
conformance check.) The implementation of the visibility changes appear to
be minor (pointing the exported instance at the full one for external usage
seems trivial - of course, you can't reason about other people's compilers).
What's not to like?? ;-)

****************************************************************

From: Robert A. Duff
Date: Monday, October 22, 2007  8:00 PM

> Careful here: by wanting too much the last time, you caused us to look for
> other solutions; and the net effect is that you got none. ...

Understood.  There seems to be no perfect solution.  So let's be honest about
the drawbacks of each, but not let those drawbacks paralyze us.

****************************************************************

From: Tucker Taft
Date: Monday, October 22, 2007  8:53 PM

> But this is a variation of ideas that we could not get to work in the past.
> The problem was two-fold: implementation earthquakes for various models, and
> privacy violations. It's actually three-fold for this one, because partially
> instantiating nested instantiations means that where the body of those
> nested instantiations are elaborated would differ depending on how the
> instantiation is written. That seems bad: it's not uncommon to use an
> instantiation to get a body elaborated in a spec (Ada provides no other way
> to do that). So, given a generic like:
> 
>      generic
>          type Foo (<>) is ...
>      package Bar is
>          package Nested is new <some generic> (Foo);
>          subtype Bounds is Integer range 1 .. Nested.Func;
>      end Bar;
> 
> A partial instantiation of Bar would raise Program_Error on the call of
> Nested.Func (because the body is not elaborated) but would work correctly
> for a normal instantiation. That's unpleasant.

If we were to take this approach, then the implementor
of a generic that had a formal type with unknown
discriminants would have to decide whether or not
to support partial instantiations.  If they intended
to support a partial instantiation, then they would
clearly have to test it with one.  I think formals
with unknown discriminants are rare enough that
that wouldn't be an undue burden.

In general, nested instantiations are pretty
rare, and nested instantiations that pass
a formal type with unknown discriminants are
even rarer, and nested instantiations where
one takes advantage of immediate elaboration
of the body to make a call are even rarer still.
If this very rare generic turns out to be
something where it would be useful to support
instantiations with a private type, then having
to redesign it a bit for that purpose seems reasonable.

>>     3) At the full instantiation, freeze the actuals
>>        corresponding to formals with unknown discrims,
>>        then instantiate the bodies of any nested
>>        instantiations only partially instantiated
>>        thus far, and then instantiate the body of
>>        the "main" generic.
>>
> ...
>> It also has the advantage that all actual
>> types in a generic instance may still be presumed
>> to be "pre-frozen," *unless* the formal has unknown
>> discriminants.  Furthermore, because formals with
>> unknown discriminants cannot be used as components
>> of other types, it is safe to use any type exported
>> by the generic instance in the full definition of
>> a private type passed as the actual, since it can't
>> create a circularity.
> 
> But the real problem is that you haven't addressed the anomalies that Steve
> and Erhard pointed out the last time we discussed this (and that effectively
> killed it). The problem was that a lot of partially defined stuff
> potentially gets exported from a partial view, and this goes beyond the
> sorts of things that we currently have to deal with.

Can you elaborate on this?  I don't think anything is partially
defined.  Some bodies aren't elaborated, but that is no
surprise in a package spec.  Most bodies aren't elaborated
in a package spec.

Despite surface similarities, I think the problems Erhard and Steve were
talking about don't apply to this proposal.  In the earlier
proposal, we were trying to prevent the partial instantiation
from freezing *anything*.  In the proposal we are discussing
here, the only thing that is special is that the actual
types that are associated with formal types with unknown
discriminants are not "pre-frozen."  But if they get used
in a spec in a way that would normally require freezing,
then they get frozen.  The spec is expanded with completely
"normal" semantics.  What gets postponed is the instantiation
(and elaboration) of the body, and the freezing of certain
actuals.

If I am wrong about this, it would be helpful if you could
identify which variant of AI-359 you think this most recent
proposal matches, and which ARG meeting Erhard and Steve
found the flaws.  I just looked at a bunch of minutes, and
they all seemed to be concerned with proposals where we
were trying to defer all freezing, or make everything into
a partial view, or defer all elaboration, rather than
just the elaboration of the body.  September 2004 in Madison
seemed to be the one you were probably remembering.

Unless I am mistaken, we never discussed
a proposal where the partial instantiation
did all the usual things associated with the instantiation
of a spec, creating full types, etc., with the only change
being that we eliminated the pre-freezing of a subset of
the actuals, and deferred the instantiation of the body.

> My personal opinion is that the "partial instantiation" (versus a "limited
> instantiation") is never going to work. I've been trying to make the
> "limited instantiation" more acceptable, but I haven't gotten any real
> feedback from you.
> 
> I realize that some people don't like the idea for some reason; what I'd
> like to find out is what about the proposal is disliked so that it can be
> enhanced. (I'm convinced that it would be best to start with a mechanism
> that works and would not clobber implementers, and then see if we can safely
> extend it to be more powerful.)

I felt that it was useful to try to come up with a proposal
that did *not* require the explicit use of access types.

Your proposal is providing only incomplete types, since it is
providing only a limited view of the instance from within
the package.  Hence access types must be used in any reference
to types declared in the package.  That is annoying if the
generic already introduces a level of indirection, and
has a bunch of code to ensure leak-free storage management.
The client can't take advantage of that when only given
incomplete types.

If all that we get are incomplete types, then I guess
I prefer the generality of the "end private;" over the
unique and somewhat counter-intuitive semantics of
the  "limited internal view" approach.

As far as a specific comment on the limited internal
view idea, syntactically, I would stick with something simple
like "package I is limited new ..." for now, always
provide the limited view internally, and save the
Norm-Cohen-string-of-reserved-words for late nights
at the bar after ARG meetings. ;-)

****************************************************************

From: Randy Brukardt
Date: Monday, October 22, 2007  9:46 PM

...
> > But the real problem is that you haven't addressed the anomalies that Steve
> > and Erhard pointed out the last time we discussed this (and that effectively
> > killed it). The problem was that a lot of partially defined stuff
> > potentially gets exported from a partial view, and this goes beyond the
> > sorts of things that we currently have to deal with.
>
> Can you elaborate on this?  I don't think anything is partially
> defined.  Some bodies aren't elaborated, but that is no
> surprise in a package spec.  Most bodies aren't elaborated
> in a package spec.

Please read the Atlanta minutes.

> Despite surface similarities, I think the problems Erhard and Steve were
> talking about don't apply to this proposal.  In the earlier
> proposal, we were trying to prevent the partial instantiation
> from freezing *anything*.  In the proposal we are discussing
> here, the only thing that is special is that the actual
> types that are associated with formal types with unknown
> discriminants are not "pre-frozen."  But if they get used
> in a spec in a way that would normally require freezing,
> then they get frozen.  The spec is expanded with completely
> "normal" semantics.  What gets postponed is the instantiation
> (and elaboration) of the body, and the freezing of certain
> actuals.

Quoting from the Atlanta minutes:

"Tucker has changed the freezing rules to require freezing at the end of an
instance and then have piecemeal instantiation. This is necessary so a type
can contain a component of its own type. Pascal objects that this is a
change that we had previously agreed to not make. Randy agrees; he notes
that it is privacy breaking. Pascal says that piecemeal freezing is a dead
boy issue. He continues that the important issue is to solve the export of
containers from specs of private types. Tucker notes also that the signature
package problem is also covered even without piecemeal freezing. Randy notes
that we could still allow this in 2015 if it actually proves to be
important."

And now you are reintroducing this again - some ideas just never die.

> If I am wrong about this, it would be helpful if you could
> identify which variant of AI-359 you think this most recent
> proposal matches, and which ARG meeting Erhard and Steve
> found the flaws.  I just looked at a bunch of minutes, and
> they all seemed to be concerned with proposals where we
> were trying to defer all freezing, or make everything into
> a partial view, or defer all elaboration, rather than
> just the elaboration of the body.  September 2004 in Madison
> seemed to be the one you were probably remembering.

Then we started having problems with things exported from the instantiation.
The solution (as I see it as least) is to not export so much. You seem to
see the solution as to defer freezing and break privacy.

> Unless I am mistaken, we never discussed
> a proposal where the partial instantiation
> did all the usual things associated with the instantiation
> of a spec, creating full types, etc., with the only change
> being that we eliminated the pre-freezing of a subset of
> the actuals, and deferred the instantiation of the body.

No we didn't. But you again are talking about partial freezing and privacy
breaking, and that was a non-starter in the past. We need proposals that
don't have those properties, not more of the same.

Moreover, I'm not very interested in a proposal that will only work with a
few "special" generics. The standard definite (and presumably bounded)
containers will not work with this proposal. So some users (safety-critical,
for instance) seem to be out of luck. Similarly, the majority of existing
generics don't have formals with unknown discriminants; you want everyone to
rewrite their generics. Yuck.

> > My personal opinion is that the "partial instantiation" (versus a "limited
> > instantiation") is never going to work. I've been trying to make the
> > "limited instantiation" more acceptable, but I haven't gotten any real
> > feedback from you.
> >
> > I realize that some people don't like the idea for some reason; what I'd
> > like to find out is what about the proposal is disliked so that it can be
> > enhanced. (I'm convinced that it would be best to start with a mechanism
> > that works and would not clobber implementers, and then see if we can safely
> > extend it to be more powerful.)
>
> I felt that it was useful to try to come up with a proposal
> that did *not* require the explicit use of access types.
>
> Your proposal is providing only incomplete types, since it is
> providing only a limited view of the instance from within
> the package.  Hence access types must be used in any reference
> to types declared in the package.  That is annoying if the
> generic already introduces a level of indirection, and
> has a bunch of code to ensure leak-free storage management.
> The client can't take advantage of that when only given
> incomplete types.

The client *shouldn't* be able to take advantage of the private
implementation of a generic unit: that's privacy breaking. One could argue
that it would be nice to take advantage of a visible use of indirection --
but that's pretty unlikely in a well-designed generic. It doesn't seem worth
a lot of implementation pain to provide that capability.

To really allow this, I think it would have to be declared in the contract
of the generic somehow. (No, unknown discriminants does not do that, as
there would usually not be any objects at all. And they're too limiting in
any case.) And clearly it would have to be enforced end-to-end.

> If all that we get are incomplete types, then I guess
> I prefer the generality of the "end private;" over the
> unique and somewhat counter-intuitive semantics of
> the  "limited internal view" approach.

The problem is that is way too general, and certainly will introduce new
anomalies. (I can only imagine what the effect on visibility in public
children would be.) Moreover, the 0 or 1 or infinite rule implies that we
would need to add an unlimited number of these new parts -- and visibility
changes are always an implementation earthquake. It also scatters the
visible declarations all over, and forces them into a particular order,
which may not be logical (forcing the instances very late, for example).

It would make just as much sense to follow Robert Eachus' suggestion and
allow individual declarations to be made visible or hidden from clients
(making the private part an obsolescent feature). Then we could be sure that
the earthquake would be complete but also that we wouldn't be forcing any
ugly constructs on people.

> As far as a specific comment on the limited internal
> view idea, syntactically, I would stick with something simple
> like "package I is limited new ..." for now, always
> provide the limited view internally, and save the
> Norm-Cohen-string-of-reserved-words for late nights
> at the bar after ARG meetings. ;-)

I think you miss the point: the critical use is to allow signatures and
containers to be instantiated visibly. The syntax needs to reflect that
somehow, and "is limited" doesn't do that (in fact, it seems to imply that
client use is restricted, when the exact opposite is true). Think of what a
client of a package would see if they don't peek in the private part.

The Bob Duff use for recursive data types is likely to be rare, and I didn't
want the syntax to emphasize that use (which is irrelevant for clients
anyway). (If that was the only problem, I would have been in favor of "No
Action" from the beginning.)

That said, people will get used to anything, so I don't care that much what
the syntax is.

****************************************************************

From: Tucker Taft
Date: Monday, October 22, 2007 10:13 PM

> Quoting from the Atlanta minutes:
> 
> "Tucker has changed the freezing rules to require freezing at the end of an
> instance and then have piecemeal instantiation. This is necessary so a type
> can contain a component of its own type. Pascal objects that this is a
> change that we had previously agreed to not make. Randy agrees; he notes
> that it is privacy breaking. Pascal says that piecemeal freezing is a dead
> boy issue. He continues that the important issue is to solve the export of
> containers from specs of private types. Tucker notes also that the signature
> package problem is also covered even without piecemeal freezing. Randy notes
> that we could still allow this in 2015 if it actually proves to be
> important."
> 
> And now you are reintroducing this again - some ideas just never die.

It may *sound* the same, but it really isn't.  This is
not the same as the "piecemeal" freezing we discussed.
Also, the privacy breaking is dramatically reduced if you limit
this to formal types with unknown discriminants, since they
can't be used as components anyway.  By eliminating this
issue, the breakage doesn't seem any worse than the various other
legality rules that are checked in the private parts of
instances.

In any case, I can see that at least *you* aren't interested
in this proposal, even if I can convince you that it really
*is* new. ;-)  It felt like a step forward to me...

****************************************************************

From: Jean-Pierre Rosen
Date: Wednesday, October 24, 2007 11:06 AM

I've been following this thread, and I must say I am worried by the 
complexity, from a user's point of view, of all the solutions presented.

In short: only members of the ARG will be able to use that. Casual users 
already know very few about the real possibilities of the language (OK, 
that's what makes our business as consultants). I doubt that they will 
find out the appropriate spell if they want to instantiate a generic on 
a private type (the real need).

The only acceptable solution is to allow:

with Gen;
package pack is
    type T is private;

    package Inst is new Gen (T);

and nothing else. Can it be done? For example, decide that in this case 
actual instantiation does not happen logically at the place where it is 
declared, but immediately after the full declaration of T. But of course 
the visibility of the instantiation would be public. No use of the 
instance would be allowed until it is "logically" instantiated.

Am I pipe-dreaming here?

****************************************************************

From: Edmond Schonberg
Date: Wednesday, October 24, 2007  2:06 PM

> I've been following this thread, and I must say I am worried by the  
> complexity, from a user's point of view, of all the solutions  
> presented.

You can add:  "and from the implementor's point of view".

>
> In short: only members of the ARG will be able to use that. Casual  
> users already know very few about the real possibilities of the  
> language (OK, that's what makes our business as consultants). I  
> doubt that they will find out the appropriate spell if they want to  
> instantiate a generic on a private type (the real need).
>
> The only acceptable solution is to allow:
>
> with Gen;
> package pack is
>    type T is private;
>
>    package Inst is new Gen (T);
>
> and nothing else. Can it be done? For example, decide that in this  
> case actual instantiation does not happen logically at the place  
> where it is declared, but immediately after the full declaration of  
> T. But of course the visibility of the instantiation would be  
> public. No use of the instance would be allowed until it is  
> "logically" instantiated.

I agree that for a new language design the only reasonable solution  
would be to allow types with only partial views to appear as actuals.  
However, as was mentioned repeatedly, any change to the freezing  
rules is extremely disruptive. It is nothing we would be willing to  
do without a very VERY strong user demand, and that is at present  
totally absent.  If  Ada2005 programmers start using containers in  
more ambitious forms maybe they will find it convenient to have the  
premature instantiations that have motivated this AI. In the  
meantime, there is no harm in exploring other designs, but we may  
have more pressing things to fix.

****************************************************************

From: Tucker Taft
Date: Wednesday, October 24, 2007  2:19 PM

I agree, simplicity is important.  But it is
always dangerous to guess at the simplicity of
the final result based on the complexity of
the discussion leading up to it.  There
is often no correlation (either way ;-).

I think the "end private;" idea is pretty
simple from the user perspective, but
I also agree we need to have a complex
discussion about what it really means
from a detailed visibility point of view,
so that it has the right "intuitive" semantics
from the user point of view.

I also think Randy's idea could appear as
simple to a user, so long as the syntax is kept
very simple.  I like using the word
"limited" or the phrase "with private" since both
of those suggest what it going on, as in:

    package Inst is limited new Gen (T);
or
    package Inst is new Gen(T) with private;

followed by a full normal instantiation
in the private part, after the full
definition of T.

  private
    type T is ...
    package Inst is new Gen(T);

This seems relatively simple, and the
partial instantiation syntax is relatively
familiar from either the new "limited new ..." syntax
on type derivation (for the first syntax), or
private extensions (for the second syntax).
I can't see adding a bunch of new
keywords for this relatively simple
idea.

I also don't see sufficient benefit to have two
variants, one that makes a limited view of
the instance visible prior to the full
instantiation, and one that doesn't.

I think I understand Randy's reasons for
the new keywords and the two variants, but
the cost/benefit ratio is too high for
both in my view.  "Limited" already has
multiple meanings, and so we can choose how
we want to define a "limited" instantiation,
and saying that it provides only incomplete
types prior to the full instantiation
seems reasonable.  Essentially, the instance
is "incompletely defined" prior to the full
instantiation, in the same way a private
type is incompletely defined prior to the
full type definition.

Similarly, "with private" currently means
"partial view of an extension"
so it could also mean "partial view of
an instantiation" (and along the way,
"with private" is suggestive of the
fact that the actual parameter part can
include one or more private types).

I suspect that users could grasp either
pretty quickly.

As far as not having a distinct syntax for
the partial instantiation, we have discussed
something very close to that proposal in the
past, where the fact that one of the actuals
is not fully defined automatically causes a
different interpretation.  Unfortunately, the complexity
of the discussion was astronomical, and
ultimately we abandoned it, despite its
apparent simplicity from a user point of view.

So I think we actually have two reasonable
suggestions, one involving partial instantiations
and one allowing multiple visible parts.
If the syntax is chosen carefully, I think
either one of these could pass the simplicity
test.  Even "normal" users already understand there
are limits on how a private type can be
used prior to its full definition.

So please keep pushing simplicity, but don't
prejudge based on the complexity of the discussion.
Try to block out the whole ARG discussion, and decide
whether the final result could have a reasonable syntax
and relatively intuitive semantics.

****************************************************************

From: Randy Brukardt
Date: Wednesday, October 24, 2007  2:28 PM

> The only acceptable solution is to allow:
>
> with Gen;
> package pack is
>     type T is private;
>
>     package Inst is new Gen (T);
>
> and nothing else. Can it be done?

In short, no. ;-) It shouldn't be a surprise that complex problems require
complex solutions -- or, for that matter, that two part declarations also
require two-part instances (just like using a private type in a subprogram
in the spec requires the body of the subprogram to be elsewhere).

> For example, decide that in this case
> actual instantiation does not happen logically at the place where it is
> declared, but immediately after the full declaration of T. But of course
> the visibility of the instantiation would be public. No use of the
> instance would be allowed until it is "logically" instantiated.
>
> Am I pipe-dreaming here?

It's hard to tell for sure, but I think you are.

(1) This does not solve the "Duff" problem of using "list of T" (and other
data structures) inside of T. You can do that with the built-in data
structures of Ada; it is unfortunate that you can't do it with containers. I
don't think this is the most important issue by far, but it is surely
important.

(2) There is a maintenance hazard here, as changing the specification of the
generic or of the package would change where the generic instantiation
actually occurs. This isn't as bad as in some of the other proposals (in
this case, you'd get a compile-time error if there was a problem), but it
still is an issue. [There also might be issues if the instantiation was
mainly used for a side-effect of its elaboration, moving that elaboration
could cause problems. But such code is tricky anyway, so I can't get too
worried.]

One of the reasons for the two-part instantiation is so that the location of
the "real" instantiation is obvious in the source code. That eliminates this
maintenance hazard.

(3) You don't specify in which cases this magic happens. Does it happen with
all generics, or just a special subset. Assuming it happens with all
generics, when do the other generic parameters (those not dependent on
private types) get evaluated? Either answer seems bad in some cases.

(4) Someone had suggested an idea on this line (not exactly this idea) back
in 2004. It eventually was dropped because of various problems.

The net effect is that I don't think this works. To solve (1), you have to
complicate the freezing and elaboration rules and allow partial exporting of
the contents of the generic. (Essentially what Tucker was trying to do.)
Which has been roundly rejected in the past, as it increases the maintenance
hazard and implementation complications.

I understand your concern about complexity for the user. I don't think it is
too bad, as compilers and mentors can recommend the "magic incantation" of
adding a limited instance when it is needed -- and there are plenty of other
examples of two-part declarations in Ada (a user who is surprised by that is
a real newbie). Still it is fair to have the opinion that the solutions are
complex enough that it is not worth solving the problem at all. (This is not
an opinion that I share!)

I do think that all of these solutions look more complex when described in
detail than they would in actual use. (Which includes your proposal above,
if it was described to the same level of detail.) We don't get to handwave
the details, because the devil is always in the details. My main concern has
been that the syntax of the limited instance (or partial instance, for that
matter) doesn't really intuitively describe what is going on. Which is why I
was trying more complex syntaxes that hopefully made more sense when reading
(but Tucker didn't seem to agree, and I guess you don't either).

****************************************************************

From: John Barnes
Date: Tuesday, October 30, 2007  2:32 AM

>> ...In these
>> discussions, I have harbored a personal dislike for the
>> idea of multiple private parts in a single package,
>> but I haven't had a good reason other than aesthetic.
>> But this to me seems like a good reason.
>
> Agreed.  I find vague aesthetics much more convincing when backed up by good
> reasons!

Is it not the case that a child package essentially opens the visible part
of the parent once more but after the private part? Is there the essence of
a solution here?  Maybe it was discussed before. I recall we did once
consider using child packages as one of the options when discussing cyclic
packages.

****************************************************************

From: Tucker Taft
Date: Tuesday, October 30, 2007  8:12 AM

In choosing semantics for a post-private visible
part, I have suggested we use the visible part of
a child as a model.

I also suggested in a separate note of having
a way to specify that a given child is to
be implicitly "with"ed by all units that "with"
the parent.  I had suggested "end with Parent.Child;"
but no one liked that syntax.  Somewhat better might
be "at end with ...", e.g.:

    at end with Parent.Child1, Parent.Child2, ...;
    package Parent is ...
       ...
    end Parent;

or conceivably:

    package Parent is
       ...
    private
       ...
    end Parent with Parent.Child1, Parent.Child2;

These give you the ability to have a child that provides
an instantiation that will appear to clients of Parent
to be included inside Parent, but they don't give you any ability
to use that instantiation before the end of the parent package.
You would still need to be able to insert some declarations
after establishing visibility on the child packages,
if you want to use the instantiations to help define
one of the types exported from the parent package.
Given that, it seems simpler to just use a post-private
visible part to both specify the instantiations
needed, as well as any additional declarations that
link the instantiations up to types exported from
the package.

****************************************************************


Questions? Ask the ACAA Technical Agent