Version 1.14 of ais/ai-50217.txt

Unformatted version of ais/ai-50217.txt version 1.14
Other versions for file ais/ai-50217.txt

!standard 10.01.01 (12)          04-05-21 AI95-00217-06/08
!standard 10.01.01 (26)
!standard 10.01.02 (04)
!standard 10.01.02 (06)
!standard 10.01.02 (08)
!standard 10.01.04 (03)
!standard 10.01.04 (06)
!standard 10.02 (06)
!standard 3.10.01 (10)
!standard 4.01 (09)
!standard 8.03 (20)
!standard 8.04 (05)
!standard 8.04 (07)
!standard 8.04 (08)
!standard 8.05.03 (03)
!class amendment 03-02-04
!status Amendment 200Y 04-01-09
!status WG9 Approved 04-06-18
!status ARG Approved 11-0-1 03-12-11
!status work item 03-02-04
!status received 03-02-04
!priority High
!difficulty Hard
!subject Limited With Clauses
!summary
A new kind of with_clause is proposed.
!problem
Ada allows mutually recursive types to be declared only if they are all declared within the same library unit. This can force all of the major data structures of a program into a single library unit, which is clearly undesirable in some situations.
The goal of the proposed feature is to allow mutual recursion among types declared in separate packages (types containing (pointers to) each other as components, and primitive operations with (pointers to) each other as parameters), and to do this in a way that doesn't place any undue restrictions on the programmer.
!proposal
A new kind of with_clause is added, of the form "limited with <list of library package names>;". This is called a "limited_with_clause". The old fashioned kind is called a "nonlimited_with_clause".
There are two key features of a limited_with_clause such as "limited with X;":
- It introduces no semantic dependence on X (and hence no
elaboration dependence on X). The implementation may choose to elaborate X either before or after the current unit.
- Instead of providing visibility of the named packages and their
content, it provides visibility of only the names of the packages and the packages nested within them, and an incomplete view of all types in the packages and nested packages. Such incomplete views imply the usual restrictions on incomplete types. If the type is tagged, the view is tagged incomplete.
Thus, cyclic chains of with_clauses are allowed, so long as the chain is broken by at least one limited_with_clause.
!wording
Replace 3.10.1(10) by:
A dereference (whether implicit or explicit -- see 4.1) is only allowed to be of an incomplete type T if it does not occur in the immediate scope of [the incomplete type] T and:
o it occurs in the scope of a nonlimited_with_clause that mentions the library
package in which that completion is declared; or
o its expected type is the completion of T.
<Author's note: this wording is provisional; final wording will be provided by AI-326, which is expected to define incomplete types in terms of views.>
Replace 4.1(9) by:
If the type of the name in a dereference is some access-to-object type T, then the dereference denotes a view of an object. The nominal subtype of that view is the designated subtype D of T, unless D is an incomplete type, in which case the nominal subtype is the completion of D.
<Author's note: same comment as for 3.10.1(10), above.>
Change the beginning of 8.3(20):
The declaration of a library unit (including a library_unit_renaming_declaration) is hidden from all visibility except at places that are within its declarative region or within the scope of a nonlimited_with_clause that mentions it. The limited view of a library package is hidden from all visibility except at places that are within the scope of a limited_with_clause that mentions it but not within the scope of a nonlimited_with_clause that mentions it.
Replace 8.4(5) by:
A package_name of a use_package_clause shall denote a nonlimited view of a package.
Insert after 8.4(7):
A package is named in a use_package_clause if it is denoted by a package_name of that clause. A type is named in a use_type_clause if it is determined by a subtype_mark of that clause.
Replace 8.4(8) by (this is just using the new terminology):
For each package named in a use_package_clause whose scope encloses a place, each declaration that occurs immediately within the declarative region of the package is potentially use-visible at this place if the declaration is visible at this place. For each type T or T'Class named in a use_type_clause whose scope encloses a place, the declaration of each primitive operator of type T is potentially use-visible at this place if its declaration is visible at this place.
Replace 8.5.3(3) by:
The renamed entity shall be a nonlimited view of a package.
Add after 10.1.1(12):
For each library package_declaration in the environment, there is an implicit declaration of a limited view of that library package. The limited view of a package contains:
- For each nested package_declaration, a declaration of the limited view of
that package, with the same defining_program_unit_name.
- For each type_declaration in the visible part, an incomplete_type_declaration
with the same defining_identifier, whose completion is the type_declaration. If the type_declaration is tagged, then the incomplete_type_declaration is tagged incomplete.
<Author's note: this wording is provisional; final wording will be provided by AI 326, which is expected to define incomplete types in terms of views. This wording would have problems with types that have multiple views (such as private types), as incomplete_type_declarations would exist for both views.>
The limited view of a library package_declaration is private if that library package_declaration is immediately preceded by the reserved word private.
[There is no syntax for declaring limited views of packages, because they are always implicit.] The implicit declaration of a limited view of a package is _not_ the declaration of a library unit (the library package_declaration is); nonetheless, it is a library_item.
A library package_declaration is the completion of the declaration of its limited view.
The elaboration of the limited view of a package has no effect.
Insert in 10.1.1(26):
The implicit declaration of the limited view of a library package depends semantically upon the implicit declaration of the limited view of its parent. The declaration of a library package depends semantically upon the implicit declaration of its limited view.
Replace 10.1.2(4) by:
with_clause ::= limited_with_clause | nonlimited_with_clause limited_with_clause ::= limited with library_unit_name {, library_unit_name}; nonlimited_with_clause ::= with library_unit_name {, library_unit_name};
Replace 10.1.2(6) by:
A library_item is named in a with_clause if it is denoted by a library_unit_name in the with_clause. A library_item is mentioned in a with_clause if it is named in the with_clause or if it is denoted by a prefix in the with_clause.
Add after 10.1.2(8):
A library_item mentioned in a limited_with_clause shall be a package_declaration[, not a subprogram_declaration, generic_declaration, generic_instantiation, or package_renaming_declaration].
A limited_with_clause shall not appear on a library_unit_body or subunit.
A limited_with_clause which names a library_item shall not appear:
o in the same context_clause as a nonlimited_with_clause which mentions the
same library_item; or
o in the same context_clause as a use_clause which names an entity declared
within the declarative region of the library_item; or
o in the scope of a nonlimited_with_clause which mentions the same
library_item; or
o in the scope of a use_clause which names an entity declared within the
declarative region of the library_item.
Add after 10.1.4(3):
The mechanisms for adding a unit mentioned in a limited_with_clause within an environment are implementation defined.
Change 10.1.4(6):
The implementation may require that a compilation unit be legal before {it can be mentioned in a limited_with_clause or it can be inserted} [inserting it] into the environment.
Add after 10.2(6):
o If the limited view of a unit is needed, then the full view of the unit is
needed.
!discussion
It is natural to think of a with_clause as causing a library unit to become visible. However, that's not what the Standard says. Instead, the Standard says that lack of a with_clause causes a library unit not to be visible. That's kind of odd, but to understand the wording, you have to understand the existing model in this way.
The Standard (in 10.1.4(1)) defines the notion of an "environment declarative_part". During compilation, this fanciful declarative_part is imagined to contain all the library_items of interest. The order of these library_items is such that there are no forward semantic dependences. With_clauses introduce semantic dependences, so they control the order. Semantic dependences are also caused by child-parent and body-specification relationships. Subunits are imagined to be "inlined" at their stub point, so subunits can be ignored for visibility purposes.
The visibility rules for library units come from treating the environment declarative_part in the usual way, with one difference: In a normal declarative_part, you can see everything that comes before, and nothing that comes after (with a bunch of hiding rules thrown in as well). In the environment declarative_part, however, you can see things that come before, but only if mentioned in a with_clause.
Similar oddities occur in the definition of "immediate scope" and "scope".
The best way to introduce limited_with_clauses into this model is to say that there is an implicit declaration of a "limited view of package X" (occurring before any instance of "limited with X"). The normal declaration of X then appears later in its usual place. The form "limited with X" introduces a very limited view of X that only includes nested packages and incomplete types.
The changes to 10.1.2(8) ensure that, if for instance the specification of Q has a "with P.R", it shall not have a "limited with P" or a "limited with P.R". The same applies to the children of Q. On the other hand, a "limited with P.R.S" is fine.
The fact that a "limited with P" hides the full view of P from all visibility prevents ripple effects. If you are in the scope of a "limited with P", the full view of P is hidden from all visibility, even if P would otherwise be visible indirectly (e.g. by a renaming in an auxiliary package). So the limited_with_clause effectively reverts to the limited view of P, and the types it declares are incomplete, and therefore subject to the limitations described in 3.10.1. From an implementation standpoint, it means that compilers can decide early (i.e., after looking at the context_clauses of a unit and of its ancestry) what view of P needs to be made available for name resolution purposes. It is still the case that the compiler may need to look at the full view of P, e.g. if some unit in the closure was compiled against the full view of P.
Note that package instantiations and package renamings are not part of the limited view of a package, because we want to be able to build the limited view using very simple syntactic analysis (we are lucky that it's possible to identify tagged types simply by looking at the syntax). But for package renamings and instantiations, the name of the renamed package or of the generic would need to be resolved, and that would require full-fledged visibility analysis.
To simplify implementation and avoid Beaujolais effects, use clauses are not allowed to mention a limited view of a package. Consider the example below; if the use clauses were legal, changing the "with A" to "with B" would silently change the meaning of X in P.C:
package A is X : Integer := 42; end A;
package B is X : Integer := 666; end B;
limited with A; limited with B; use A, B; -- Illegal. package P is end P;
with A; -- Change this to "with B" and it changes the meaning of X. package P.C is Y : Integer := X; end P.C;
Furthermore, use clauses are inherited from the parent unit. This may be used to construct a situation where a limited_with_clause occurs in the scope of a use_clause for the same unit. We make such a limited_with_clause illegal, so that we don't have entities which are use-visible in a region where only the limited view of the enclosing library package is visible. For example:
with A; package P is package R renames A; end P;
with P; package Q is use P.R; -- Makes declarations in A use-visible. end Q;
limited with A; -- Illegal. package Q.C is -- Would declarations in A be use-visible here? -- Better not answer that question... end Q.C;
The following example shows the interactions with child units and nested packages:
package P is end P;
package P.C is package NP is end NP; end P.C;
with P.C; package Q is package PC renames P.C; end Q;
with Q; package R is use Q.PC.NP; -- Makes declarations in P.C.NP use-visible. end R;
limited with P; -- Illegal. package R.C is end R.C;
In package R.C, the declarations in package P.C.NP are use-visible. We avoid complex visibility issues by making the "limited with P" illegal. It cannot be useful anyway, because the full view of P is surely available if we have use-visibility on some entity declared within its declarative region.
A limited view of a package cannot be mentioned in a package renaming declaration. The problem comes when referencing such a renaming from a unit outside the unit containing the limited with. What view is seen? If the outside unit has a regular with clause for the target of the limited with, is the full view of the package visible via the renaming or only the limited view? These are questions that are best left unanswered.
We do not allow a limited_with_clause to mention a procedure, because that would allow calling a procedure before its specification is elaborated. The language has no mechanism to deal with that. Similarly, we do not allow a limited_with_clause to mention a generic, because that would allow instantiating a generic before its specification is elaborated.
We do not allow a limited_with_clause to mention a generic instantiation or a renaming, because that would require the full machinery of semantic analysis to determine the contents of the limited view (including visibility analysis).
An implementation need not create the limited view of a unit if that unit is "really broken". For instance, the limited view of P may not be created if the compiler is given the unit:
package Bad is type T is new Integer; type T is new Float; end Bad;
and even if the limited view is created, clients may still be rejected in an implementation defined manner. Thus, the following unit might be rejected:
limited with Bad; package Client is type A is access Bad.T; end Client;
This is not different from the current state of affairs with normal withs, except that we need a specific rule because the limited view is not covered by "inserted into the environment".
The applicable nonlimited_ and limited_with_clauses determine (1) which entities are visible, (2) which dereferencings are legal and (3) the type of dereferences. In particular:
1 - Consider an expanded name P.X. X is visible if either the name is within
the scope of a "with P" or if it is within the scope of a "limited with P" and it is part of the limited view (in other words, if it is a nested type or package).
2 - Consider a dereference X.all, where X is of an access-to-incomplete type
with designated type T. This dereference is illegal unless it occurs in the scope of a "with P" (where P is the package that declares T), within the immediate scope of the completion of T, or in a context where the expected type is the completion of T.
3 - Dereferencing an access-to-incomplete type (when it is legal) yields a name
whose nominal subtype is the completion of T. This makes it possible to write an expression like X.all.Comp (assuming that we have the visibility necessary to know that Comp exists).
There are various places in Chapter 8 that refer to "the declaration of a library unit", presuming that there's only one. We need to change the wording. One option is to say that library packages now have two declarations: the limited view, plus the normal package_declaration. Another option is to say that "the declaration of a library unit" is the normal package_declaration, the limited view being "something else". The latter option seems simpler, as it only requires the tweaking of 8.3(20) and doesn't interact with other parts of the language (e.g. freezing rules, library unit pragmas) like the former option would.
The part about the "declarative region of the incomplete type" in the wording of 3.10.1(10) is intended to forbid a case like:
package P is private type T; type Ref is access T; Ptr : Ref; end P;
with P; package body P is Too_Early : Integer := Ptr.all'Size; -- Illegal. type T is ...; end P;
Here the dereference is illegal because it occurs in the declarative region of the incomplete type T, and the nonlimited_with_clause doesn't make it legal. Note that a dereference that would occur after the completion of T would not be of an incomplete type, so 3.10.1(10) would not apply; consequently, such a dereference would be legal even though it would be in the declarative region of the incomplete type T. Note that the final wording regarding dereferences will be provided by AI-326 based on the definition of partial views for incomplete types.
Presuming that AI-262 is adopted, limited_with_clauses also combine with private_with_clauses, with the following syntax:
limited private with P;
Such a clause give visibility on the private view of P, but only at the beginning of the private part of the package where it appears. (The wording sections of AI-217 and AI-262 will obviously need to be merged when building the Amendment.)
!example
Here is the classic case of mutual dependence, where an employee belongs to a particular department, and a department has a manager who is an employee. The use of anonymous access types where possible eases the burden of type conversions on subprogram calls.
limited with Departments; package Employees is type Employee is tagged private; procedure Assign_Employee(E : in out Employee; D : access Departments.Department'Class); type Dept_Ptr is access all Departments.Department'Class; function Current_Department(D : in Employee) return Dept_Ptr; ... end Employees;
limited with Employees; package Departments is type Department is tagged private; procedure Choose_Manager(D : in out Department; Manager : access Employees.Employee'Class); ... end Departments;
!ACATS test
ACATS tests need to be created for this feature.
--Note: The following change is undone by AI-326. I don't have a way to --eliminate changes (they can only be overridden) in the creation program, so --I've just commented this one out. Put it back in the unlikely event that --AI-326 is killed. (It's first because the program doesn't recognize comments, --either.] -- --!corrigendum 4.1(9) -- --@drepl --If the type of the name in a dereference is some access-to-object type T, then --the dereference denotes a view of an object, the @i<nominal subtype> of the --view being the designated subtype of T. --@dby --If the type of the name in a dereference is some access-to-object type T, then --the dereference denotes a view of an object. The @i<nominal subtype> of that --view is the designated subtype D of T, unless D is an incomplete type, in which case --the nominal subtype is the completion of D.
!corrigendum 3.10.1(10)
Replace the paragraph:
A dereference (whether implicit or explicit -- see 4.1) shall not be of an incomplete type.
by:
A dereference (whether implicit or explicit -- see 4.1) is only allowed to be of an incomplete type T if it does not occur in the immediate scope of [the incomplete type] T and:
!corrigendum 8.3(20)
Replace the paragraph:
by:
!corrigendum 8.4(5)
Replace the paragraph:
A package_name of a use_package_clause shall denote a package.
by:
A package_name of a use_package_clause shall denote a nonlimited view of a package.
!corrigendum 8.4(7)
Insert after the paragraph:
For a use_clause immediately within a declarative region, the scope is the portion of the declarative region starting just after the use_clause and extending to the end of the declarative region. However, the scope of a use_clause in the private part of a library unit does not include the visible part of any public descendant of that library unit.
the new paragraph:
A package is named in a use_package_clause if it is denoted by a package_name of that clause. A type is named in a use_type_clause if it is determined by a subtype_mark of that clause.
!corrigendum 8.4(8)
Replace the paragraph:
For each package denoted by a package_name of a use_package_clause whose scope encloses a place, each declaration that occurs immediately within the declarative region of the package is potentially use-visible at this place if the declaration is visible at this place. For each type T or T'Class determined by a subtype_mark of a use_type_clause whose scope encloses a place, the declaration of each primitive operator of type T is potentially use-visible at this place if its declaration is visible at this place.
by:
For each package named in a use_package_clause whose scope encloses a place, each declaration that occurs immediately within the declarative region of the package is potentially use-visible at this place if the declaration is visible at this place. For each type T or T'Class named in a use_type_clause whose scope encloses a place, the declaration of each primitive operator of type T is potentially use-visible at this place if its declaration is visible at this place.
!corrigendum 8.5.3(3)
Replace the paragraph:
The renamed entity shall be a package.
by:
The renamed entity shall be a nonlimited view of a package.
!corrigendum 10.1.1(12)
Insert after the paragraph:
A library_unit_declaration or a library_unit_renaming_declaration is private if the declaration is immediately preceded by the reserved word private; it is otherwise public. A library unit is private or public according to its declaration. The public descendants of a library unit are the library unit itself, and the public descendants of its public children. Its other descendants are private descendants.
the new paragraphs:
For each library package_declaration in the environment, there is an implicit declaration of a limited view of that library package. The limited view of a package contains:
The limited view of a library package_declaration is private if that library package_declaration is immediately preceded by the reserved word private.
There is no syntax for declaring limited views of packages, because they are always implicit. The implicit declaration of a limited view of a package is not the declaration of a library unit (the library package_declaration is); nonetheless, it is a library_item.
A library package_declaration is the completion of the declaration of its limited view.
The elaboration of the limited view of a package has no effect.
!corrigendum 10.1.1(26)
Replace the paragraph:
A library_item depends semantically upon its parent declaration. A subunit depends semantically upon its parent body. A library_unit_body depends semantically upon the corresponding library_unit_declaration, if any. A compilation unit depends semantically upon each library_item mentioned in a with_clause of the compilation unit. In addition, if a given compilation unit contains an attribute_reference of a type defined in another compilation unit, then the given compilation unit depends semantically upon the other compilation unit. The semantic dependence relationship is transitive.
by:
A library_item depends semantically upon its parent declaration. A subunit depends semantically upon its parent body. A library_unit_body depends semantically upon the corresponding library_unit_declaration, if any. The implicit declaration of the limited view of a library package depends semantically upon the implicit declaration of the limited view of its parent. The declaration of a library package depends semantically upon the implicit declaration of its limited view. A compilation unit depends semantically upon each library_item mentioned in a with_clause of the compilation unit. In addition, if a given compilation unit contains an attribute_reference of a type defined in another compilation unit, then the given compilation unit depends semantically upon the other compilation unit. The semantic dependence relationship is transitive.
!corrigendum 10.1.2(4)
Replace the paragraph:
with_clause ::= with library_unit_name {, library_unit_name};
by:
with_clause ::= limited_with_clause | nonlimited_with_clause limited_with_clause ::= limited with library_unit_name {, library_unit_name}; nonlimited_with_clause ::= with library_unit_name {, library_unit_name};
!corrigendum 10.1.2(6)
Replace the paragraph:
A library_item is mentioned in a with_clause if it is denoted by a library_unit_name or a prefix in the with_clause.
by:
A library_item is named in a with_clause if it is denoted by a library_unit_name in the with_clause. A library_item is mentioned in a with_clause if it is named in the with_clause or if it is denoted by a prefix in the with_clause.
!corrigendum 10.1.2(8)
Insert after the paragraph:
If a with_clause of a given compilation_unit mentions a private child of some library unit, then the given compilation_unit shall be either the declaration of a private descendant of that library unit or the body or a subunit of a (public or private) descendant of that library unit.
the new paragraphs:
A library_item mentioned in a limited_with_clause shall be a package_declaration[, not a subprogram_declaration, generic_declaration, generic_instantiation, or package_renaming_declaration].
A limited_with_clause shall not appear on a library_unit_body or subunit.
A limited_with_clause which names a library_item shall not appear:
!corrigendum 10.1.4(3)
Replace the paragraph:
The mechanisms for creating an environment and for adding and replacing compilation units within an environment are implementation defined.
by:
The mechanisms for creating an environment and for adding and replacing compilation units within an environment are implementation defined. The mechanisms for adding a unit mentioned in a limited_with_clause within an environment are implementation defined.
!corrigendum 10.1.4(6)
Replace the paragraph:
The implementation may require that a compilation unit be legal before inserting it into the environment.
by:
The implementation may require that a compilation unit be legal before it can be mentioned in a limited_with_clause or it can be inserted into the environment.
!corrigendum 10.2(6)
Replace the paragraph:
by:
!appendix

From: Pascal Leroy
Sent: Monday, February  3, 2003  5:19 AM

A while back, Dan wrote, as a side comment: "Well, the nicest solution for the
user is just to allow specs to with each other, as other languages do."  This
got me thinking, as this is an approach that we have discussed briefly once or
twice, but writers of library-based compilers (like me) started to yell and
scream that it would be impossible to implement.  Well, I am starting to think
that all approaches considered so far are nearly impossible to implement, and
furthermore are quite hard to use.  So I started to investigate a solution where
units are allowed to with each others.  Here are my half-baked ideas:

1 - A new form of context clause is added:

    weak [private] with library_unit_name {, library_unit_name}

Weak with clauses don't have to form an acyclic graph.

2 - A compilation unit is added to the environment in two phases.  During the
first phase we say that the unit is "superficially added"; during the second
phase that it is "fully added" (better terminology welcome).  How this is
achieved is not specified by the language.

3 - To superficially add a unit to the environment, there is no requirement that
any other unit already exists in the environment.  (Not even its parent, heck,
not even Standard!)

4 - When a unit is superficially added to the environment, the only names that
can be referenced from outside the unit are those of packages (both the unit
itself and nested packages) and of types.  Moreover, every type is at this point
considered to be an incomplete type, and subject to the restrictions mentioned
in 3.10.1; the taggedness of the types is visible.  (The alert reader may notice
some similarity between a unit superficially entered into the environment and a
package abstract.)

5 - To fully add a unit to the environment, all of the units upon which it
depends semantically must already have been fully entered into the environment.
Furthermore, all of the units that it names in a weak with clause must already
have been superficially entered into the environment.  The only names that can
be referenced from such units are those of types and packages, and types are
considered incomplete.

6 - To fully add a unit to the environment, this unit must be made of the same
lexical elements (except for comments) as when it was superficially added to the
environment.

7 - The celebrated textbook example found in the AI is then simply written as
follows:

    weak with Departments;
    package Employees is
        type Dept_Ptr is access all Departments.Department'Class;
        type Employee is tagged private;
        procedure Assign_Employee(E : in out Employee;
                                  D : in out Departments.Department);
        ...
        function Current_Department(D : in Employee) return Dept_Ptr;
    end Employees;

    weak with Employees;
    package Departments is
        type Emp_Ptr is access all Employees.Employee'Class;
        type Department is tagged private;
        procedure Choose_Manager(D : in out Department;
                                 Manager : in out Employees.Employee);
        ...
    end Departments;

8 - Marketing hype a la Tucker:

    - No three-part types
    - No new compilation units
    - No new declarations
    - No changes to the visibility rules
    - Little new syntax
    - Support for circularities involving types in nested packages

I am no expert in source-based compilers, but I believe that this model should
be quite close to what GNAT currently does for "with type".  Superficially
adding a unit to the environment would merely mean that the source file exists,
and the "weak with" would give the compiler permission to peak at it.

For library-based compilers this proposal adds some complexity, but not
necessarily more than those proposals which change the visibility rules or
otherwise break the invariants upon which the compilers depend.  In the case of
our implementation a tree in the library can be in two states: parsed (as
produced by the parser, no names resolved) or analyzed (with all the decorations
produced by the semantic analyzer, names resolved, etc.).  These states would
correspond directly to a unit superficially/fully entered in the environment.
We would need to beef up the parsed representation a bit to add (1) tables
listing the types and package names, and (2) a hash code representing the
sequence of tokens in the unit (to perform the check of #6 above).  Intense
hand-waving here, as I have not done anything resembling a serious design.

Feel free to explain why this is a dumb idea...

**************************************************************

From: Arnaud Charlet
Sent: Monday, February  3, 2003  5:35 AM

I certainly like Pascal's proposal (weak with) very much. It is the only
proposal (apart from the old with type that had its bunch of issues) so far
that looks appealing to me. It addresses well the issue of interfacing
with other languages and from a user point of view, it is very close to an
ideal solution.

As Robert said, we don't have any real technical constraint in GNAT to
implement any of the proposed solution. This one would probably be
close to the current with type implementation.

I am not sure the wording about adding a unit in the environment, and
requiring compilers to have two (or more for that matter) phases should be
stated as such, but I guess this can probably be sorted out without too much
difficulty if this proposal gets some more supporters.

**************************************************************

From: Robert Dewar
Sent: Monday, February  3, 2003  6:39 AM

Pascal's proposal for weak with is the first one I have seen since WITH TYPE
that I like, and in fact I like it better than WITH TYPE (I am not taling
deep semantics here, just a surface impression). It also seems to satisfy
the requirement for easy generation of equivalent Ada from e.g. Java stuff.

**************************************************************

From: Robert Dewar
Sent: Monday, February  3, 2003  6:42 AM

By the way I don't like "weak" very much because it has the wrong
connotations to be. The issue is that this with is preliminary or
limited (so if you are in a mood to reuse keywords.

limited with x;

is more like it ...

A more radical proposal is that all WITH's that meet the criteria are
weak -- could that work?

**************************************************************

From: Pascal Leroy
Sent: Monday, February  3, 2003  6:51 AM

I am not wedded to a particular syntax.  If we think that the idea is worth
pursuing, we can probably come up with an acceptable syntax (we don't lack
creativity in the area of syntax).

> A more radical proposal is that all WITH's that meet the criteria are
> weak -- could that work?

From a methodological standpoint, I'd say that introducing a circularity should
be the result of a conscious decision, not something that happens by chance.
So the user should have to make it explicit that a circularity is what she
wants--by typing "weak", "limited" or whatever.

**************************************************************

From: Robert Dewar
Sent: Monday, February  3, 2003  7:03 AM

I am not so sure I agree with this. At one level of abstraction you WITH
something because you need something in it, and it is not really your
business if it needs something in yourself.

**************************************************************

From: Jean-Pierre Rosen
Sent: Monday, February  3, 2003  7:02 AM

> limited with x;
>
> is more like it ...
>
Or even
   with X abstract;

;-)

**************************************************************

From: Robert A. Duff
Sent: Monday, February  3, 2003  8:18 AM

> A while back, Dan wrote, as a side comment: "Well, the nicest solution for the
> user is just to allow specs to with each other, as other languages
> do."

Dan was exactly correct.  That's why I can't get too excited about
Tucker's aesthetic gut feelings about the various other proposals --
they are *all* less aesthetically pleasing than the "right" solution.

>...This
> got me thinking, as this is an approach that we have discussed briefly once or
> twice, but writers of library-based compilers (like me) started to yell and
> scream that it would be impossible to implement.  Well, I am starting to think
> that all approaches considered so far are nearly impossible to
>implement, ...

That seems exaggerated, ...

>...and
> furthermore are quite hard to use.  So I started to investigate a solution where
> units are allowed to with each others.  Here are my half-baked ideas:
>
> 1 - A new form of context clause is added:
>
>     weak [private] with library_unit_name {, library_unit_name}
>
> Weak with clauses don't have to form an acyclic graph.

This is the solution I chose for my from-scratch language design I do as
a hobby.  My syntax is "import X;" to import X and require X to be
elaborated earlier than this unit -- essentially the same as Ada's "with
X;" And "import forward X;" to import X and elaborate X *after* this
unit.  This is essentially the same as your "weak with X;".  ("import
forward" is like a forward reference.)

The only thing needed in the RM is the syntax for weak with's, make sure
the rule about no cycles applies only to cycles formed by strong with's.

All the stuff below is really a description of how the Rational compiler
would work (and others with a similar library model).  There's no need
for any of it in the RM.

(Aside: In my "hobbyist" language, I am able to achieve complete
*compile-time* checking of elaboration order problems, even in the case
of indirect/dispatching calls, with no sacrifice in expressive power.
I thought that was pretty cool.)

> 2 - A compilation unit is added to the environment in two phases.  During the
> first phase we say that the unit is "superficially added"; during the second
> phase that it is "fully added" (better terminology welcome).  How this is
> achieved is not specified by the language.
>
> 3 - To superficially add a unit to the environment, there is no requirement that
> any other unit already exists in the environment.  (Not even its parent, heck,
> not even Standard!)
>
> 4 - When a unit is superficially added to the environment, the only names that
> can be referenced from outside the unit are those of packages (both the unit
> itself and nested packages) and of types.  Moreover, every type is at this point
> considered to be an incomplete type, and subject to the restrictions mentioned
> in 3.10.1; the taggedness of the types is visible.  (The alert reader may notice
> some similarity between a unit superficially entered into the environment and a
> package abstract.)
>
> 5 - To fully add a unit to the environment, all of the units upon which it
> depends semantically must already have been fully entered into the environment.
> Furthermore, all of the units that it names in a weak with clause must already
> have been superficially entered into the environment.  The only names that can
> be referenced from such units are those of types and packages, and types are
> considered incomplete.
>
> 6 - To fully add a unit to the environment, this unit must be made of the same
> lexical elements (except for comments) as when it was superficially added to the
> environment.

Note in particular that this consistency check is an artifact of your
compilation model.  That is, your compiler is going to read the same
file twice.  But I can imagine compilers that will only read the source
once, so the "superficial" and "full" distinction is represented purely
in memory during compilation.

I think any consistency requirements are already covered by the RM's
words about not including two different versions of the same unit in a
partition.

> 7 - The celebrated textbook example found in the AI is then simply written as
> follows:
>
>     weak with Departments;
>     package Employees is
>         type Dept_Ptr is access all Departments.Department'Class;
>         type Employee is tagged private;
>         procedure Assign_Employee(E : in out Employee;
>                                   D : in out Departments.Department);
>         ...
>         function Current_Department(D : in Employee) return Dept_Ptr;
>     end Employees;
>
>     weak with Employees;
>     package Departments is
>         type Emp_Ptr is access all Employees.Employee'Class;
>         type Department is tagged private;
>         procedure Choose_Manager(D : in out Department;
>                                  Manager : in out Employees.Employee);
>         ...
>     end Departments;

Yes, although only one of the with's *needs* to be weak.  For symmetry,
they probably both should be weak.

> 8 - Marketing hype a la Tucker:
>
>     - No three-part types
>     - No new compilation units
>     - No new declarations
>     - No changes to the visibility rules
>     - Little new syntax
>     - Support for circularities involving types in nested packages
>
> I am no expert in source-based compilers, but I believe that this model should
> be quite close to what GNAT currently does for "with type".  Superficially
> adding a unit to the environment would merely mean that the source file exists,
> and the "weak with" would give the compiler permission to peak at it.
>
> For library-based compilers this proposal adds some complexity, but not
> necessarily more than those proposals which change the visibility rules or
> otherwise break the invariants upon which the compilers depend.  In the case of
> our implementation a tree in the library can be in two states: parsed (as
> produced by the parser, no names resolved) or analyzed (with all the decorations
> produced by the semantic analyzer, names resolved, etc.).  These states would
> correspond directly to a unit superficially/fully entered in the environment.
> We would need to beef up the parsed representation a bit to add (1) tables
> listing the types and package names, and (2) a hash code representing the
> sequence of tokens in the unit (to perform the check of #6 above).  Intense
> hand-waving here, as I have not done anything resembling a serious design.
>
> Feel free to explain why this is a dumb idea...

Because your evil twin was yelling and screaming about the
implementation difficulty of this idea...  ;-)

**************************************************************

From: Tucker Taft
Sent: Monday, February  3, 2003  8:24 AM

I like this, although it is
more work for us than almost any
other proposal.  We do all semantics
on the fly, and don't have a mode that
parses without doing semantic analysis.

In general, I believe the "type C.T;" approach
solves the basic problem, and will be one of the
easiest to implement.

As far as syntax, I like the suggestion of using
"limited" as a prefix on the "with" clause,
and it goes awfully nicely with "private" ;-).

    [limited] [private] with library_item_name {, library_item_name};

It would be a major headache (dare I say "huge" headache?)
for us if the "limited"ness of the "with" were not explicit.

**************************************************************

From: Robert A. Duff
Sent: Monday, February  3, 2003  8:34 AM

Robert wrote:

> A more radical proposal is that all WITH's that meet the criteria are
> weak -- could that work?

No, I don't think so.  The problem is that if there's a cycle of with's,
the compiler needs to choose *one* of them as "weak" to break the
cycle.  The one that is chosen affects the elaboration order.  Clearly,
the user needs a way to control this order, and "weak with" is that
feature.

In other words, how is the compiler to know *which* with's meet the
criteria?

I don't think the usual army of elaboration-control pragmas can help.
Pragma Elaborate(X) should probably be illegal for weak withs.
Anyway, it only goes one step, which as we know from Ada 83 is not good
enough.  Pragma Elaborate_All should ignore weak withs that are found in
the chain.  Otherwise, it can't work when used in part of the cycle.

Lots of other languages do as Robert suggests -- just allow cyclic
imports, with no indication of which ones are "weak".  The reason they
can get away with it, and we can't is:
Some such languages don't have run-time elaboration semantics,
so there's no issue.
Other such languages have botched the run-time elaboration semantics
so badly that this issue is in the noise.

> I am not so sure I agree with this. At one level of abstraction you WITH
> something because you need something in it, and it is not really your
> business if it needs something in yourself.

I agree with Pascal on this point.  It seems to me that a mutually
recursive design always requires *some* mutually recursive knowledge on
the part of the designer.  I don't see it happening by accident.  At
least not unless the packages involved are incoherent collections of
unrelated stuff.

Normal layered "with"s have the property that packages don't need to
know about their clients.  Mutual recursion seems rather different to
me: two packages need to know about each other -- they at least need to
know about each other's *existence*.  So I think weak with's deserve
their own syntax.

I guess I agree with Robert that "weak with" is not the best syntax,
but that's a minor detail.  Using "with" as a verb is already an
annoyance...  Oh, well.

**************************************************************

From: Robert Dewar
Sent: Monday, February  3, 2003  8:43 AM

> No, I don't think so.  The problem is that if there's a cycle of with's,
> the compiler needs to choose *one* of them as "weak" to break the
> cycle.  The one that is chosen affects the elaboration order.  Clearly,
> the user needs a way to control this order, and "weak with" is that
> feature.
>
> In other words, how is the compiler to know *which* with's meet the
> criteria?

Yes, that makes sense, so that's not a possibility that is worth discussing.
I withdraw the solution.

**************************************************************

From: Pascal Leroy
Sent: Monday, February  3, 2003  8:37 AM

> The only thing needed in the RM is the syntax for weak with's, make sure
> the rule about no cycles applies only to cycles formed by strong with's.
>
> All the stuff below is really a description of how the Rational compiler
> would work (and others with a similar library model).  There's no need
> for any of it in the RM.

That statement surprises me.  At the very least, it would seem that the RM would
need to specify what names/entities are available after a unit has been
superficially entered into the environment, and what are the properties of these
entities.  I am proposing that you can see only packages and types, and even so
the types would look like they are incomplete.  I can't see how you could deduce
this from the current RM.

PS: But then I know that Bob Duff, like the Pope, is infallible in matters of
language doctrine ;-)

**************************************************************

From: Robert A. Duff
Sent: Monday, February  3, 2003  8:51 AM

Yes, you're right -- the RM needs to say *something*.  But it's not
much.  And I don't think it needs to define this concept of
"superficially in the environment" -- that concept belongs in the
Rational user's manual, I think.

The RM should say something about how if the only visibility to package
X is via "weak with", then uses of X are restricted along the lines you
say above.

More thought is needed, if we are to go this route.  The only real
problem I see is implementation difficulty.  I think all the language
rules can be worked out, but that will require some thought.
Interactions with elaboration-control pragmas.  What happens when you've
got both a weak and a strong with on the same thing?  Does "weak with
A.B.C;" import all three weakly?  Is there any concern that changing
"with" to "weak with", or vice versa, would introduce surprises into the
program?  I suspect all these questions have easy straightforward
answers.

> PS: But then I know that Bob Duff, like the Pope, is infallible in matters of
> language doctrine ;-)

;-)

No way!

**************************************************************

From: Robert A. Duff
Sent: Monday, February  3, 2003  12:56 PM

In Pascal's proposal, a parent package P can weakly 'with' its children:
"weak with P.C;", if you want a mutually dependent pair of types,
one in P and one in P.C.  The spec of P will be elaborated before the
spec of P.C, as usual.

However, it is pointless for P.C to say "weak with P;", because the
child-to-parent dependency is necessarily strong (given that there's no
proposal to add syntax saying otherwise -- "weak children?").

No problems here.  I'm just making some observations.

I guess I favor Robert's proposed "limited with" syntax.

**************************************************************

From: Robert Eachus
Sent: Monday, February  3, 2003  2:26 PM

So do I, and I am beginning to like this proposal.  As far as I can see
it is clearly better than package abstracts in two ways, there is no new
compilation unit type, and there is no problem of keeping the abstract
consistant with the package itself.

In other words from a theoretical point of view, the "limited with"
creates a package abstract and then withs it.  Since the completion of
the types and the subprograms (and parameters) get lost in the process,
there is no visible circularity.

But I think that there are two "gotchas" that have to be dealt with.
 The first is where you have a type declaration that depends on another
type:

type Color is (Red, Green, Blue);
type Properties(C: Color) is...;

Does the incomplete type for Properties include the discriminant?
 Choices are yes, and it is illegal to do anything with it, type Color
is not incomplete, and the type Properties has undefined discriminants.
(Others?)  I prefer the choice where only private, record and tagged
types are incomplete in the limited view.

This brings me directly to the second gotcha.  I think it is clear (at
least to me) that for this to be useful, the limited view of an access
type has to be an access type.  But there have to be limits on how this
access type can be used in the limited view.  (Translation, if you
create a type with a component of  (a limited view of) an access type,
then it is illegal to have a non-null default value for a component of
that type.  Other rules are possible.)

**************************************************************

From: Robert A. Duff
Sent: Monday, February  3, 2003  2:26 PM

> This brings me directly to the second gotcha.  I think it is clear (at
> least to me) that for this to be useful, the limited view of an access
> type has to be an access type.  But there have to be limits on how this
> access type can be used in the limited view.

Can't we say that it's an access type whose designated type is
incomplete?  And then you can't do things like X.all.

>...  (Translation, if you
> create a type with a component of  (a limited view of) an access type,
> then it is illegal to have a non-null default value for a component of
> that type.  Other rules are possible.)

I think you can't have an allocator (as a default or anything else),
but I don't see why it couldn't be a function call that happens to
return the access type.  Perhaps an example?

**************************************************************

From: Dan Eilers
Sent: Monday, February  3, 2003  3:04 PM

I like Pascal's "weak with" proposal a lot, although I'm not yet convinced
we need to syntactically distinguish "weak with" from normal withs.

I came to this conclusion by perhaps a slightly different route:

1) we have a bunch of proposed syntax extensions to solve the circular
   types problem, any of which are believed to be feasible to implement;

2) essentially none of the proposed syntax extensions have any dual-use
   benefit towards the solution of any other problem;

3) we envision a tool that can automatically translate circular specs
   written in other languages into the proposed new syntax;

4) a similar tool could also translate Ada that allowed circular with's
   into any of the proposed syntax extensions;

5) given such a tool, the users would happily write Ada with circular
   with's, and use the tool to create their package abstracts (in
   whatever syntax);

6) this could be made automatic.  When the compiler sees a "with" clause
   for a unit that has not been compiled, instead of giving an error, it
   can automatically invoke the translation tool to create the package
   abstract, and then compile the abstract before continuing to compile
   the original unit.

6) this can be further optimized if desired so the translation tool is
   integrated into the compiler, rather than needing to be a stand-alone
   tool.


Notes:

Recursive subprogram calls don't use different syntax than non-recursive
ones, so ideally recursive with's wouldn't either.

We need to address the elaboration order concerns, but presumably that
is more naturally done with elaboration pragmas than with different
syntax for "with" clauses.

I still think there is some possible dual-use synergy with the separate
private issue.  As Tuck corrected me, you can't just put the private
part in a separate file.  So instead of "private is separate" you would
have to say "the remainder of this package is separate".  A "with" clause
on such a package would continue to see both the root and the subunit.
A "with p'abstract" would see only the root.  I believe such new syntax
would at the same time solve the separate private issue and the circular
types issue, and perhaps also the separate constants issue.

**************************************************************

From: Robert Dewar
Sent: Monday, February  3, 2003  4:07 PM

> 6) this could be made automatic.  When the compiler sees a "with" clause
>    for a unit that has not been compiled, instead of giving an error, it
>    can automatically invoke the translation tool to create the package
>    abstract, and then compile the abstract before continuing to compile
>    the original unit.

That won't do at all for a source based compilation system, there is no
concept of "a unit that has not been compiled". One of the critical
guarantees in GNAt is that order of compilation NEVER affects the results
and we would strenuously object to a change in semantics which would
destroy this guarantee. For one thing it means that the meaning of a
program is not fully conveyed by its sources, and that is pragmatically
and philsophically unacceptable.

**************************************************************

From: Robert A. Duff
Sent: Monday, February  3, 2003  4:16 PM

Yes, I agree.

This makes me think that these tools that translate from other languages
(or create bindings thereto, I guess) ought to make sure that their
generated Ada code doesn't do anything at elab time -- at least in the
presence of cycles.  That seems achievable, and simpler than having user
options to specify elab order.

**************************************************************

From: Robert Dewar
Sent: Monday, February  3, 2003  4:25 PM

Note that another objection is that this means that automated tools for
determining the compilation order would have to do semantic analysis.

**************************************************************

From: Robert A. Duff
Sent: Monday, February  3, 2003  4:12 PM

Dan Eilers wrote:

>...
> 3) we envision a tool that can automatically translate circular specs
>    written in other languages into the proposed new syntax;

Right, but those languages typically don't have the same
elaboration-order issues that Ada does.

> 4) a similar tool could also translate Ada that allowed circular with's
>    into any of the proposed syntax extensions;

True, but...

> 5) given such a tool, the users would happily write Ada with circular
>    with's, and use the tool to create their package abstracts (in
>    whatever syntax);

I fear that such a tool would be inherently error prone.  Suppose we
have package specs A and B, which 'with' each other.  The tool would
have to choose which one to elaborate first, and it might guess wrong.
I think it's better to let the user specify the order via "limited
with".

Besides, now that I think about it, how would this tool work?  "Limited
with" implies a very restricted usage of the imported package.  The tool
would have to check that such restrictions are obeyed before turning a
"with" into a "limited with".

> 6) this could be made automatic.  When the compiler sees a "with" clause
>    for a unit that has not been compiled, instead of giving an error, it
>    can automatically invoke the translation tool to create the package
>    abstract, and then compile the abstract before continuing to compile
>    the original unit.

Users normally don't compile.  They run a build tool, which
automatically compiles everything that needs to be compiled, in some
order that is not easily known to the programmer.  It seems error-prone
to have the semantics depending on what happens to have already been
compiled.  That's true even if the user bypasses the build tool and
explicitly invokes the compiler in some order.

> 6) this can be further optimized if desired so the translation tool is
>    integrated into the compiler, rather than needing to be a stand-alone
>    tool.
>
>
> Notes:
>
> Recursive subprogram calls don't use different syntax than non-recursive
> ones, so ideally recursive with's wouldn't either.

True, but this case seems different to me.

If I see:

    with B;
    package A is ...

I currently know for certain that the spec of B is elaborated before A.
I don't have to go looking at B to see that that's true.  I don't think
we should break that property.

> We need to address the elaboration order concerns, but presumably that
> is more naturally done with elaboration pragmas than with different
> syntax for "with" clauses.

I don't think the existing pragmas can be used for this purpose.
Suppose the cycle is "A with's B with's C with's A".  Adding a pragma
Elaborate in A pointing to B doesn't work, because it's not transitive
to C.  Adding Elaborate_All doesn't work, because that would require A
to be elaborated before itself (because it's transitive).

So we would have to add a new pragma to explicitly break one link in the
cycle:

    with X; pragma Don't_Elaborate(X); -- ;-)

But that means precisely what "limited with X;" would mean.

Elaboration pragmas were a mistake in the first place.  They were added
late in the game to Ada 83 to solve the "Brosgol anomalies".  We added
more pragmas in Ada 95, and I'm afraid we botched the job -- we *still*
didn't solve the problems.  I suggest we avoid compounding the mistake
of using pragmas to control elaboration.

Furthermore, "limited with X" implies various restrictions on the use of
X -- you can only refer to types in X, and they are all incomplete (or
access to incomplete).  Pragmas shouldn't be in *that* business.

> I still think there is some possible dual-use synergy with the separate
> private issue.  As Tuck corrected me, you can't just put the private
> part in a separate file.  So instead of "private is separate" you would
> have to say "the remainder of this package is separate".  A "with" clause
> on such a package would continue to see both the root and the subunit.
> A "with p'abstract" would see only the root.  I believe such new syntax
> would at the same time solve the separate private issue and the circular
> types issue, and perhaps also the separate constants issue.

Perhaps, but I skept.  I'm having trouble seeing the separate-private
issue as anything more than a simple textual translation, with no
run-time consequences whatsoever.  I haven't seen a fully worked-out
proposal that solves both problems...

P.S. This discussion makes me think of another observation.  If a given
cycle contains two or more "limited with"s, then elaboration order will
presumably be implementation defined.  I suppose that's OK -- if you
care, you can break the cycle with just one "limited with", and use
normal "with" for the rest.

**************************************************************

From: Dan Eilers
Sent: Monday, February  3, 2003  6:23 PM

Bob Duff wrote:

> Perhaps, but I skept.  I'm having trouble seeing the separate-private
> issue as anything more than a simple textual translation, with no
> run-time consequences whatsoever.  I haven't seen a fully worked-out
> proposal that solves both problems...

OK, here's the canonical example worked out:

-- Note: with'ing package P creates a semantic dependency on all subunits
-- such as P.Extension declared in P, and makes all declarations in
-- P.Extension visible as if they were declared directly in P.

-- with'ing P'abstract does not create a semantic dependency on subunits
-- such as P.Extension, and does not make p.Extension's declarations visible,
-- and does not allow declaring objects of types from P whose full type is
-- deferred to P.Extension.

    package Employees is
        type Employee is private;       -- completed in subunit
        type Emp_Ptr is access all Employee;
        package Extension is separate;
    end Employees;

    package Departments is
        type Department is private;
        type Dept_Ptr is access all Department;
        package Extension is separate;
    end Departments;

    with Departments'abstract;
    separate(Employees)
    package Extension is
        type Emp_Ptr is access all Employee;
        procedure Assign_Employee(E : access Employee;
                                  D : access Departments.Department);
        function Current_Department(D : access constant Employee) return
            Departments.Dept_Ptr;
    private
        type Employee is record
           dept: Departments.Dept_Ptr;
        end record;
    end Extension;

    with Employees'abstract;
    separate(Departments)
    package Extension is
        type Dept_Ptr is access all Department;
        procedure Choose_Manager(D : access Department;
                                 Manager : access Employees.Employee);
    private
        type Department is
           mgr: Employees.Emp_Ptr;
        end record;
    end Extension;


For the "private is separate" problem, instead of:

    package p is
      ...
    private is separate;
    end P;

you would have:

    package p is
      ...
    private
      package extension is separate;
    end;

**************************************************************

From: Robert A Duff
Sent: Tuesday, February 11, 2003  6:07 PM

Dan Eilers reminded me in private e-mail to look at this e-mail he sent
a week ago.

It does seem like a reasonable proposal, and it does seem like it allows
the full type for a private type to go in a separate file.

I presume the rules for completing private types would be relaxed to
allow the full type in a package subunit.

I presume that if you say "with P'abstract", then P.T is an incomplete
type (view).

This proposal seems very similar to the one Tucker proposed a while ago,
where package P declares that it has a child C.  The syntax here is more
"subunit like" rather than "child like".

Dan wrote:

> Bob Duff wrote:
>
> > > I still think there is some possible dual-use synergy with the separate
> > > private issue.  As Tuck corrected me, you can't just put the private
> > > part in a separate file.  So instead of "private is separate" you would
> > > have to say "the remainder of this package is separate".  A "with" clause
> > > on such a package would continue to see both the root and the subunit.
> > > A "with p'abstract" would see only the root.  I believe such new syntax
> > > would at the same time solve the separate private issue and the circular
> > > types issue, and perhaps also the separate constants issue.
> >
> > Perhaps, but I skept.  I'm having trouble seeing the separate-private
> > issue as anything more than a simple textual translation, with no
> > run-time consequences whatsoever.  I haven't seen a fully worked-out
> > proposal that solves both problems...
>
> OK, here's the canonical example worked out:
>
> -- Note: with'ing package P creates a semantic dependency on all subunits
> -- such as P.Extension declared in P, and makes all declarations in
> -- P.Extension visible as if they were declared directly in P.
>
> -- with'ing P'abstract does not create a semantic dependency on subunits
> -- such as P.Extension, and does not make p.Extension's declarations visible,
> -- and does not allow declaring objects of types from P whose full type is
> -- deferred to P.Extension.

By "not allow declaring objects" I presume all the limitations of
incomplete types are applied.  You can declare access-to-it, and you can
declare parameters (if it's tagged), I guess.

>     package Employees is
>         type Employee is private;       -- completed in subunit
>         type Emp_Ptr is access all Employee;
>         package Extension is separate;
>     end Employees;
>
>     package Departments is
>         type Department is private;
>         type Dept_Ptr is access all Department;
>         package Extension is separate;
>     end Departments;
>
>     with Departments'abstract;
>     separate(Employees)
>     package Extension is
>         type Emp_Ptr is access all Employee;

I don't see why you want to declare Emp_Ptr here.

>         procedure Assign_Employee(E : access Employee;
>                                   D : access Departments.Department);
>         function Current_Department(D : access constant Employee) return
>             Departments.Dept_Ptr;
>     private
>         type Employee is record
>            dept: Departments.Dept_Ptr;
>         end record;
>     end Extension;
>
>     with Employees'abstract;
>     separate(Departments)
>     package Extension is
>         type Dept_Ptr is access all Department;
>         procedure Choose_Manager(D : access Department;
>                                  Manager : access Employees.Employee);
>     private
>         type Department is
>            mgr: Employees.Emp_Ptr;
>         end record;
>     end Extension;

Hmm.  The existing language allows:

    package P is
        type T is private;

        package Q is
        private
            type T is new Integer;
        end Q;
    private
        type T is new Boolean;
    end P;

And the type Q.T has nothing to do with P.T.  I'm not sure how the rules
would deal with this sort of thing.  Exactly when is a full type
considered to complete a private type, and how do we prevent duplicate
completions.

The current language always completes things in the same declarative
region, so this would add the complexity of completing things
elsewhere.

Well, now that I think about it, I take that back, somewhat: An
accept_statement can be more nested than the corresponding entry, and
there can be more than one of them.  So that's a similar situation.

> For the "private is separate" problem, instead of:
>
>     package p is
>       ...
>     private is separate;
>     end P;
>
> you would have:
>
>     package p is
>       ...
>     private
>       package extension is separate;
>     end;

Seems plausible, although I'm still not sure we need to combine the
solutions to these two problems.  After all, the separate-private as a
purely textual translation seems OK.  Along with the "private with"
idea.

**************************************************************

From: Dan Eilers
Sent: Tuesday, February 11, 2003  7:40 PM


Bob Duff wrote:
> It does seem like a reasonable proposal, and it does seem like it allows
> the full type for a private type to go in a separate file.
...
> This proposal seems very similar to the one Tucker proposed a while ago,
> where package P declares that it has a child C.  The syntax here is more
> "subunit like" rather than "child like".

Yes, the two proposals are very similar.  They both involve an
incomplete type whose completion is deferred to a child or stub
that is mentioned in the parent.

The advantages to using subunits are:

1)  the subunit proposal leaves the clients unchanged.  The child
    proposal requires library-level renames in order to leave the
    clients unchanged;

2)  it is natural in Ada for a parent to declare a subunit,
    but not for a parent to declare a child unit;

3)  the subunit proposal serves a dual purpose in solving the
    need for separately compiled private parts;

The advantage to using child units is:

1)  the subunit proposal requires new syntax for "with" clauses,
    to indicate no semantic dependence on any subunits.


> Seems plausible, although I'm still not sure we need to combine the
> solutions to these two problems.  After all, the separate-private as a
> purely textual translation seems OK.  Along with the "private with"
> idea.

This "purely textual translation" is essentially "pragma include"
from pre-Ada83 days.  It was excluded from Ada in favor of subunits
because:

    1) subunits often need their own context clauses, which include files
          don't provide;
    2) subunits specify their parent and can only have one parent;
    3) subunits are well-formed, and can be compiled independently
         from their parent

These considerations also apply to separately compiled private parts,
making subunits a much better solution than the combination of two
hacks: "pragma include" and "private with".

**************************************************************

From: Tucker Taft
Sent: Monday, February  3, 2003  4:27 PM

Robert A Duff wrote:
>
> > This brings me directly to the second gotcha.  I think it is clear (at
> > least to me) that for this to be useful, the limited view of an access
> > type has to be an access type.  But there have to be limits on how this
> > access type can be used in the limited view.
>
> Can't we say that it's an access type whose designated type is
> incomplete?  And then you can't do things like X.all.

You are starting to give me the "willies."
We will have to process rep-clauses if the access type
in a limited-with-ed package is not itself considered an
incomplete type.

If we want a non-incomplete access type, then I think we need to
declare it in a non-limited-withed place.  Hence,
this doesn't solve the problem associated with proliferation
of access type declarations.

And I do still like that "type C.T;" ;-).
(what a pain I can be...)

> >...  (Translation, if you
> > create a type with a component of  (a limited view of) an access type,
> > then it is illegal to have a non-null default value for a component of
> > that type.  Other rules are possible.)
>
> I think you can't have an allocator (as a default or anything else),
> but I don't see why it couldn't be a function call that happens to
> return the access type.  Perhaps an example?

Gack.  You are back to requiring that the limited-with-er chooses
a representation for a type that it isn't declaring.
In this era where we are on the cusp of moving from 32-bit pointers
to 64-bit pointers, I fear that rep clauses on access types may
be more common, rather than less.



By the way, I have discovered (remembered?) that we have a "loose parser"
built into our compiler.  It's job is to do lookahead where strict one-pass
semantic analysis is difficult.  For example, it scans bodies looking for
labels, and then inserts them at the "begin."  It scans generic formal
parts looking ahead until it finds out in what parent unit the generic
is declared, so it can interpret the names in the formal part properly.
In any case, this is a full, but very stupid and quite "lax" parser.
It could probably build up a tree of package and type names.  It could
certainly not make any semantic sense out of a rep clause, and since the
rep clause could use named numbers declared in other packages
(e.g. "for My_Acc_Type'Size use Targ_Dep.Size_Of_Access_Type;"),
it really couldn't be made to work.

So if we consider this limited-with approach, I think we want to make
minimal assumptions about syntactic processing, and zero assumptions
about representation processing, of the items in the limited-withed unit.

I also agree with Bob that the less said about the way the magic
happens the better.  Clearly there will be some kind of compilation
dependence on the source of the limited-withed unit, but the key point is there
is no semantic or elaboration dependence on the unit.  It seems fine if
the source of the unit is edited "after" something that mentioned it in a limited-with
clause is compiled, but before the unit itself is compiled.  But of course,
whenever the source is changed, everything that has a compilation-dependence
on it will probably need to be recompiled prior to linking.

The "trick" that makes it all possible is that the *fully compiled* representation of
both units in a cyclic dependence depend on the *source text* of both units.
The fully compiled representations don't both depend on the others fully compiled
representation.  Of course, there may be intermediate forms between source
text and fully compiled representations, but those don't need to be mentioned.
And the other important point is there is no transitive compilation dependence
if the unit mentioned in a limited-with clause itself depends on other
units, via withs or limited-withs.

Presumably, mutual inlining, if supported, requires similar tricks.

Other observations if we consider limited-with:

I suppose to be consistent, we should interpret "limited with P.Q.R;"
as equivalent to "limited with P, P.Q, P.Q.R;".  Also it seems clear
that limited with is ignored if there is a non-limited with for the
same unit.

Another important point is that if a unit is mentioned in a limited with
clause, it is a *post*-compilation rule that the unit be semantically
(or even syntactically) correct, as opposed to a legality rule.  When compiling
the unit having the limited with, the compiler may of course check some part of
this post-compilation rule early (provided of course it doesn't create an inappropriate
semantic dependence), but an ACATS test should not require that any particular
errors in the unit mentioned in the limited-with be identified at this time.
Certainly compilers will vary in how picky is the parser that they
use to implement limited with, especially given that Ada's "syntax"
rules slop over into semantic processing fairly frequently, due to the
lack of direct LALR(1) parsability.

The only requirement should be that *if* the unit is semantically correct,
then you can safely mention it in a limited-with.  But you cannot rely
on the compiler detecting all (or any) errors in a unit mentioned in a limited-with,
though it can be arbitrarily picky about it.  Eventually, prior to linking,
it will require that the unit exist and be semantically correct.  This
means that when a unit mentions a second unit in a limited-with
clause, the second unit is "needed by" the first unit, in the
sense of 10.2(2-6).

The above is related to 10.1.4(5) which says:

   When a compilation unit is compiled, all compilation units upon which
   it depends semantically shall already exist in the environment...

We could change this to "... upon which it depends semantically or
which it mentions in a limited_with_clause shall ..." and it would still be OK.

However, 10.1.4(6) which says:

   The implementation may require that a compilation unit be legal before
   inserting it into the environment.

is too strict.  It could be changed to:

   The implementation may require that a compilation unit satisfy the Syntax
   Rules before inserting it into the environment.  The implementation may
   require that a compilation unit be legal before allowing it to be mentioned
   in a with_clause other than a limited_with_clause.

**************************************************************

From: Robert A Duff
Sent: Monday, February  3, 2003  5:17 PM

Tucker wrote:

> Robert A Duff wrote:
> > Can't we say that it's an access type whose designated type is
> > incomplete?  And then you can't do things like X.all.
>
> You are starting to give me the "willies."
> [...description of willies]

OK, I withdraw my suggestion.

But it seems like whatever solution to the mutual recursion problem we
choose, we really need to solve the problem of proliferation of silly
type conversions.  Otherwise, these proposals will be nearly unusable.

That is, there ought to be an *implicit* conversion between access types
in the "safe" cases.  I know we discussed that before.  Is it part of
any of these proposals, or is it a separate AI?  I seem to recall that
it might be wrapped up with other access-type issues, like allowing "not
null" constraints, and access-to-constant parameters.

> And I do still like that "type C.T;" ;-).
> (what a pain I can be...)

That's a fine solution, IMHO.  Your earlier e-mail about
mutually-recursive types being part of the same abstraction
makes me comfortable with the idea of tying the solution to
child packages.  (See, it *is* possible to convey sensible information
about purely aesthetic concerns.  ;-))

But I would choose the "limited with" idea, unless it is considered too
hard to implement.

> Presumably, mutual inlining, if supported, requires similar tricks.

And mutual generic instantiation?

> Other observations if we consider limited-with:
>
> I suppose to be consistent, we should interpret "limited with P.Q.R;"
> as equivalent to "limited with P, P.Q, P.Q.R;".  Also it seems clear
> that limited with is ignored if there is a non-limited with for the
> same unit.

Or any other "strong" dependence, such as child upon parent.

I guess "strong dependence", as I've been calling it, is exactly the
same notion as "semantic dependence".

> Another important point is that if a unit is mentioned in a limited with
> clause, it is a *post*-compilation rule that the unit be semantically
> (or even syntactically) correct, as opposed to a legality rule. ...

I don't see the need for any verbiage in the RM about this point.
And I don't see how this differs (in theory) from normal with_clauses.
(It might differ in practise, depending on how the compiler actually
works.)

I mean, if B says "with A;", there is currently no rule saying "A shall
be legal."  A compiler could properly say nothing more than "no errors
found in B", even if there are errors in A.  (That would be unfriendly,
IMHO.)  Of course, sometime before running the program, A must be
compiled, and the errors detected.

> ...This
> means that when a unit mentions a second unit in a limited-with
> clause, the second unit is "needed by" the first unit, in the
> sense of 10.2(2-6).

Yes, that's a key point.  It might be that the second unit is mentioned
*only* in limited_with_clauses, in which case there are no elaboration
dependendences upon it, but it still needs to be included in the
program, and elaborated at some point (possibly very late).

> The above is related to 10.1.4(5) which says:
>
>    When a compilation unit is compiled, all compilation units upon which
>    it depends semantically shall already exist in the environment...
>
> We could change this to "... upon which it depends semantically or
> which it mentions in a limited_with_clause shall ..." and it would still be OK.

I don't see any need to change this, given that "existing in the env" is
such a vague concept.  In GNAT, it just means the file is sitting there
on the disk.  In AdaMagic, it just means it is sitting there on the disk
and has been "registered".  Registration only does minimal syntax
checking.  In Rational's compiler, it means something more.

> However, 10.1.4(6) which says:
>
>    The implementation may require that a compilation unit be legal before
>    inserting it into the environment.
>
> is too strict.  It could be changed to:

I don't see why.  The GNAT model does not take advantage of this
permission.  The permission is there for the benefit of compilers like
Rational, where "inserting" involves running the compiler, and therefore
checking various rules.  I suppose AdaMagic takes partial advantage of
this permission, since insertion involves some small amount of syntax
checking.

But why should we care how much checking is done on B before "limited
with B" can be swallowed by the compiler?  If B is illegal, the compiler
can process "limited with B" if it likes, or (if it happens to notice
the illegality), it can complain that "limited with B" is nonsense.
It doesn't matter whether the illegality is syntactic or semantic.

>    The implementation may require that a compilation unit satisfy the Syntax
>    Rules before inserting it into the environment.  The implementation may
>    require that a compilation unit be legal before allowing it to be mentioned
>    in a with_clause other than a limited_with_clause.

I see no need for this added complexity.  And it could be a burden -- if
the compiler chooses to implement some simple Legality Rule in the
parser, that should be allowed.

**************************************************************

From: Tucker Taft
Sent: Monday, February  3, 2003  6:20 PM

Robert A Duff wrote:
> Tucker wrote:
> ...

>>The above is related to 10.1.4(5) which says:
>>
>>   When a compilation unit is compiled, all compilation units upon which
>>   it depends semantically shall already exist in the environment...
>>
>>We could change this to "... upon which it depends semantically or
>>which it mentions in a limited_with_clause shall ..." and it would still be OK.
>
> I don't see any need to change this, given that "existing in the env" is
> such a vague concept.  In GNAT, it just means the file is sitting there
> on the disk.  In AdaMagic, it just means it is sitting there on the disk
> and has been "registered".  Registration only does minimal syntax
> checking.  In Rational's compiler, it means something more.

I think we need to say *something* about the requirements
on a unit mentioned in a limited-with clause.  Why not say
it must exist in the environment?  As you said, that is
pretty vague.  If there are no requirements on it, then
the implementation has no place to "hang" whatever requirements
it really does have.  It can't just invent requirements that
don't exist.  But it can say, "as far as our implementation
is concerned, that unit doesn't exist in the environment,
so we refuse to compile the limited-with for it."

>>However, 10.1.4(6) which says:
>>
>>   The implementation may require that a compilation unit be legal before
>>   inserting it into the environment.
>>
>>is too strict.  It could be changed to:
>
> I don't see why.

Again, we need a place for the implementation to "hang"
its requirements.  We have given them the first handle,
by allowing them to require existence.  Now we are
giving them the second handle, allowing them to impose
certain other requirements on the unit.  They *clearly*
can't impose complete legality requirements, if the
units are inserted one at a time, since there is no
order of insertion that would allow that.

> ...
> The GNAT model does not take advantage of this
> permission.  The permission is there for the benefit of compilers like
> Rational, where "inserting" involves running the compiler, and therefore
> checking various rules.  I suppose AdaMagic takes partial advantage of
> this permission, since insertion involves some small amount of syntax
> checking.
>
> But why should we care how much checking is done on B before "limited
> with B" can be swallowed by the compiler?  If B is illegal, the compiler
> can process "limited with B" if it likes, or (if it happens to notice
> the illegality), it can complain that "limited with B" is nonsense.
> It doesn't matter whether the illegality is syntactic or semantic.

You are missing the point.  I am simply *allowing* compilers
to impose requirements, but only up to a point.
Without such verbiage, I don't believe compilers would be
allowed to impose any requirements.

I suppose you could argue all of this verbiage is unnecessary
already, but since we felt the need to put it there in the
first place, and we presumably aren't going to take it out,
I believe it needs to be updated to *allow* compilers to impose
certain requirements on limited-with'ed units.  But I definitely
don't want to force them to impose these requirements
(and I don't think my suggested wording does that).

>>   The implementation may require that a compilation unit satisfy the Syntax
>>   Rules before inserting it into the environment.  The implementation may
>>   require that a compilation unit be legal before allowing it to be mentioned
>>   in a with_clause other than a limited_with_clause.
>
> I see no need for this added complexity.  And it could be a burden -- if
> the compiler chooses to implement some simple Legality Rule in the
> parser, that should be allowed.

I'm not sure what we are discussing any more, since you
don't seem to think that the unit even need to exist
in the environment to be mentioned in the limited-with
clause.  I think we need to *allow* implementations
to impose some requirements, and we might as well tie
it to the notion of "exist" in the environment.
*If* we tie it to that notion, we want to be sure that
limited-with can in fact be used.  That seems to mean
that you must be able to insert units into the environment
before they can be (fully) compiled, strictly for the purposes
of doing a "limited with" of them.

Why don't you suggest some wording so we actually
have an alternative to consider?

**************************************************************

From: Robert A Duff
Sent: Monday, February  3, 2003  6:46 PM

Tucker wrote:

> I think we need to say *something* about the requirements
> on a unit mentioned in a limited-with clause.  Why not say
> it must exist in the environment?  As you said, that is
> pretty vague.  If there are no requirements on it, then
> the implementation has no place to "hang" whatever requirements
> it really does have.  It can't just invent requirements that
> don't exist.  But it can say, "as far as our implementation
> is concerned, that unit doesn't exist in the environment,
> so we refuse to compile the limited-with for it."

OK, you have convinced me on that point.  But as to 10.1.4(6):

> Again, we need a place for the implementation to "hang"
> its requirements.  We have given them the first handle,
> by allowing them to require existence.  Now we are
> giving them the second handle, allowing them to impose
> certain other requirements on the unit.

But 10.1.4(6) as is already gives sufficient permission.
Your proposed change gives *less* permission.
I claim there is no need to be more restrictive (on implementations),
and that being more restrictive requires more complex RM wording.

10.1.4(6) as is allows compilers to refuse to deal with "limited with X"
on an illegal X.  That is sufficient permission.  If X is in the env,
and is legal, the compiler must deal with "limited with X", because the
only permission to refuse to admit X into the env is if it's illegal.

>  They *clearly*
> can't impose complete legality requirements, if the
> units are inserted one at a time, since there is no
> order of insertion that would allow that.

There is no requirement that units are inserted one at a time (and in
fact they are not, in AdaMagic).  Hence, it is not "*clearly*" true that
they can't impose legality requirements.

Your wording for 10.1.4(6) does not allow a compiler to impose such
legality requirements.  I think that's overspecification, and could be
an implementation burden.  For example, it is reasonable to implement
some legality rules in the parser, and some compilers do that.
One might wish to run the parser on X when seeing "limited with X".
Your wording would disallow such an implementation.

Imagine a front end structured as three phases:

    1. Parse and build syntax tree.
    2. Walk tree, build symbol table nodes for explicit declarations,
       collect a list of all type decls, and check *some* legality
       rules.
    3. Overload resolution, and check the rest of the legality rules.

The "limited with" wants that list of type decls from phase 2.
But your wording seems to forbid phase 2 from printing any error
messages, because they're not syntax errors.

> I'm not sure what we are discussing any more, since you
> don't seem to think that the unit even need to exist
> in the environment to be mentioned in the limited-with
> clause.

I backed off on that part.

>...  I think we need to *allow* implementations
> to impose some requirements, and we might as well tie
> it to the notion of "exist" in the environment.
> *If* we tie it to that notion, we want to be sure that
> limited-with can in fact be used.  That seems to mean
> that you must be able to insert units into the environment
> before they can be (fully) compiled, strictly for the purposes
> of doing a "limited with" of them.
>
> Why don't you suggest some wording so we actually
> have an alternative to consider?

OK, I propose changing 10.1.4(5) in the way you suggested, and leaving
10.1.4(6) as is.

**************************************************************

From: Tucker Taft
Sent: Monday, February  3, 2003  7:02 PM

Robert A Duff wrote:
 > ...
> OK, I propose changing 10.1.4(5) in the way you suggested, and leaving
> 10.1.4(6) as is.

I suppose there is an unwritten rule that there
must be some way for two packages with cylic
dependence to be inserted into the environment.
Clearly if the "insertion" process requires legality,
and you can only insert one unit at a time,
there is a problem.  Either the compiler
must allow simultaneous multiple-file insertion
(that sounds like it might be illegal in Nebraska ;-),
or it must allow units that are not yet demonstrably
legal to be inserted.

**************************************************************

From: Robert A Duff
Sent: Monday, February  3, 2003  7:14 PM

Don't you think it's OK to leave that rule unwritten?
I mean if that's a hole, it's already a hole.

If I have some legal code, I must be able to insert it into the env
*somehow*, since there's no permission for the implementation to refuse
legal code.  The fact that the implementation is incapable of
determining whether it's legal seems irrelevant.  *I* know it's legal.

I think you're worrying overmuch about implementation issues (in
considering RM wording).  Legal code must be accepted.  Therefore, code
must be accepted if the compiler does not (yet) know whether it's legal.
The compiler can only refuse if it can *prove* it's illegal.  Make
sense?

**************************************************************

From: Tucker Taft
Sent: Monday, February  3, 2003  8:33 PM

After thinking more about this, I think we need
to make some change to the wording of 10.1.4(6).  It really
doesn't make any sense if we change 10.1.4(5) as proposed.
We seem to agree that this sentence is primarily for
non-source-based compilers, where the only way to insert a
unit into the environment is by processing it in some way.
But it is exactly those compilers that will have to change to
support limited-with.  They *cannot* require legality of all
units inserted into the environment, if we have just
changed 10.1.4(5) to require that the unit mentioned in
a limited-with must already *be* in the environment.

I think we should say something closer to this:

   Before inserting a compilation unit into the environment,
   the implementation may require that it obey all the Syntax
   Rules, and any other rules that it can check given
   what other units already exist in the environment.

> ...
> I think you're worrying overmuch about implementation issues (in
> considering RM wording).  Legal code must be accepted.  Therefore, code
> must be accepted if the compiler does not (yet) know whether it's legal.
> The compiler can only refuse if it can *prove* it's illegal.  Make
> sense?

No. ;-).

I think either we should make 10.1.4(6) completely
implementation-defined, or replace it with something
that reflects the problem that non-source-based
compilers will have with limited withs, since we
know that 10.1.4(6) was crafted specifically for
their needs.

Remember, this is in the section called "implementation
permissions" so it is specifically talking to implementors,
not users, and it is specifically talking about implementation
issues, not abstract semantic issues.

**************************************************************

From: Robert A Duff
Sent: Monday, February  3, 2003  7:06 PM

Pascal,

This "weak with" ("limited with") thing is your proposal, but if you
like, I might be willing to work out the RM wording tomorrow (Tuesday).
This idea seems to be gaining favor, and if it's to be chosen, I don't
really want that to take 17 more ARG meetings.  Producing wording now
might speed things up.

On the other hand, if folks think it's too hard to implement, please say
so before I waste my time.

Should I do it?

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:21 AM

It would be very useful if you did that.  You are much more competent than me at
crafting RM wording, and furthermore I don't think I would have time to do that
between now and the meeting.

Since there seems to be some interest for the idea, it would be good to have a
precise proposal on the table to discuss it.  Experience has proven that
discussing an AI without an initial write-up is sterile.

**************************************************************

From: Randy Brukardt
Sent: Monday, February  3, 2003  8:35 PM

I'm not Pascal, but I'd suggest going ahead and writing up the wording. I'm
having a hard time getting a handle on this proposal, and I'm sure that
would help. OTOH, since it is a complete start from scratch, it's going to
take 17 meetings to get it right in any case. At most, you can reduce that
number by one. :-)

> On the other hand, if folks think it's too hard to implement, please say
> so before I waste my time.

At the moment, I think it's too hard to implement (given that type stubs are
expected to take 4 hours of work; this would be more like 4 weeks). But that
may be irrelevant. More important is that I don't see any way, within the
current design of Janus/Ada, to implement something like this.

The problem is with the obsoleteness check. Janus/Ada uses a serial number
created from the time stamp of the symbol table file created for a library
unit for checking. Also, the Janus/Ada compiler knows nothing about source
files other than the one it is compiling at the moment. (Other tools are a
bit smarter, but the compiler does not use that information itself.) It only
knows about symbol table files.

One presumes that you'd have a new kind of symbol table file that held a
'lightly' compiled package spec for limited withs. And you'd use the time
stamp of that file to create the serial number for obsoleteness checking.
Clearly, when you 'really' compiled the spec, the new kind of package spec
would have to be updated (otherwise, it wouldn't necessarily match the
source file, because the source file could have been edited). But that would
necessarily change the serial number, and then the links would fail. And
there would be no way to avoid the problem.

Given that there is no reliable way to determine if a file has been changed
(file time stamps are too crude, often changing when nothing actually is
changed, and aren't kept to enough accuracy on Windows), I don't see any way
to implement this. The compiled=changed model works fine for Ada 95, but is
a complete failure for limited with. Essentially, you have to adopt a
source-based model, and that is a major change.

This also has a giantic impact on build tools. They'd have to be able to
process this new clause and come up with an appropriate order in the face of
cycles. Of course, tools just punt if that happens now.

It may be I just haven't thought about it enough. But I have a lot of things
that I need to do before I leave for the meeting, and thinking about this
isn't high on the list.

**************************************************************

From: Tucker Taft
Sent: Monday, February  3, 2003  8:53 PM

Randy's comments make me notice something
a bit unfortunate about the limited-with
proposal.  Although we don't need a new
kind of compilation unit, we do need a new
way of compiling a unit in the non-source-based
environment.  (I'll call this new way of compiling
"preregistering" the unit.)  Supporting limited-with
means that either the programmer preregisters
all compilation units on the off chance they might
be mentioned in a limited-with, or the compilation-order-
determination tool gets smarter and figures out which
units need to be "preregistered" and which don't.

Having this additional step could have significant
ripple effects into "make" scripts, regression
testing scripts, ACATS testing scripts, compilation-order-
determination tools, etc.  It also means a new
step to explain to programmers.

None of the above is particularly rocket science, but
it does add to the implementation and "deployment" cost
of this proposal.

In general, I think this might emerge as a very
elegant solution, possibly as elegant as the package
prefix/abstract approach, but significantly more
implementation effort than either the "type C.T;"
or the possibly modified/restricted type stub proposal.

By the way, although it sounds like implementing
"type C.T;" in Randy's compiler might be a bit
more work than a restricted type stub, it seems like that is
small potatoes compared to the heavy lifting involved
in the limited-with proposal.

**************************************************************

From: Robert Dewar
Sent: Monday, February  3, 2003  8:53 PM

BY the way, this discussion of what is and is not required of compilers
reminds me that I think it would be good to have an annex for the standard
that defined a standard interchange format, right down to the bit-level
coding and file format. That would for example, partly satisy Dan's
concern about incomaptible source representation issues.

**************************************************************

From: Gary Dismukes
Sent: Tuesday, February  4, 2003  1:28 AM

> But it seems like whatever solution to the mutual recursion problem we
> choose, we really need to solve the problem of proliferation of silly
> type conversions.  Otherwise, these proposals will be nearly unusable.

I tend to agree.  Certainly with the latest "limited with" proposal there
are generally going to be multiple access types for the same designated type
and that's true for most of the other proposals as well.  It seems though
that the models using child packages may be less prone to that since
the child package could use an access type declared in the parent
and reexport using a subtype if needed.

> That is, there ought to be an *implicit* conversion between access types
> in the "safe" cases.  I know we discussed that before.  Is it part of
> any of these proposals, or is it a separate AI?  I seem to recall that
> it might be wrapped up with other access-type issues, like allowing "not
> null" constraints, and access-to-constant parameters.

Perhaps you're thinking of AIs 230 (generalized access types) and 231
(access-to-constant parameters and null-excluding subtypes).  AI-230
would allow more things to be declared using anonymous general access
types, which would increase the number of contexts where implicit
conversions can occur, though I'm not sure how much help that will be
for the mutual-dependence cases where there will be multiple distinct
access types and pointers of those distinct types floating around.
I suppose it will help to some degree.  In any case, I think it's
probably best not to wrap a requirement for easier conversions in with
the AI-217 proposals, as I think that may tend to muddy the discussion
(even more more than it already is;).

> > And I do still like that "type C.T;" ;-).
> > (what a pain I can be...)
>
> That's a fine solution, IMHO.  Your earlier e-mail about
> mutually-recursive types being part of the same abstraction
> makes me comfortable with the idea of tying the solution to
> child packages.  (See, it *is* possible to convey sensible information
> about purely aesthetic concerns.  ;-))
>
> But I would choose the "limited with" idea, unless it is considered too
> hard to implement.

My feeling as well.  Tucker's modified child package proposal definitely
seems the simplest both in description and implementation effort
(well, I guess Randy doesn't agree with that...), but like others
I'm somewhat dissatisfied with requiring a child-based model to
get mutual recursion.  I was finding my mind being drawn back to
wishing we could just do things as simply as other languages are
able to, but had decided that such an approach would be tilted too
much in favor of source-based compilers.  But now that Pascal's
bravely put forward that very suggestion, I'm happy to line up behind
it if it stands up to scrutiny.  I think the critical question is how
much work that approach will be for non-source-based implementations.

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  4:04 AM

> Perhaps you're thinking of AIs 230 (generalized access types) and 231
> (access-to-constant parameters and null-excluding subtypes).  AI-230
> would allow more things to be declared using anonymous general access
> types, which would increase the number of contexts where implicit
> conversions can occur, though I'm not sure how much help that will be
> for the mutual-dependence cases where there will be multiple distinct
> access types and pointers of those distinct types floating around.
> I suppose it will help to some degree.

AI 230 looked quite promising, but it has been on Tuck's action item list for
about two years, and he didn't do his homework (but then he'll have some time on
the plane, so you never know...)

> In any case, I think it's
> probably best not to wrap a requirement for easier conversions in with
> the AI-217 proposals, as I think that may tend to muddy the discussion
> (even more more than it already is;).

Agreed.

**************************************************************

From: Robert A Duff
Sent: Tuesday, February  4, 2003  4:11 PM

I don't want to muddy the discussion *too* much.  But many (if not all)
of the cyclic-dependency solutions are going to introduce extra access
type decls.  I don't want folks to reject proposals for that reason.
Therefore, we need to at least keep an open mind -- assume that the
access-type-conversion annoyance can be solved.

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:02 AM

> In Pascal's proposal, a parent package P can weakly 'with' its children:
> "weak with P.C;", if you want a mutually dependent pair of types,
> one in P and one in P.C.  The spec of P will be elaborated before the
> spec of P.C, as usual.
>
> However, it is pointless for P.C to say "weak with P;", because the
> child-to-parent dependency is necessarily strong (given that there's no
> proposal to add syntax saying otherwise -- "weak children?").
>
> No problems here.  I'm just making some observations.

In my proposal it would also be possible for unit P to say "limited with P", but
I think we would want to forbid that, as it seems methodologically dubious.  (If
you follow the model to its logical conclusions, it would for instance allow
forward references to packages and types within the spec of P.)

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:21 AM

>  The first is where you have a type declaration that depends on another
> type:
>
> type Color is (Red, Green, Blue);
> type Properties(C: Color) is...;
>
> Does the incomplete type for Properties include the discriminant?

I have no doubt that the incomplete type for Properties doesn't include the
discriminant.  We discussed that issue when we did "with type" and we concluded
that any other choice was causing endless trouble.

Discriminants of incomplete types are not too useful anyway, they can only be
used in building contrained access-to-object types.  And the added complexity
is not worth it.

> I prefer the choice where only private, record and tagged
> types are incomplete in the limited view.

But then you would need to do a lot of semantic analysis, and you would have to
define very precisely what happens during a "superficial" compilation.
Consider for instance:

    type R is record ... end record;
    type T is new Integer range R'Alignment .. R'Size;

How can T be properly defined if R is incomplete?  And don't tell me that T
would be some new kind of "incomplete integer type", because that would add so
much complexity to the language that I'd rather forget about the entire
proposal.

As Tuck pointed out in his "willies" message, the only way that this model can
be made to work is if the first phase can be performed by a brain-dead parser.

**************************************************************

From: Robert I. Eachus
Sent: Tuesday, February  4, 2003  11:38 AM

Pascal Leroy wrote:

>I have no doubt that the incomplete type for Properties doesn't include the
>discriminant.  We discussed that issue when we did "with type" and we concluded
>that any other choice was causing endless trouble.

About 4 AM, I realized that what is needed is the fact that Properties
is a type with discriminants, notthing more.  But we already have that
concept well defined, and it covers incomplete tagged types. Then I went
back to sleep.  So I propose that (incomplete) tagged types and types
like Properties have unknown discriminants.  This allows incomplete
discrete types to be treated like other incomplete types, and incomplete
array types to also be treated as incomplete with unknown discriminants.

>But then you would need to do a lot of semantic analysis, and you would have to
>define very precisely what happens during a "superficial" compilation.  Consider
>for instance:
>
>    type R is record ... end record;
>    type T is new Integer range R'Alignment .. R'Size;
>
>How can T be properly defined if R is incomplete?  And don't tell me that T
>would be some new kind of "incomplete integer type", because that would add so
>much complexity to the language that I'd rather forget about the entire
>proposal.

The nightmare that woke me up was not quite this case, but related.
 (Using T'First, etc., of another discrete type.Hiding the literals is
not enough.)

As for access types, I still think we need to special case those.  At
first I was only worried about the proliferation of access types with
the same target.  But then I realized that the type matching nightmare
gets worse in cases where some of the potential access types are hidden
by incompleteness.  The messy case is X: T := F.all.  F can be an
overloaded function, and I would hate to have the legality checked
twice, especially if one of the checks depended on elaboration order.

There is another painful case though, and I am not sure what to do about
it...

limited with B;
package A is
   type Foo is record...end record;
   ...
end A;

with A;
package B is
    type TA is access A.Foo;
    type Bar is record
       TAC: TA;
    end record;
    ...
end B;

(You can produce variations on this theme by creating a third package in
the cycle.) We are not talking pathological here, this is part of the
core functionality.  Now, what is the status of B.TA inside the spec of
A?  If it is incomplete we get access type proliferation.  If it is an
access type, is it an incomplete access type? Inquiring minds want to know.

If you prefer, write the example as:

limited with A.B;
package A is
    type TA is access B.T;
    ...
end A;

package A.B is
   type T is tagged record...end record;
   ...
end A.B;

I can live with this version working where the first one breaks, but now
we get back to the issue of representation of the access type.  Whether
it has to be a fat pointer or not will depend on features of T.  Again,
I can live with that, we just have to determine which features are
visible.  (Unknown discriminants for T in this case seems minimal.)

**************************************************************

From: Tucker Taft
Sent: Tuesday, February  4, 2003  2:20 PM

"Robert I. Eachus" wrote:

> About 4 AM, I realized that what is needed is the fact that Properties
> is a type with discriminants, notthing more.

Why?

>... But we already have that
>concept well defined, and it covers incomplete tagged types. Then I went
>back to sleep.  So I propose that (incomplete) tagged types and types
>like Properties have unknown discriminants.  This allows incomplete
>discrete types to be treated like other incomplete types, and incomplete
>array types to also be treated as incomplete with unknown discriminants.

Why can't we treat these all as plain old incomplete types?
You haven't explained that.


>>But then you would need to do a lot of semantic analysis, and you would have to
>>define very precisely what happens during a "superficial" compilation.  Consider
>>for instance:
>>
>>    type R is record ... end record;
>>    type T is new Integer range R'Alignment .. R'Size;
>>
>>How can T be properly defined if R is incomplete?  And don't tell me that T
>>would be some new kind of "incomplete integer type", because that would add so
>>much complexity to the language that I'd rather forget about the entire
>>proposal.
>>
>The nightmare that woke me up was not quite this case, but related.
> (Using T'First, etc., of another discrete type.Hiding the literals is
>not enough.)

I am totally lost.  These types are incomplete.  You can't do anything
with them.

> As for access types, I still think we need to special case those.  At
> first I was only worried about the proliferation of access types with
> the same target.  But then I realized that the type matching nightmare
> gets worse in cases where some of the potential access types are hidden
> by incompleteness.  The messy case is X: T := F.all.  F can be an
> overloaded function, and I would hate to have the legality checked
> twice, especially if one of the checks depended on elaboration order.

I must be missing something fundamental here.  All of these types are
just plain old incomplete types.  Yes, I agree we ought to special
case tagged and allow a bit more, but that seems irrelevant to
what you are talking about.  What is messy about the messy case?

>...
>(You can produce variations on this theme by creating a third package in
>the cycle.) We are not talking pathological here, this is part of the
>core functionality.  Now, what is the status of B.TA inside the spec of
>A?  If it is incomplete we get access type proliferation.  If it is an
>access type, is it an incomplete access type? Inquiring minds want to know.

It is incomplete and we get access type proliferation.

I don't see that "limited with" can solve the access type proliferation
problem, and trying to make it do so is doomed, in my mind.

...
>I can live with this version working where the first one breaks, but now
>we get back to the issue of representation of the access type.  Whether
>it has to be a fat pointer or not will depend on features of T.  Again,
>I can live with that, we just have to determine which features are
>visible.  (Unknown discriminants for T in this case seems minimal.)

I am presuming that all non-tagged types are incomplete, period.  No new
notion of "incomplete access types" is being added.

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:00 PM

> > So I propose that (incomplete) tagged types and types
> > like Properties have unknown discriminants.  This allows incomplete
> > discrete types to be treated like other incomplete types, and incomplete
> > array types to also be treated as incomplete with unknown discriminants.
>
> Why can't we treat these all as plain old incomplete types?
> You haven't explained that.

I am as puzzled as Tuck.  It is my understanding that unknown discriminants
for incomplete types are pretty much equivalent to no discriminants.  So
it's fine with me to insist that these types have unknown discriminants, but
it doesn't make the slightest difference.  (Remember, we are talking
_incomplete_ types here, so you cannot do much with them anyway.)

> I am presuming that all non-tagged types are incomplete, period.  No new
> notion of "incomplete access types" is being added.

Agreed.  This seems like the only way to avoid madness.  Granted, this
proposal doesn't solve the problem of proliferation of access types, but
that problem shows up in contexts that are unrelated to circular type
dependencies, so I think that it should be addressed by a different
mechanism.

**************************************************************

From: Randy Brukardt
Sent: Tuesday, February  4, 2003  2:53 PM

Tucker said, replying to Robert Eachus:
> > If it is incomplete we get access type proliferation.  If it is an
> > access type, is it an incomplete access type? Inquiring
> > minds want to know.
>
> It is incomplete and we get access type proliferation.
>
> I don't see that "limited with" can solve the access type proliferation
> problem, and trying to make it do so is doomed, in my mind.

At which point, I lose interest in the proposal.

All of type stubs, restricted type stubs, and Tuck's type C.T idea do solve
the access type proliferation problem. Admittedly the structure of the
declarations is unnatural. But the only real alternative is to adopt
something like AI-230. That bulks up the proposal a ton.

So, limited with:
   -- Is much harder to implement in library based compilers. It's probably
harder to implement than type stubs/type C.T in source-based compilers as
well (certainly no easier). Affects compilation tools in all environments;
   -- Doesn't solve the access type proliferation problem.

The only advantage seems to be:
   -- A more natural expression of package structure.

But that's dubious, since you need to resort to unnatural package structures
in order to share access types. So you're pretty much left with "much harder
to implement".

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:09 PM

> > I don't see that "limited with" can solve the access type proliferation
> > problem, and trying to make it do so is doomed, in my mind.
>
> At which point, I lose interest in the proposal.

Access type proliferation in and of itself is not a problem.  What is a
problem is access type conversions.  That problem shows up in all sorts of
contexts, notably in OOP where you have access types (class-wide or
specific) designating some type in a hierarchy, and all conversions must be
explicit, even those that cannot possibly fail.  That is a big pain.

This has nothing to do with circular type dependences.  Even if we select a
solution to the problem of circular type dependences that doesn't cause
proliferation of access types, the problem of access type conversions will
remain.

In C, if you have a type t you automatically get a pointer type t*.  In Ada,
I often wish we had a similar capability, where I wouldn't have to declare
all these silly access types and convert between them explicitly.

**************************************************************

From: Robert I. Eachus
Sent: Tuesday, February  4, 2003  5:20 PM

Tucker Taft wrote:

>It is incomplete and we get access type proliferation.
>
>I don't see that "limited with" can solve the access type proliferation
>problem, and trying to make it do so is doomed, in my mind.

And in my mind the proposal is doomed (or not the best one on the table)
if it doesn't address the access type proliferation problem.

>I am presuming that all non-tagged types are incomplete, period.  No new
>notion of "incomplete access types" is being added.

Hmm.  I must not be making myself clear.  My idea is not to add
"incomplete access types," but to try to make the limited with idea work
with access types to these new special incomplete types.  The refractory
issue is when you have access types that carry descriptor information
and when you don't.  Where that data is stored is irrelevant.  Even if
you store the descriptor for an unconstrained array type with the data
and pointers to both unconstrained and constrained arrays are the same
size, some conversions and assignments are impossible or require
recopying the data:

package P is
     type Str is new String;
     subtype Str4 is Str(1..4);
     type PT1 is access Str;
     type PT2 is access Str4;
end P;

or if you prefer:

limited with P;
package Q is
  type PT3 is access P.Str;
  type PT4 is access P.Str4;
end Q;

Now I need to compile some code--perhaps the body of Q--that needs to do
those damned explicit type conversions, or even just some simple
assignments.  As a programmer I have a problem, but without more
information about the types in P than Tuck seems willing to allow, the
compiler is SOL.  It doesn't know how to create objects of type PT3
and/or PT4, and even has trouble with legality issues:
...
PO3: PT3;
PO4: PT4;
...
PO3.all := PO4.all; -- legal?

Right now the voting seems to be that either the limited with idea is
doomed because it is too much work, or it is doomed because it can't
deal with the issue of access type proliferation.  I'll let sleeping
dogs lie.

**************************************************************

From: Tucker Taft
Sent: Tuesday, February  4, 2003  5:58 PM

"Robert I. Eachus" wrote:
>
> Tucker Taft wrote:
>
> >It is incomplete and we get access type proliferation.
> >
> >I don't see that "limited with" can solve the access type proliferation
> >problem, and trying to make it do so is doomed, in my mind.
> >
> And in my mind the proposal is doomed (or not the best one on the table)
> if it doesn't address the access type proliferation problem.

Well, I think trying to make it address this problem very tricky,
since it means looking at the designated subtype indication,
and that will get you back to worrying about use visibility, etc.

> >I am presuming that all non-tagged types are incomplete, period.  No new
> >notion of "incomplete access types" is being added.
> >
> Hmm.  I must not be making myself clear.  My idea is not to add
> "incomplete access types," but to try to make the limited with idea work
> with access types to these new special incomplete types.  The refractory
> issue is when you have access types that carry descriptor information
> and when you don't.  Where that data is stored is irrelevant.  Even if
> you store the descriptor for an unconstrained array type with the data
> and pointers to both unconstrained and constrained arrays are the same
> size, some conversions and assignments are impossible or require
> recopying the data:

I am getting an inkling of your concern.  At least one of the
problems you seem to be worrying about is an implementation
issue (as opposed to a semantics issue), and is one that bedevils
the "unrestricted" type stub proposal, and the "with type"
proposal.  Namely that when choosing a representation for access types
to the full type, the compiler may not be aware that there are
access types to the incomplete type.  I don't think this is quite
as serious as the problem with the "with type" proposal, since
the compiler is doing some serious peeking at the full type
when it creates the incomplete type, so *if* it has multiple
access type representations, it can mark the incomplete type
with enough extra information so that it will choose an appropriate
representation for any access type that is declared later which
"sees" only the incomplete type.

Unfortunately, derived types create a bit of a problem, since they
don't reveal syntactically what sort of type they are (e.g.,
whether their first subtype is an unconstrained array subtype and
hence pointers to it should be default be "fat" pointers).
This would mean that the "peeker" would have to make a "guess"
about whether the incomplete type is an unconstrained array.
Probably it would assume all derived types are not
unconstrained arrays.  In any case, whatever guess it makes,
it would want to record that in the limited view of the package.
When the full view is compiled, as part of the "compatibility"
check it would also have to check that the "guess" was correct.
If the guess was wrong, and the type is in fact an unconstrained array,
say, then it would have to record that fact on the incomplete
type, and bump the timestamp (or equivalent) of the limited view
of the package.  Ultimately the units that depend on this limited
view would get recompiled.

For a source-based compiler, there isn't really any place to
permanently record the "correct" answer, so it would probably
end up being a link-time check, and if there were an incompatibility,
the offending units would be recompiled with a switch directing
them to treat the problematic incomplete type as an unconstrained array,
even though it wasn't obvious.


>
> package P is
>      type Str is new String;
>      subtype Str4 is Str(1..4);
>      type PT1 is access Str;
>      type PT2 is access Str4;
> end P;
>
> or if you prefer:
>
> limited with P;
> package Q is
>   type PT3 is access P.Str;
>   type PT4 is access P.Str4;

This wouldn't be legal, since Str4 is defined by a subtype, not
a type declaration.

> end Q;
>
> Now I need to compile some code--perhaps the body of Q--that needs to do
> those damned explicit type conversions, or even just some simple
> assignments.  As a programmer I have a problem, but without more
> information about the types in P than Tuck seems willing to allow, the
> compiler is SOL.  It doesn't know how to create objects of type PT3
> and/or PT4, and even has trouble with legality issues:

I don't see how legality issues enter into the problem of access
type representation.  You can't do much of anything with incomplete types.
And you can't dereference access-to-incomplete, unless perhaps
there is a complete type "nearby".

> ...
> PO3: PT3;
> PO4: PT4;
> ...
> PO3.all := PO4.all; -- legal?
>
> Right now the voting seems to be that either the limited with idea is
> doomed because it is too much work, or it is doomed because it can't
> deal with the issue of access type proliferation.  I'll let sleeping
> dogs lie.

I don't think it has been abandoned, so it is still important
to try to identify (real ;-) problems associated with the
proposal.

**************************************************************

From: Robert I. Eachus
Sent: Tuesday, February  4, 2003  6:34 PM

Tucker Taft wrote:

>Well, I think trying to make it address this problem very tricky,
>since it means looking at the designated subtype indication,
>and that will get you back to worrying about use visibility, etc.

Yep.

...
>For a source-based compiler, there isn't really any place to
>permanently record the "correct" answer, so it would probably
>end up being a link-time check, and if there were an incompatibility,
>the offending units would be recompiled with a switch directing
>them to treat the problematic incomplete type as an unconstrained array,
>even though it wasn't obvious.

Yep, it is a problem in any solution to the mutual dependence problem.
 The issue here is deciding what aggrevations to add--to the user, the
implementor, or both.  We could make fairly draconian rules that stated
that any unit that actually uses a subtype, access type to, or type
derived from one of these magic incomplete types  must depend on the
limited withed unit, and thus see the full type.  I think so far this
has been an implicit assumption, but the devil is in the details.

To take your discussion above a bit further, what happens when you
modify the limited withed unit in a way that changes the necessary
representation of an access type declared elsewhere?  It begins to look
like a very real dependence somehow magically wished away.  You end up
having to recompile some units due to changes in units they don't depend
on....

>I don't think it has been abandoned, so it is still important
>to try to identify (real ;-) problems associated with the
>proposal.
>
Consider the (dead) horse to have been additionally flogged.

**************************************************************

From: Robert A Duff
Sent: Tuesday, February  4, 2003  7:06 PM

Note that we already have a case where you can create
access-to-incomplete without knowing what the complete type
looks like.  Namely, an incomplete type declared in a package
spec, and completed in the body.

**************************************************************

From: Robert I. Eachus
Sent: Tuesday, February  4, 2003  10:34 PM

What did we used to call that?  Ah, yes, I remember the Tucker Taft
amendment. ;-)  The only good thing about it was that the incomplete
type had to be private, so it couldn't be seen outside the package.

If you weren't following LMC/ARG actions then look up (Ada-83) AI-7.
 For one meeting, John Goodenough put the AI's for consideration in two
books--AI-2 and AI-7, plus related AI's, and everything else.  (The
final version of AI-00007/19-BI-WJ--yes that is version 19--is actually
a consolidation of about a dozen separate AI's)

As I recall, AI-7 underwent combinatorial explosion, we discovered lots
of nits and crannies, and finally came up with a smoking gun case that
did not require the TTA to fire off.  But if you think my concern about
access to incomplete types with discriminants  is overdone, you really
should go back and look at all those AI's.  As I recall, the final
version of AI-7 allowed the legality check on discriminant constraints
of access to incomplete objects to be done in one of three
places--because there were cases when each of the three wouldn't work,
and in some cases more than one.

Any "solution" to the interlocking types problem that re-opens that can
of worms should be shot then burned and the ashes stirred and buried
under a crossroads at midnight.  I think net that AI-7 and its relatives
took the equivalent of several three-day meetings to resolve. (This was
"fixed" in Ada 95 by deferring all compatibility checks until object
creation.

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:55 AM

> At the moment, I think it's too hard to implement (given that type stubs are
> expected to take 4 hours of work; this would be more like 4 weeks). But that
> may be irrelevant. More important is that I don't see any way, within the
> current design of Janus/Ada, to implement something like this.

Not sure how you came up with the 4 hours estimate, but I'm sure it would take
me more than that to do a detailed design.  At this point I'd say that the
implementation effort for any of these proposals would be in the range 4 weeks
.. 4 months, and I can't be more specific without doing a detailed analysis.
But anyway, I don't think that numbers are very interesting at this point.  It's
really the essence of the implementation difficulties that need to be looked at.

As far as I can tell, you and I have the same set of problems, because we are
both library-based.  I realize that the devil is in the details, and that the
magnitude of changes can vary substantially from one implementation to another,
but I don't buy that the "limited with" proposal would force you to go to a
source-based model.  (For Rational, going to a source-based model is a no-no;
we'd rather get out of the Ada business; so hopefully there has to be an
implementation technique where we stick to our current library model.)

> The problem is with the obsoleteness check. Janus/Ada uses a serial number
> created from the time stamp of the symbol table file created for a library
> unit for checking. Also, the Janus/Ada compiler knows nothing about source
> files other than the one it is compiling at the moment. (Other tools are a
> bit smarter, but the compiler does not use that information itself.) It only
> knows about symbol table files.

Fine.  If I substitute "Diana tree" for "symbol table", that's essentially true
of Apex too.  (Ignoring all the incremental compilation crap.)

> One presumes that you'd have a new kind of symbol table file that held a
> 'lightly' compiled package spec for limited withs. And you'd use the time
> stamp of that file to create the serial number for obsoleteness checking.
> Clearly, when you 'really' compiled the spec, the new kind of package spec
> would have to be updated (otherwise, it wouldn't necessarily match the
> source file, because the source file could have been edited).

And we would indeed have the same problem.  We would first created a "parsed"
Diana tree (during "superficial" compilation) and then we would rewrite it to
make it "analyzed" (during "full" compilation).  Surely that would change the
timestamp.

> But that would
> necessarily change the serial number, and then the links would fail. And
> there would be no way to avoid the problem.
>
> Given that there is no reliable way to determine if a file has been changed

The way that I plan to do this is to store in the "parsed" Diana tree (what you
call the "light symbol table") a hash code.  This hash code could be based on
the sequence of tokens in the source, or it could be more clever and be based
only on the names of types and packages (the information that is available after
"superficial" compilation).  Whenever clients currently use a timestamp to check
for obsolescence, they would use the combination <timestamp, hash>.  If the
timestamps match, everything is fine.  Otherwise, the Diana trees need to be
opened, and the hashes need to be compared.  I realize that this will result in
some extra open system calls, but Ada's separate compilation already requires a
zillion system calls, so I won't lose any sleep on that.

Of course, a hash code is not exactly 100% safe, as there could be collisions.
But if you design it well enough, you won't see a collision in your lifetime.
(We use 96-bit hash codes all over the place for incremental compilation, and we
have never run into a collision in all these years.)

> This also has a giantic impact on build tools. They'd have to be able to
> process this new clause and come up with an appropriate order in the face of
> cycles. Of course, tools just punt if that happens now.

But the trick is that there are no prerequisites for "superficial" compilation.
So one approach is to blindly run "superficial" compilation over all the units,
in any order you like (e.g. the order of i-nodes in the filesystem if you like).
And then run "full" compilation in an order compatible with normal with clauses
and other semantic dependencies, just like you do now (and ignoring "limited
with" clauses).  No need to process the new clause, no need to change the
algorithm that determines the compilation order.

Of course you might want to be more clever and avoid the "superficial" phase if
it's not needed, but that's an optimization, not something you would have to do
for correctness.

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  4:00 AM

> Supporting limited-with
> means that either the programmer preregisters
> all compilation units on the off chance they might
> be mentioned in a limited-with, or the compilation-order-
> determination tool gets smarter and figures out which
> units need to be "preregistered" and which don't.

As I mentioned in my previous response to Randy, it would be just fine if the
compilation-order-determination tool were to systematically preregister each and
every unit, and then run the normal ordering algorithm, ignoring "limited with"
clauses.  I would think that compiler writers would prefer to go that way,
rather than breaking each and every compilation script in the universe.  So I am
arguing that the new step should be essentially invisible to programmers.

In an implementation where the preregistration step would be costly, then yes,
the ordering algorithm might have to be modified to avoid preregistration when
possible.

**************************************************************

From: Erhard Ploedereder
Sent: Tuesday, February  4, 2003  7:38 AM

I like "limited with". It is the closest we got to what the user expects.
I would like to see a writeup.

(I have had bellyaches with Tuck's proposal of tying the cyclic dependency
 to child packages. I waited for the aches to go away. They did not.)

**************************************************************

From: Randy Brukardt
Sent: Tuesday, February  4, 2003  3:00 PM

> > At the moment, I think it's too hard to implement (given that type stubs are
> > expected to take 4 hours of work; this would be more like 4 weeks). But that
> > may be irrelevant. More important is that I don't see any way, within the
> > current design of Janus/Ada, to implement something like this.
>
> Not sure how you came up with the 4 hours estimate, but I'm sure it would take
> me more than that to do a detailed design.

I did a detailed design (in my head) when I was unable to sleep Saturday
night.

> At this point I'd say that the implementation effort for any of these proposals
> would be in the range 4 weeks .. 4 months, and I can't be more specific without
> doing a detailed analysis. But anyway, I don't think that numbers are very
> interesting at this point.  It's really the essence of the implementation
> difficulties that need to be looked at.

The reason I was thinking about it was that the restricted type stubs model
pretty much matches how Janus/Ada works internally anyway. So the main cost
is "connecting" the stub to the completion. (And many stubs works, because
it's a one-way pointer. But I wouldn't expect that to be true on other
implementations.) Since incomplete and private types use the same code in
most circumstances, the visibility implications of "availability" should be
free. (That is the biggest assumption, of course, because there are no such
rules in Ada 95.) So all of the work is doing the connection, and a brute
force version (walk all of the symbol table looking for stubs, then when a
stub is found, walk the whole symbol table looking for a connection) would
only need to be called in one place. Most of the work is figuring out where
to call that routine. A better version would save the stubs in a list as
they are loaded, so we wouldn't have to search for them. That version would
have no distributed cost and not a lot of cost even when stubs are used, so
I doubt its possible to do better.

Tucker's C.T adds a lot of messing around with ghost packages, but otherwise
is identical (we don't have to look quite so far for a completion, but the
process is the same). Which is why I'm certain its more work.

The current stub stubs requires two stubs of the same completion to match,
even when the completion isn't available. That's more work, but I don't
think it's a very significant amount (it's just another weird special case
in the type matcher, just like anonymous access types and T'Class).

In all of these cases, I wouldn't be surprised to find glitches, but that
would require a pretty complete test suite. Building that would take longer
than doing the implementation (that is often true).

Limited with would require a new kind of symbol file, or something (Pascal
suggests a hash). It also would require going in an adding a bunch of stuff
to the make tool. No one understands how that thing really works, so that
alone is a daunting project. And the failure to provide any sort of solution
to the access type problem makes it feel like "with type" all over again
(which isn't surprising, because it *IS* essentially "with type", just as
"type C.T" is just a different syntax for type stubs).

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:17 PM

> Since incomplete and private types use the same code in
> most circumstances, the visibility implications of "availability" should be
> free.

Aren't you concerned about types having three views (incomplete, partial,
full) in the case of an incomplete type completed by a private type?  That
gives me the willies, to borrow Tuck's words, because it's currently very
complicated in our compiler to decide what properties of a type are visible
at a given place (because of the separation between partial and full view,
because of characteristics that become visible "later within the immediate
scope", etc.).  Adding a third view is not going to make this easier.

**************************************************************

From: Randy Brukardt
Sent: Tuesday, February  4, 2003  3:43 PM

No, because we have separate type records for each view, and they have their
own visibility. For type matching purposes, the only question is whether you
can or cannot walk the pointer to the 'next' view, as we always match (and
use for other purposes) the "fullest" view available. That depends on
visibility, etc. (We have a separate set of routines that always goes to the
completion, which the intermediate code generation uses. To heck with
visibility at that point.) Since this stuff is recursive (well, actually it
uses a loop), you'd first try to go from the incomplete block to the private
block (depending on the visibility of the private), and then, once you had
the private, you'd do the checks that we currently have. So I don't see a
problem with three parts or ten parts for that matter.

But I realize that if you somehow managed to implement types with a single
record for both views (we tried that and concluded it was impossible. But
perhaps we weren't clever enough. Or perhaps its impossible because of
shared generics.), it might be much worse.

But I think that any proposal has to allow stubs (or whatever) of private
types. So you're going to have the three view issues, and if that is a
problem, any solution to this problem will be a nightmare to implement. In
which case you ought to stay out of the implementation difficulty debate
altogether. :-) :-)

**************************************************************

From: Gary Dismukes
Sent: Tuesday, February  4, 2003  3:20 PM

Pascal wrote:
> In my proposal it would also be possible for unit P to say "limited with P", but
> I think we would want to forbid that, as it seems methodologically dubious.  (If
> you follow the model to its logical conclusions, it would for instance allow
> forward references to packages and types within the spec of P.)

One question is, what does it mean when you say "limited with P.C.D".
Is it equivalent to having limited with for P, P.C, and P.C.D, in
analogy with normal with clauses?  That was my first thought, and
it seems natural to make it behave the same, but perhaps it's more
reasonable to say it only applies to the final named unit.  That
would avoid the issue of self-circular references when a parent
withs its child and we would also want the rules to disallow
a direct limited with of yourself.

Another small issue is whether use clauses are allowed for packages
named in these with clauses.  I imagine that the implementation is
effectively going to create a package entity containing an incomplete
type representative for each type in the real package's visible part
(along with representatives for any nested packages presumably)
and since this will look essentially the same as other packages
in the symbol table environment I don't see any technical problem
with permitting a use clause for these packages.  Perhaps there are
methodological reasons for not allowing it, though I can't think
of any.  Changing a limited with to a normal with could introduce
new illegalities due to name clashes, but that's no worse than
adding a new with and use clause for some package.

**************************************************************

From: Pascal Leroy
Sent: Tuesday, February  4, 2003  3:30 PM

> One question is, what does it mean when you say "limited with P.C.D".
> Is it equivalent to having limited with for P, P.C, and P.C.D, in
> analogy with normal with clauses?  That was my first thought, and
> it seems natural to make it behave the same, but perhaps it's more
> reasonable to say it only applies to the final named unit.  That
> would avoid the issue of self-circular references when a parent
> withs its child and we would also want the rules to disallow
> a direct limited with of yourself.

From a pedagogical standpoint it would seem simpler to say that "limited
with" works like "with", i.e. that "limited with P.C.D" is equivalent to
"with P, P.C, P.C.D".  Any other option is going to unnecessary confuse
users.

So maybe the right answer is that if a package has a "limited with" of
itself it has no effect (but is not illegal).

> Another small issue is whether use clauses are allowed for packages
> named in these with clauses.  I imagine that the implementation is
> effectively going to create a package entity containing an incomplete
> type representative for each type in the real package's visible part
> (along with representatives for any nested packages presumably)
> and since this will look essentially the same as other packages
> in the symbol table environment I don't see any technical problem
> with permitting a use clause for these packages.

Yes, I was thinking that use clauses and package renamings would be OK in
that context, although if they lead to problems we could disallow them.  But
I can't think of any problem at this point.

**************************************************************

From: Robert Dewar
Sent: Tuesday, February  4, 2003  10:19 PM

> In C, if you have a type t you automatically get a pointer type t*.  In Ada,
> I often wish we had a similar capability, where I wouldn't have to declare
> all these silly access types and convert between them explicitly.

For some reason, I never really understood it, this proposal, which I was strongly
in favor of, never got significant support (I had suggested early on calling it
type'access).

**************************************************************

From: Tucker Taft
Sent: Tuesday, February  4, 2003  3:27 PM

In discussing this further with Bob Duff, and mulling it over
more in my mind, I see the following restrictions would probably
be needed on the "limited with" capability.  I am using the term
"limited view" of a package to be what you get when you mention
a package in a limited-with-clause.  I am presuming that a limited
view of a package contains only nested packages and types, and all
the types are [tagged] incomple.

A "limited with" *cannot* make the following declarations visible:
  1) Package instantiation
       Because the generic being instantiated needs to be visible
       if we are to determine what types it contains, and identifying
       the generic may require resolving a name that is only use-visible,
       or that is from some other compilation unit

  2) Package renaming
       Because the package being renamed needs to be visible
       if we are to determine what types it contains, and identifying
       the renamed package may require resolving a name that is only use-visible,
       or that is from some other compilation unit

A "limited view" of a package *cannot* be mentioned in:
  3) A "use" clause
       Because we can get a funky kind of Beaujolais effect if we have
       a "use" clause for two different packages, and due to
       changing a "normal" with clauses inherited from some ancestor unit
       the meaning of an identifier switches from one thing to another; e.g.:

        package A is
             X : Integer := 7;
        end A;

        package B is
             X : Integer := 203;
        end B;

        with A;  -- change this to "with B;" and see what happens
        package P is end;

        limited with A, B;
        use A;
        use B;  -- Probably shouldn't be allowed if only have limited view
        package P.C is
            Y : Integer := X;  -- which X is this?
        end P.C;

       This is presuming that if we were to allow a "use" clause for a limited
       view of a package, it would make only the types and subpackages directly visible.
       If the "use" clause made everything in the package visible part
       visible, but only the types were actually "usable" that could work,
       but having visible but unusable declarations of *all* kinds,
       rather than just types and packages, could significantly increase the effort
       of building up a limited view.

  4) A package renaming declaration
      This restriction is probably not as critical as the others.
      The problem comes when someone from outside the unit containing
      the limited with references this package renaming.  What does it see?
      If it has a "regular" with for the target of the limited-with, does
      it see the "full" view of the package via the renaming, or only the
      limited view of it.

Possible restriction (4) brings up the issue of the opposite situation, where
a given unit has a limited view on a package, but also has visibility
on a renaming of a full view of the package.  There seem to be various
possibilities:

   a) The renaming also provides only a limited view when referenced from the given unit.
   b) The given unit has a full view of the package, through either the
      renaming or the name introduced by the limited-with clause
   c) The given unit has a limited view via the name introduced by the limited-with
      clause, and a full view via the renaming, and they are essentially
      completely unrelated packages.
   d) (c), except the packages are recognized as different views of the same
      package, and the incomplete types in the limited view are recognized
      as being completed in the renaming, and so are treated as non-limited
      types for all intents and purposes.

(a) is probably the most consistent with the way package renamings work now,
in that what children you see via a renaming of a library unit package is determined
by the "with" clauses in the unit referencing the renaming, rather than by the
with-clauses in the unit containing the renaming.

(b) and (d) are both similar to the choice I suggested for "type C.T;" where
if the completing type declaration is visible (including presumably via
a renaming), then the incomplete type declaration is hidden from
all visibility.  In the "type C.T;" proposal, this presumably implies
that it is as though the "type C.T;" declaration were not there at all,
so if a renaming "package R renames C;" is visible, but "C" itself is
not visible (because C was not directly "withed"), then you *cannot*
refer to the type at all via "C.T."   You can refer to it via "R.T"
and then of course it is a full type.

There are other possible ways of handling this for "type C.T;"
but this approach required the smallest change to 18.3(19) and at
least makes some kind of sense.

For all proposals, the issue of visible renamings of the enclosing package
when the package itself is not visible is thorny.  A related question is when the
completing type is *not* visible (including not via a renaming), but a declaration of a subtype
*is* visible, or an object of the type is visible.  The questions are always:

   i) Is the type named via the "limited view" name complete or incomplete?

   ii) If incomplete, does it nonetheless "match" the complete type in certain
       contexts?

To put it more concretely:

  package P1 is
    type A is access <incomplete view of type P2.T>;
      -- incomplete view is visible via a limited with, type stub, type C.T, or whatever
    Y, Z : A;
  end;

  package P2 is
      type T is <full type definition>;
  end P2;

  with P2;
  package P3 is
    X : <subtype of complete view of P2.T>;
  end P3;

  with P1, P3;  -- No "with" of P2, no visibility on P2.T
  procedure P4 is begin
      ...
    P3.X := P1.Y.all;  -- when is this legal?   (*)
    P1.Y.all := P3.X;  -- same question         (**)
    if P1.Y.all = P1.Z.all then  -- same question (***)
      ...

Hopefully the answers to (*) and (**) are the same, though
the asymmetric Name Resolution wording of 5.2(4) on assignment
statements seems like (*) and (**) might be treated differently
unless we are careful.

Note that last time we worked on the "availability" rules for type stubs,
we required that either we be in the scope of a with clause for
the enclosing package (or a renaming thereof, I presume), or we be in a context
where we have the complete type "nearby" (I forget the wording at this
point).  This would say that having a visible (but not "with"ed)
renaming of the enclosing package, or having a visible subtype,
wouldn't help.  For the above cases, this would mean that probably
(*) and (**) would be legal, but (***) would not be legal.

Have a nice day... ;-)

**************************************************************

From: Tucker Taft
Sent: Tuesday, February  4, 2003  3:48 PM

Pascal Leroy wrote:
> Randy wrote:
> > One presumes that you'd have a new kind of symbol table file that held a
> > 'lightly' compiled package spec for limited withs. And you'd use the time
> > stamp of that file to create the serial number for obsoleteness checking.
> > Clearly, when you 'really' compiled the spec, the new kind of package spec
> > would have to be updated (otherwise, it wouldn't necessarily match the
> > source file, because the source file could have been edited).
>
> And we would indeed have the same problem.  We would first created a "parsed"
> Diana tree (during "superficial" compilation) and then we would rewrite it to
> make it "analyzed" (during "full" compilation).  Surely that would change the
> timestamp...

After talking with Bob Duff about this a bit, I don't think this
really works.  I think you need to keep around indefinitely both a "full view"
of the package and a "limited view" of the package.  Even after you
do a "full" compile, some units can still do "limited with" and should only
see the limited view.  What I would recommend is you treat them as
pretty much distinct units.  When you fully compile, if there is a limited
view already present, you could check that it is compatible with the full view
(i.e. has the same names in the package/type tree, and the same types are
tagged).

I think if they are not compatible, then and only then does it seem worth
obsoleting the limited view and any units compiled against it.  There seems no
point in hashing the incomplete view, since it is easy enough to check
compatibility.  Hashing would only be an optimization if this compatibility
check turned out to be really expensive.

If a unit requests a limited view, and the only non-obsolete thing you have around
is the full view, I would at that point create the limited view from the full view.
I would bump the time stamp on the limited view only when it is created or updated,
and units compiled against the limited view only worry about that time stamp.
The time stamp of the full view is irrelevant to them.

**************************************************************

From: Randy Brukardt
Sent: Tuesday, February  4, 2003  4:13 PM

"Easy enough to check compatibility"? You've got to be kidding. The only
thing that we know how to do with a symbol file is load them into a symbol
table. But you can't do that to "check compatibility", because you've got to
load the full view into the same place (can't have two packages with the
same library-level name).

Pascal's hash idea (instead of a time stamp) makes much more sense. You'd
create the limited symbol file for every compilation (full and limited) that
you did, but the hash would only change if it actually is different.

Keep in mind that no one actually "obsoletes" anything. How we do
obsoleteness checking is simply to compare the time stamp serial numbers of
every unit withed transitively for every unit withed. They better all match.
Any mismatch is reported as an error, and the compilation aborted. And we of
course repeat the check when linking. We use the order only to provide
better error messages, and units are NEVER removed from the program library
(unless the programmer does so manually).

**************************************************************

From: Tucker Taft
Sent: Tuesday, February  4, 2003  5:05 PM

> "Easy enough to check compatibility"? You've got to be kidding. The only
> thing that we know how to do with a symbol file is load them into a symbol
> table. But you can't do that to "check compatibility", because you've got to
> load the full view into the same place (can't have two packages with the
> same library-level name).

This is my whole point.  You *do* need to have both views available
at once, since one unit might have a "limited with" of package P,
while some unit that it depends on indirectly has a full "with" of P.
To avoid ripple effects, we want the one with the limited-with
to only see the limited view.

You mentioned in another note that you have separate symbol table
entries for various different forms of a type.  I think you will
need separate symbol table entries for the limited view and
the full view.  In other words, for a library package P, you really have two units,
one whose library-level name is "P-full" and one whose name is "P-limited".

> Keep in mind that no one actually "obsoletes" anything. How we do
> obsoleteness checking is simply to compare the time stamp serial numbers of
> every unit withed transitively for every unit withed. They better all match.

Yes, I understand that approach. I probably should have simply
talked in terms of bumping time stamps.  I think the same
point can be made in those terms.  We have a similar model,
albeit only in the program library we build up in memory.
[Since we allow the compiler to run for a "long" time and
to process different versions of the same file during a single
execution, we have all the obsoleteness checking mechanism
in there as well.]

**************************************************************

From: Randy Brukardt
Sent: Tuesday, February  4, 2003  5:34 PM

> This is my whole point.  You *do* need to have both views available
> at once, since one unit might have a "limited with" of package P,
> while some unit that it depends on indirectly has a full "with" of P.
> To avoid ripple effects, we want the one with the limited-with
> to only see the limited view.

I don't see that. Each compilation has its own symbol table, and I don't see
any reason why the mere fact of withing something that saw a limited view of
some package was anything to do with a package that sees the full view.

> You mentioned in another note that you have separate symbol table
> entries for various different forms of a type.

No, "types" aren't in the symbol table at all. They have their own separate
table. Type names are in the symbol table, of course, as are component
names. But a type name has nothing to do with a type -- they're completely
separate concepts (as they are in Ada).

> I think you will
> need separate symbol table entries for the limited view and
> the full view.  In other words, for a library package P, you really have
> two units, one whose library-level name is "P-full" and one whose name
> is "P-limited".

If that's true, we're getting into horrific complexity territory. A
fundamental basis of the symbol table is that most entities have only one
name and cannot be overloaded. You're asking that all of the lookup code be
changed to be able to handle packages with two views. Along with all of the
declaration code (so it can write into the correct view). Moreover, that
would be true in every compiler (source based or library based).

If the proposal requires both copies in the symboltable, then I think the
proposal should be killed as soon as possible. If not sooner.

**************************************************************

From: Pascal Leroy
Sent: Wednesday, February  5, 2003  3:40 AM

> I think you need to keep around indefinitely both a "full view" of
> the package and a "limited view" of the package.  Even after you
> do a "full" compile, some units can still do "limited with" and should only
> see the limited view.  What I would recommend is you treat them as
> pretty much distinct units.

In terms of language description, you're right, a unit that does a "limited
with" must see the limited view even for a unit that has been fully compiled.

However, the implementation you suggest might make sense for your compiler, it
doesn't make sense for ours (or for Randy's if I understand him right).  We
fundamentally depend on the invariant: 1 Ada unit = 1 Diana tree.  Changing this
is not an option.  This means that if we see "limited with P" and P has been
fully compiled, then we need to simulate/synthesize a limited view for P.

For name resolution, this is simple enough.  When we see the name P.T we do a
lookup of the string "T" in the identifier table for P.  The lookup has to
succeed if and only if T is part of the limited view.  We would probably do that
by examining the Diana tree for T.  Another option would be to store a bit "yes,
I am part of the limited view" on the defining identifier for T.

Once name resolution has been done, we will have access to the full tree for T.
At this point we will need to behave as if T was incomplete.  I can think of at
least 3 ways to achieve this in our compiler, but the simplest one is probably
to use a predicate "is this an incomplete type?" where appropriate, and have
that predicate determine whether visibility was obtained though a "limited
with".  We already have such a predicate, but I'm pretty sure that we don't call
it everywhere it would be needed.

This certainly looks like work, but we are not talking man-years here.  And
there is certainly no need to keep two trees for each unit.

**************************************************************

From: Jean-Pierre Rosen
Sent: Wednesday, February  5, 2003  3:32 AM

> Add a legality rule:
>
> A library_item mentioned in a limited_with_clause shall be a
> package_declaration[, not a subprogram_declaration, generic_declaration,
> or generic_instantiation].

Why not generic_instantiation?
1) not really different from a package declaration
2) quite useful, since instantiations are commonly used for building objects with
   multiple facets.

Later you say:
>We do not allow a limited_with_clause to mention a generic
>instantiation, because that would require all kinds of semantic analysis
>stuff, such as visibility.
Well, the instantiation has been compiled at that point, so all visibility
should be solved.
Maybe worth some further investigation

**************************************************************

From: Pascal Leroy
Sent: Wednesday, February  5, 2003  3:59 AM

The restriction is perfectly sensible.  We don't want to have to do any name
resolution to build the limited view, and in order to determine what generic we
are talking about, we would have to do a pretty extensive name resolution (use
clauses, parent units, renamings, etc.).  Moreover, in order to build the
instantiation, we would have to have compiled the specification of the generic
already, but limited views cannot have compilation prerequisites (if they had,
we could run into circularity problems).

**************************************************************

From: Robert A. Duff
Sent: Wednesday, February  5, 2003  8:22 AM

Tucker and I discussed this issue yesterday, and we decided that
although it *is* troubling to make rules that cause package instances to
be different from normal packages, it is not feasible to implement the
proposal (in some compilers) without this restriction.

If there's a cycle, then the instantiation might *not* have been
compiled yet.  How about:

    limited with A;
    package B is new Some_Generic(...);

    limited with B;
    package A is new Some_Generic(...);

It cannot be the case that A and B have both been *fully* compiled
before each other.

I really think a key feature of this proposal is that when the compiler
sees "limited with X", it can determine the list of incomplete types in
X in a purely syntactic manner.  It should not have to do any kind of
heavy-duty semantic analysis.  It should not have to look at any source
text outside of X.

It's not *so* bad, because if you wanted to do cycles like the above,
you can always break the cycle by adding more generic formal parameters.

**************************************************************

From: Jean-Pierre Rosen
Sent: Wednesday, February  5, 2003  11:28 AM

Yes. I see the problem, and I have sympathy for the poor compiler writers.

But with my teacher's hat on, I don't feel very easy to explain that in some
cases, an instantiation is not equivalent to a regular package.

**************************************************************

From: Tucker Taft
Sent: Wednesday, February  5, 2003  5:47 AM

Pascal Leroy wrote:
>...

>
> From a pedagogical standpoint it would seem simpler to say that "limited
> with" works like "with", i.e. that "limited with P.C.D" is equivalent to
> "with P, P.C, P.C.D".  Any other option is going to unnecessary confuse
> users.
>
> So maybe the right answer is that if a package has a "limited with" of
> itself it has no effect (but is not illegal).

Yes, that is preferable.

> Yes, I was thinking that use clauses and package renamings would be OK in
> that context, although if they lead to problems we could disallow them.  But
> I can't think of any problem at this point.

I sent a note about this, titled "Restrictions on limited-with".
Have you seen it?  Any comments?
I think "use" clauses are a bad idea.
Renaming is tricky, and needs careful thought.

**************************************************************

From: Pascal Leroy
Sent: Wednesday, February  5, 2003  6:23 AM

Yes, I saw it, but after sending the above message.  I agree that use clauses
cause a Beaujolais-ish effect (maybe this should be named the Chianti effect ;-)
and should be disallowed.  I also agree that renamings should probably be
disallowed since they cause more trouble than they are worth.

**************************************************************

From: Robert Dewar
Sent: Wednesday, February  5, 2003  7:18 AM

So far it seems to me that the limited-with discussion looks like it
is very promising. I must say I was unhappy with the idea of furiously
trying to get a resolution in Padua (and I was trying to figure out
how to molest my schedule to attend :-) but now I am actually getting
some confidence that

a) this is going in the right direction
b) there really is a good possibility of agreement

Very encouraging :-)

We will definitely try to prototype this in GNAT as soon as there is a
reasonably well defined proposal.

**************************************************************

From: Tucker Taft
Sent: Tuesday, February 11, 2003  2:21 PM

Here is some initial analysis of implementation issues associated
with "limited with."  I tried to make most of the analysis
technology independent, but of course that is nearly impossible,
so it may apply less or more to technologies other than AdaMagic.
-Tuck

-----------------
Thoughts on the implementation of "limited with" in AdaMagic

          $Revision: 1.14 $ $Date: 2004/06/25 01:30:52 $

The "limited with" clause makes a "limited view"
of a package visible in the compilation unit
that has the limited-with clause, as well as all of its
descendants.  However, the limited view is
hidden from all visibility (as are all the incomplete
types and nested package limited views within it) under
certain circumstances.  After much discussion at
the Padua ARG meeting, the best rule seemed
to be that if the full view of the package is
visible, including via one or more renamings,
then the limited view is hidden from all visibility.
[This was considerd necessary so as to avoid having two different
views of the same package usable at the same point
in the source text, which was felt to create both
implementation complexity and possible user confusion.]

SEPARATE "LIMITED" COMPILATION UNIT

Our model for implementing a limited view of a library package
will be that it has its "own" compilation-unit name,
constructed from its "real" name but with suffix "'Limited".
Hence P.Q.R'Limited is the limited view of P.Q.R.
[Note that "P.Q.R.Limited" could also be used as the
name, since "limited" is a reserved word, and would
never be the real name of a package.  However, it is
very important that in any case P.Q.R.Limited or
P.Q.R'Limited not depend semantically on P.Q.R itself.]

In the representation of the library, it is important
that P.Q.R'Limited can exist before P.Q.R has been
fully compiled, and can retain some continuing existence
while P.Q.R is being fully compiled, and thereafter.
It is also important to realize that the packages
nested within P.Q.R'Limited are themselves limited
views.  Hence, P.Q.R'Limited.NP always denotes a limited view
of a package, if P.Q.R.NP denotes a nested package spec.  Similarly,
P.Q.R'Limited.T denotes an incomplete type declaration for T,
if P.Q.R.T denotes a full type declaration.

SEPARATE TIMESTAMP FOR LIMITED VIEW

P.Q.R'Limited has its own timestamp, for the purposes
of out-of-dateness checking.  It is important that this
*not* change when a "full" compile of P.Q.R is started,
because that would make the units that P.Q.R depends
on that themselves depend on P.Q.R'Limited, automatically
appear out of date.

Once the full compile of P.Q.R is complete, a check for
consistency between P.Q.R'Limited and P.Q.R should be
performed.  If the views are no longer consistent, due to
some change in P.Q.R, a new version of P.Q.R'Limited should
be constructed, and its timestamp changed.  This will effectively
put all the dependents of P.Q.R'Limited out of date,
and may in fact put P.Q.R itself out of date.  If so,
then P.Q.R will have to be compiled again.  Ideally, this
will be performed immediately, since the source code for
P.Q.R is readily available at that time.  However, depending on the
compilation mechanism, it may be first necessary to compile
other dependents of P.Q.R'Limited, so they are back in
an up-to-date state before re-attempting the full
compilation of P.Q.R.  This might be left until link time,
or to a separate "make" tool.

Of course if P.Q.R'Limited is initially created by simply parsing
the source text for P.Q.R, the timestamp of P.Q.R'Limited would be
set at parse time.

For those compilation systems that retain the date of modification
of the source file as the time stamp of the corresponding units
in the program library, some of this becomes simpler.
Nevertheless, it seems a helpful, and in some cases
necessary, "optimization" to *not* change the timestamp of
the limited view every time the source text timestamp changes.
Because the check for consistency between a full view and
a limited view should be relatively easy to perform, there
seems no need to change the timestamp on the limited view
so long as it remains consistent with the full view, even
if the full view corresponds to source text with a different
timestamp.  This means that units that depend only on the
P.Q.R'Limited are isolated from "minor" changes to P.Q.R,
and are only affected when a type or nested package is
added or removed from P.Q.R.   In other words, you get
a "poor man's" version of incremental compilation as a side
effect of implementing limited views of packages.

HIDING THE LIMITED VIEW

As mentioned above, a limited view of a package can be hidden from
all visibility.  The anticipated rule is that the limited
view is hidden from all visibility if the full view is visible
either "normally" or via a rename.  This implies that if
P.Q.R is visible, including via a rename, then P.Q.R'Limited
is not visible.  Furthermore, even when P.Q.R'Limited
is visible, it is possible that P.Q.R'Limited.NP might
not be visible, because P.Q.R.NP might be visible via
a renaming.

A relatively simple implementation of this requirement
would seem to be to build up a set representing those (external) packages
that have a full view visible, including via a rename, within the
current compilation unit.  (By "external" we mean it is from some
other compilation unit.)  This set will not grow once the
context clause has been fully processed, since it is not
possible to locally declare a renaming of a package that is not
already visible.

This set of visible packages would probably be represented
using a hash table of some sort.  It would start out empty
when starting to compile a new compilation unit, and would
grow each time a (real, not limited) "with" for a package or a package
renaming is processed.  Each of these "with"ed packages
would be scanned for enclosed renamings, recursively.
[Alternatively, the representation of every package could
include a set of all enclosed renamings (directly or indirectly).]
The "with"ed package, plus all of the packages for which it
contains a renaming, would be unioned into this set.

Later, whenever a limited view of a package is encountered
within a name, a check is made to see whether a corresponding
full view exists in this set.  If so, the limited view may
not be named, and so the name is illegal.

Another anticipated rule is that a limited with clause is
illegal if the package is already visible, including
via a rename.  This would seem to be friendlier than allowing
the programmer to put a "limited with" on a compilation unit and
then not be able to use it at all.  However, there will still
be "limited with"s inherited from ancestor units, and these
will be essentially ignored if the corresponding package
is fully visible some other way.  Also, in a limited with
clause like "limited with P.Q.R;" presumably implicit
limited withs for P and P.Q are provided, but such an implicit
limtied with again must be ignored if the package mentioned
is already fully visible via some other means.

To enforce the above rule regarding disallowing and/or ignoring
limited withs, it may be necessary to do a second pass over
the context clause, after the full set of visible packages has
been built up.  In many cases, a second pass is already necessary
in Ada 95 to correctly deal with "with"s of private units, since it is
not known whether those with's are legal until the name of the
compilation unit being compiled is known, so it may be possible
to combine these error-checking passes.

WHEN IS THE INCOMPLETE TYPE COMPLETE?

Our current plan is to disallow "use" clauses and renamings
for limited views.  Given that, the only way to name one
of the incomplete types inside a limited view is by naming
the enclosing package, so there is no need to have a separate mechanism
for determining whether the incomplete type declaration is visible.
It is visible exactly when the enclosing limited view can be named.

However, there are access types that might have been declared
using the incomplete type, and it is important to know when a
dereference of the access type is considered "complete."
The basic rule would be that when the full type is visible,
including via a rename of the enclosing package, (or equivalently,
when the incomplete type declaration is hidden from all visibility),
the access type may be dereferenced to produce the full type.
This implies that any time such an access type is dereferenced,
a check should be made whether the designated type's enclosing limited
view(s) are all still visible.  If they are all still visible,
then the incomplete type is still visible, and the dereference
produces an incomplete object.  If any one of them is hidden from
all visibility, then the full type is visible, and the dereference
produces an object of a "full" type.  Some optimization of this
check is probably possible.

Even when a dereference of the access type is considered
incomplete, it may still be legal in a context where a full
type would normally be required, so long as the expected type
is the full type, or the full type is "nearby."  One of the
versions of the type-stub proposal laid out the rules for
this matching between incomplete and full type, and those
rules seem still relevant for all of the other proposals as well.

Here is a copy of those rules (from AI-00217-04/04):

    A dereference (implicit or explicit) of a value of an access type whose
    designated type D is incomplete is allowed only in the following
    contexts:

    * in a place where the completion of D is available (see above);

    * in a context where the expected type is E and
       o E covers the completion of D,
       o E is tagged and covers D,
       o E covers D'Class or its completion, or
       o E'Class covers D or its completion;

    * as the target of an assignment_statement where the type of the value
      being assigned is V, and V or V'Class is the completion of D.

    In these contexts, the incomplete type is defined to be the same
    type as its completion, and its first subtype statically matches the
    first subtype of its completion.

The first bullet would be replaced by "where the full type declaration
for D is visible" for the limited-with proposal (and for the
"type C.T;" proposal).

**************************************************************

From: Robert A. Duff
Sent: Tuesday, February 11, 2003  5:34 PM

...
> SEPARATE "LIMITED" COMPILATION UNIT
>
> Our model for implementing a limited view of a library package
> will be that it has its "own" compilation-unit name,
> constructed from its "real" name but with suffix "'Limited".

I don't see why you want it to have a different name.  It seems to me
that there are two declarations of views, both called P.Q.R, and the
full view hides the limited view.

> Hence P.Q.R'Limited is the limited view of P.Q.R.
> [Note that "P.Q.R.Limited" could also be used as the
> name, since "limited" is a reserved word, and would
> never be the real name of a package.  However, it is
> very important that in any case P.Q.R.Limited or
> P.Q.R'Limited not depend semantically on P.Q.R itself.]

Right -- the dependence is the other way 'round.

> In the representation of the library, it is important
> that P.Q.R'Limited can exist before P.Q.R has been
> fully compiled, and can retain some continuing existence
> while P.Q.R is being fully compiled, and thereafter.
> It is also important to realize that the packages
> nested within P.Q.R'Limited are themselves limited
> views.

And *they* don't get weird names...

>...Hence, P.Q.R'Limited.NP always denotes a limited view
> of a package, if P.Q.R.NP denotes a nested package spec.  Similarly,
> P.Q.R'Limited.T denotes an incomplete type declaration for T,
> if P.Q.R.T denotes a full type declaration.
>
> SEPARATE TIMESTAMP FOR LIMITED VIEW
>
> P.Q.R'Limited has its own timestamp, for the purposes
> of out-of-dateness checking.  It is important that this
> *not* change when a "full" compile of P.Q.R is started,
> because that would make the units that P.Q.R depends
> on that themselves depend on P.Q.R'Limited, automatically
> appear out of date.

Right.  It seems to me that in a source-based library model, the
timestamp of the limited view can be the timestamp of the source file
containing that package, whereas the timestamp of the full view is
really a collection of timestamps for all the source files the package
depends upon.

> Once the full compile of P.Q.R is complete, a check for
> consistency between P.Q.R'Limited and P.Q.R should be
> performed.

I don't understand the need for this consistency check.
It seems to me that P.Q.R depends upon P.Q.R'Limited,
and you do the normal check at link time, to make sure that
two different versions of P.Q.R'Limited are not included in the same
partition.

In normal operation, you just do a "build" command, and the only way the
link-time check can fail is if you edit your source code while the build
is running.

>...  If the views are no longer consistent, due to
> some change in P.Q.R, a new version of P.Q.R'Limited should
> be constructed, and its timestamp changed.  This will effectively
> put all the dependents of P.Q.R'Limited out of date,
> and may in fact put P.Q.R itself out of date.  If so,
> then P.Q.R will have to be compiled again.  Ideally, this
> will be performed immediately, since the source code for
> P.Q.R is readily available at that time.  However, depending on the
> compilation mechanism, it may be first necessary to compile
> other dependents of P.Q.R'Limited, so they are back in
> an up-to-date state before re-attempting the full
> compilation of P.Q.R.  This might be left until link time,
> or to a separate "make" tool.

I don't see the need for such complexity in the "make" tool
(recompiling things twice and whatnot).  It seems no different
than the current language: if you edit package Foo in between compiling
two different things that depend on Foo, then you get a link-time
failure, and run the "make" tool again.

> Of course if P.Q.R'Limited is initially created by simply parsing
> the source text for P.Q.R, the timestamp of P.Q.R'Limited would be
> set at parse time.
>
> For those compilation systems that retain the date of modification
> of the source file as the time stamp of the corresponding units
> in the program library, some of this becomes simpler.
> Nevertheless, it seems a helpful, and in some cases
> necessary, "optimization" to *not* change the timestamp of
> the limited view every time the source text timestamp changes.
> Because the check for consistency between a full view and
> a limited view should be relatively easy to perform, there
> seems no need to change the timestamp on the limited view
> so long as it remains consistent with the full view, even
> if the full view corresponds to source text with a different
> timestamp.  This means that units that depend only on the
> P.Q.R'Limited are isolated from "minor" changes to P.Q.R,
> and are only affected when a type or nested package is
> added or removed from P.Q.R.   In other words, you get
> a "poor man's" version of incremental compilation as a side
> effect of implementing limited views of packages.

Seems like a nice optimization, but I don't see why it's necessary.

> HIDING THE LIMITED VIEW
>
> As mentioned above, a limited view of a package can be hidden from
> all visibility.  The anticipated rule is that the limited
> view is hidden from all visibility if the full view is visible
> either "normally" or via a rename.  This implies that if
> P.Q.R is visible, including via a rename, then P.Q.R'Limited
> is not visible.  Furthermore, even when P.Q.R'Limited
> is visible, it is possible that P.Q.R'Limited.NP might
> not be visible, because P.Q.R.NP might be visible via
> a renaming.
>
> A relatively simple implementation of this requirement
> would seem to be to build up a set representing those (external) packages
> that have a full view visible, including via a rename, within the
> current compilation unit.  (By "external" we mean it is from some
> other compilation unit.)  This set will not grow once the
> context clause has been fully processed, since it is not
> possible to locally declare a renaming of a package that is not
> already visible.

Well, sort of.  Not just the context clause, but the parent unit name,
and all the context clauses that get inherited by children and subunits
and bodies.

> This set of visible packages would probably be represented
> using a hash table of some sort.  It would start out empty
> when starting to compile a new compilation unit, and would
> grow each time a (real, not limited) "with" for a package or a package
> renaming is processed.  Each of these "with"ed packages
> would be scanned for enclosed renamings, recursively.
> [Alternatively, the representation of every package could
> include a set of all enclosed renamings (directly or indirectly).]
> The "with"ed package, plus all of the packages for which it
> contains a renaming, would be unioned into this set.
>
> Later, whenever a limited view of a package is encountered
> within a name, a check is made to see whether a corresponding
> full view exists in this set.  If so, the limited view may
> not be named, and so the name is illegal.

I don't undersstand this.  First, it seems like this is all part of
overload resolution: you see a name, and it will resolve to the full
view, if the full view is visible.

Second, why do you concentrate on names?  I mean, you run across types
in the symbol table all the time, and it seems like you need to make the
decision there as well.  (Consider the examples in my earlier e-mail,
where you refered to a variable that is of the type, a subtype of the
type, or whatever.)

> Another anticipated rule is that a limited with clause is
> illegal if the package is already visible, including
> via a rename.  This would seem to be friendlier than allowing
> the programmer to put a "limited with" on a compilation unit and
> then not be able to use it at all.

Perhaps a little bit friendlier, but it's not the simplest rule,
and you're not really protecting the user against any serious damage.

>...  However, there will still
> be "limited with"s inherited from ancestor units, and these
> will be essentially ignored if the corresponding package
> is fully visible some other way.  Also, in a limited with
> clause like "limited with P.Q.R;" presumably implicit
> limited withs for P and P.Q are provided, but such an implicit
> limtied with again must be ignored if the package mentioned
> is already fully visible via some other means.
>
> To enforce the above rule regarding disallowing and/or ignoring
> limited withs, it may be necessary to do a second pass over
> the context clause, after the full set of visible packages has
> been built up.  In many cases, a second pass is already necessary
> in Ada 95 to correctly deal with "with"s of private units, since it is
> not known whether those with's are legal until the name of the
> compilation unit being compiled is known, so it may be possible
> to combine these error-checking passes.

The existing rule requires looking at the comp unit name before
processing the context clause.  It doesn't require two passes over the
context clause.  (Either way, it's not the "one pass semantics" that you
and Robert Dewar like so much.  ;-) )

> WHEN IS THE INCOMPLETE TYPE COMPLETE?
>
> Our current plan is to disallow "use" clauses and renamings
> for limited views.  Given that, the only way to name one
> of the incomplete types inside a limited view is by naming
> the enclosing package, so there is no need to have a separate mechanism
> for determining whether the incomplete type declaration is visible.
> It is visible exactly when the enclosing limited view can be named.
>
> However, there are access types that might have been declared
> using the incomplete type, and it is important to know when a
> dereference of the access type is considered "complete."

And parameters (in the tagged case).  Normally, the body will 'with' the
full view, so it can do stuff with the type.  But if you forget that
'with', the compiler needs to notice that the type is still incomplete.

> The basic rule would be that when the full type is visible,
> including via a rename of the enclosing package, (or equivalently,
> when the incomplete type declaration is hidden from all visibility),
> the access type may be dereferenced to produce the full type.
> This implies that any time such an access type is dereferenced,
> a check should be made whether the designated type's enclosing limited
> view(s) are all still visible.  If they are all still visible,
> then the incomplete type is still visible, and the dereference
> produces an incomplete object.

I don't think you mean, "produces an incomplete object".  I think you
mean, simply, "is illegal".

I'm not sure I understand the "all" here.  I mean, just because the full
view of *one* such containing package is visible doesn't make the full
view of the type visible, does it?

>...If any one of them is hidden from
> all visibility, then the full type is visible, and the dereference
> produces an object of a "full" type.  Some optimization of this
> check is probably possible.
>
> Even when a dereference of the access type is considered
> incomplete, it may still be legal in a context where a full
> type would normally be required, so long as the expected type
> is the full type, or the full type is "nearby."  One of the
> versions of the type-stub proposal laid out the rules for
> this matching between incomplete and full type, and those
> rules seem still relevant for all of the other proposals as well.

Oh, I see.  So what I said above is wrong.  Hmm.

> Here is a copy of those rules (from AI-00217-04/04):
>
>     A dereference (implicit or explicit) of a value of an access type whose
>     designated type D is incomplete is allowed only in the following
>     contexts:
>
>     * in a place where the completion of D is available (see above);
>
>     * in a context where the expected type is E and
>        o E covers the completion of D,
>        o E is tagged and covers D,
>        o E covers D'Class or its completion, or
>        o E'Class covers D or its completion;
>
>     * as the target of an assignment_statement where the type of the value
>       being assigned is V, and V or V'Class is the completion of D.

How can V'Class be the completion of something?

>     In these contexts, the incomplete type is defined to be the same
>     type as its completion, and its first subtype statically matches the
>     first subtype of its completion.
>
> The first bullet would be replaced by "where the full type declaration
> for D is visible" for the limited-with proposal (and for the
> "type C.T;" proposal).

**************************************************************

From: Tucker Taft
Sent: Tuesday, February 11, 2003  10:28 PM

Robert A Duff wrote:
>>SEPARATE "LIMITED" COMPILATION UNIT
>>
>>Our model for implementing a limited view of a library package
>>will be that it has its "own" compilation-unit name,
>>constructed from its "real" name but with suffix "'Limited".
>
>
> I don't see why you want it to have a different name.  It seems to me
> that there are two declarations of views, both called P.Q.R, and the
> full view hides the limited view.

Some of this is in reaction to Randy and Pascal's
comments on my earlier implementation musings.
They don't like the idea of having two units in the library with
the same name.  It breaks their model badly.  So
I was proposing that they have different names,
at least as far as the program library mechanism is
concerned.  In general, the program library mechanism
is presumed to be too stupid to understand hiding.
And the limited view might be needed even when the
full view has been added to the program library, because
a unit explicitly requests the limited view via "limited with."

>>Hence P.Q.R'Limited is the limited view of P.Q.R.
>>[Note that "P.Q.R.Limited" could also be used as the
>>name, since "limited" is a reserved word, and would
>>never be the real name of a package.  However, it is
>>very important that in any case P.Q.R.Limited or
>>P.Q.R'Limited not depend semantically on P.Q.R itself.]
>
>
> Right -- the dependence is the other way 'round.

If there is any dependence at all.  They both depend
on the source text for P.Q.R, but only the full view
depends on the source text for other units on which
it is semantically dependent.

>>In the representation of the library, it is important
>>that P.Q.R'Limited can exist before P.Q.R has been
>>fully compiled, and can retain some continuing existence
>>while P.Q.R is being fully compiled, and thereafter.
>>It is also important to realize that the packages
>>nested within P.Q.R'Limited are themselves limited
>>views.
>
>
> And *they* don't get weird names...

Right.  The "weird" names are just so the program
library mechanism can be relatively stupid, and
just deliver up one and only one unit given a particular
name.

>>...Hence, P.Q.R'Limited.NP always denotes a limited view
>>of a package, if P.Q.R.NP denotes a nested package spec.  Similarly,
>>P.Q.R'Limited.T denotes an incomplete type declaration for T,
>>if P.Q.R.T denotes a full type declaration.
>>
>>SEPARATE TIMESTAMP FOR LIMITED VIEW
>>
>>P.Q.R'Limited has its own timestamp, for the purposes
>>of out-of-dateness checking.  It is important that this
>>*not* change when a "full" compile of P.Q.R is started,
>>because that would make the units that P.Q.R depends
>>on that themselves depend on P.Q.R'Limited, automatically
>>appear out of date.
>
>
> Right.  It seems to me that in a source-based library model, the
> timestamp of the limited view can be the timestamp of the source file
> containing that package, whereas the timestamp of the full view is
> really a collection of timestamps for all the source files the package
> depends upon.

I was trying to handle various approaches to time stamps.
In our model, each compiled representation keeps the
timestamps of all the source files it depends on.  As you
imply above, P.Q.R'Limited would depend on exactly one
source file ("p.q.r.spc", for example), whereas P.Q.R would
probably depend on many source files, including p.q.r.spc
as well.

I believe in Randy's model, there is a timestamp
associated with the compiled representation itself, and
it must be >= the timestamps of all other compiled representations
it depends on.  I don't believe he keeps source file timestamps
at all.  Timestamps are associated with when a unit is submitted
to the compiler, I believe.

I think Rational has aspects of both timestamp models, though
I will let Pascal talk about the details.

>>Once the full compile of P.Q.R is complete, a check for
>>consistency between P.Q.R'Limited and P.Q.R should be
>>performed.
>
> I don't understand the need for this consistency check.
> It seems to me that P.Q.R depends upon P.Q.R'Limited,
> and you do the normal check at link time, to make sure that
> two different versions of P.Q.R'Limited are not included in the same
> partition.
>
> In normal operation, you just do a "build" command, and the only way the
> link-time check can fail is if you edit your source code while the build
> is running.

Again, I was trying to accommodate a model where timestamps
are associated with compiled representations, rather than
only with source text.  I agree that a purely source-based
model can be simpler.

>>...  If the views are no longer consistent, due to
>>some change in P.Q.R, a new version of P.Q.R'Limited should
>>be constructed, and its timestamp changed.  This will effectively
>>put all the dependents of P.Q.R'Limited out of date,
>>and may in fact put P.Q.R itself out of date.  If so,
>>then P.Q.R will have to be compiled again.  Ideally, this
>>will be performed immediately, since the source code for
>>P.Q.R is readily available at that time.  However, depending on the
>>compilation mechanism, it may be first necessary to compile
>>other dependents of P.Q.R'Limited, so they are back in
>>an up-to-date state before re-attempting the full
>>compilation of P.Q.R.  This might be left until link time,
>>or to a separate "make" tool.
>
>
> I don't see the need for such complexity in the "make" tool
> (recompiling things twice and whatnot).  It seems no different
> than the current language: if you edit package Foo in between compiling
> two different things that depend on Foo, then you get a link-time
> failure, and run the "make" tool again.

Yes, I agree the source-based model can be simpler.  However,
Randy seemed to find the timestamp issue a big problem with
"limited with," so I was making some attempt to address that.
I probably shouldn't have strayed away from the source-based
model.

>>Of course if P.Q.R'Limited is initially created by simply parsing
>>the source text for P.Q.R, the timestamp of P.Q.R'Limited would be
>>set at parse time.
>>
>>For those compilation systems that retain the date of modification
>>of the source file as the time stamp of the corresponding units
>>in the program library, some of this becomes simpler.
>>Nevertheless, it seems a helpful, and in some cases
>>necessary, "optimization" to *not* change the timestamp of
>>the limited view every time the source text timestamp changes.
>>Because the check for consistency between a full view and
>>a limited view should be relatively easy to perform, there
>>seems no need to change the timestamp on the limited view
>>so long as it remains consistent with the full view, even
>>if the full view corresponds to source text with a different
>>timestamp.  This means that units that depend only on the
>>P.Q.R'Limited are isolated from "minor" changes to P.Q.R,
>>and are only affected when a type or nested package is
>>added or removed from P.Q.R.   In other words, you get
>>a "poor man's" version of incremental compilation as a side
>>effect of implementing limited views of packages.
>
>
> Seems like a nice optimization, but I don't see why it's necessary.

I think it might be necessary in the non-source-based
model, if timestamps are associated with the time when units
are submitted to the compiler, rather than with when
the source text is edited.


>
>
>>HIDING THE LIMITED VIEW
>>
>>As mentioned above, a limited view of a package can be hidden from
>>all visibility.  The anticipated rule is that the limited
>>view is hidden from all visibility if the full view is visible
>>either "normally" or via a rename.  This implies that if
>>P.Q.R is visible, including via a rename, then P.Q.R'Limited
>>is not visible.  Furthermore, even when P.Q.R'Limited
>>is visible, it is possible that P.Q.R'Limited.NP might
>>not be visible, because P.Q.R.NP might be visible via
>>a renaming.
>>
>>A relatively simple implementation of this requirement
>>would seem to be to build up a set representing those (external) packages
>>that have a full view visible, including via a rename, within the
>>current compilation unit.  (By "external" we mean it is from some
>>other compilation unit.)  This set will not grow once the
>>context clause has been fully processed, since it is not
>>possible to locally declare a renaming of a package that is not
>>already visible.
>
>
> Well, sort of.  Not just the context clause, but the parent unit name,
> and all the context clauses that get inherited by children and subunits
> and bodies.

Right.

>
>
>>This set of visible packages would probably be represented
>>using a hash table of some sort.  It would start out empty
>>when starting to compile a new compilation unit, and would
>>grow each time a (real, not limited) "with" for a package or a package
>>renaming is processed.  Each of these "with"ed packages
>>would be scanned for enclosed renamings, recursively.
>>[Alternatively, the representation of every package could
>>include a set of all enclosed renamings (directly or indirectly).]
>>The "with"ed package, plus all of the packages for which it
>>contains a renaming, would be unioned into this set.
>>
>>Later, whenever a limited view of a package is encountered
>>within a name, a check is made to see whether a corresponding
>>full view exists in this set.  If so, the limited view may
>>not be named, and so the name is illegal.
>
>
> I don't undersstand this.  First, it seems like this is all part of
> overload resolution: you see a name, and it will resolve to the full
> view, if the full view is visible.

The problem is package renames.  We concluded that if there
is a renaming of P.Q.R visible, then the limited view of
P.Q.R should become invisible, even though "P.Q.R" does not
denote the full view.  So this is a case where a name is
hidden from all visibility, even though it isn't a homograph
with the thing doing the hiding.  This is somewhat like a more
extreme version of the situation where having one overloadable
in a use-visible set hides all the non-overloadables in the
use-visible set, and vice versa.  Here, S.Renaming_Of_P_Q_R
is hiding P.Q.R'Limited from all visibility.

>
> Second, why do you concentrate on names?  I mean, you run across types
> in the symbol table all the time, and it seems like you need to make the
> decision there as well.

Well, I talk about that later.  If the types are already in the
symbol table, the question is whether they should be considered
incomplete or complete.  The name issue is whether the name is
even visible.


> ...
>  (Consider the examples in my earlier e-mail,
> where you refered to a variable that is of the type, a subtype of the
> type, or whatever.)

I gave an adequate explanation of that there, hopefully.

>
>
>>Another anticipated rule is that a limited with clause is
>>illegal if the package is already visible, including
>>via a rename.  This would seem to be friendlier than allowing
>>the programmer to put a "limited with" on a compilation unit and
>>then not be able to use it at all.
>
>
> Perhaps a little bit friendlier, but it's not the simplest rule,
> and you're not really protecting the user against any serious damage.

That's true.  But it would seem mysterious if the reason
the limited with has no effect is because there is a renaming
in some "with"ed package.  I think the programmer should be
told about that as soon as possible, and the limited with
clause seems like the ideal place.

>
>
>>...  However, there will still
>>be "limited with"s inherited from ancestor units, and these
>>will be essentially ignored if the corresponding package
>>is fully visible some other way.  Also, in a limited with
>>clause like "limited with P.Q.R;" presumably implicit
>>limited withs for P and P.Q are provided, but such an implicit
>>limtied with again must be ignored if the package mentioned
>>is already fully visible via some other means.
>>
>>To enforce the above rule regarding disallowing and/or ignoring
>>limited withs, it may be necessary to do a second pass over
>>the context clause, after the full set of visible packages has
>>been built up.  In many cases, a second pass is already necessary
>>in Ada 95 to correctly deal with "with"s of private units, since it is
>>not known whether those with's are legal until the name of the
>>compilation unit being compiled is known, so it may be possible
>>to combine these error-checking passes.
>
>
> The existing rule requires looking at the comp unit name before
> processing the context clause.  It doesn't require two passes over the
> context clause.  (Either way, it's not the "one pass semantics" that you
> and Robert Dewar like so much.  ;-) )

You can do it either way.  AdaMagic doesn't peak ahead at the comp-unit
name.  Instead, it processes all the "with" clauses and then
when it reaches the comp-unit name, it checks to be sure they
are legal.  So if you do it that way, you already have that
second pass over the context clause.

>>WHEN IS THE INCOMPLETE TYPE COMPLETE?
>>
>>Our current plan is to disallow "use" clauses and renamings
>>for limited views.  Given that, the only way to name one
>>of the incomplete types inside a limited view is by naming
>>the enclosing package, so there is no need to have a separate mechanism
>>for determining whether the incomplete type declaration is visible.
>>It is visible exactly when the enclosing limited view can be named.
>>
>>However, there are access types that might have been declared
>>using the incomplete type, and it is important to know when a
>>dereference of the access type is considered "complete."
>
>
> And parameters (in the tagged case).  Normally, the body will 'with' the
> full view, so it can do stuff with the type.  But if you forget that
> 'with', the compiler needs to notice that the type is still incomplete.

Right.

>>The basic rule would be that when the full type is visible,
>>including via a rename of the enclosing package, (or equivalently,
>>when the incomplete type declaration is hidden from all visibility),
>>the access type may be dereferenced to produce the full type.
>>This implies that any time such an access type is dereferenced,
>>a check should be made whether the designated type's enclosing limited
>>view(s) are all still visible.  If they are all still visible,
>>then the incomplete type is still visible, and the dereference
>>produces an incomplete object.
>
>
> I don't think you mean, "produces an incomplete object".  I think you
> mean, simply, "is illegal".

That used to be true.  But with incomplete tagged types, it is
legal to dereference under certain circumstances.  Also, the
"extra" matching rules also imply that the dereference can be legal,
even for a non-tagged incomplete type.

> I'm not sure I understand the "all" here.  I mean, just because the full
> view of *one* such containing package is visible doesn't make the full
> view of the type visible, does it?

Yes, if the full view of any containing package is visible,
then everything inside the package is definitely visible.
By "containing" I should say "textually" containing.  I don't
mean to include ancestor library packages.

>>...If any one of them is hidden from
>>all visibility, then the full type is visible, and the dereference
>>produces an object of a "full" type.  Some optimization of this
>>check is probably possible.
>>
>>Even when a dereference of the access type is considered
>>incomplete, it may still be legal in a context where a full
>>type would normally be required, so long as the expected type
>>is the full type, or the full type is "nearby."  One of the
>>versions of the type-stub proposal laid out the rules for
>>this matching between incomplete and full type, and those
>>rules seem still relevant for all of the other proposals as well.
>
>
> Oh, I see.  So what I said above is wrong.  Hmm.
>
>
>>Here is a copy of those rules (from AI-00217-04/04):

The second and third bullets below are what I have
been referring to as the "extra" matching rules.

>>
>>    A dereference (implicit or explicit) of a value of an access type whose
>>    designated type D is incomplete is allowed only in the following
>>    contexts:
>>
>>    * in a place where the completion of D is available (see above);
>>
>>    * in a context where the expected type is E and
>>       o E covers the completion of D,
>>       o E is tagged and covers D,
>>       o E covers D'Class or its completion, or
>>       o E'Class covers D or its completion;
>>
>>    * as the target of an assignment_statement where the type of the value
>>      being assigned is V, and V or V'Class is the completion of D.
>
>
> How can V'Class be the completion of something?

The "full" implicitly declared type P.Q.R.V'Class can be the
completion of the (implicitly declared) incomplete type
P.Q.R'Limited.V'Class.

**************************************************************

From: Robert A. Duff
Sent: Wednesday, February 12, 2003  2:40 PM

Tucker says:

> Some of this is in reaction to Randy and Pascal's
> comments on my earlier implementation musings.
> They don't like the idea of having two units in the library with
> the same name.  It breaks their model badly.  So
> I was proposing that they have different names,
> at least as far as the program library mechanism is
> concerned.

It seems just like spec vs. body.  Now we have limited view vs. spec.

> > I don't understand the need for this consistency check.
> > It seems to me that P.Q.R depends upon P.Q.R'Limited,
> > and you do the normal check at link time, to make sure that
> > two different versions of P.Q.R'Limited are not included in the same
> > partition.
> >
> > In normal operation, you just do a "build" command, and the only way the
> > link-time check can fail is if you edit your source code while the build
> > is running.
>
> Again, I was trying to accommodate a model where timestamps
> are associated with compiled representations, rather than
> only with source text.  I agree that a purely source-based
> model can be simpler.

I don't see what difference the library model makes in this regard.  If
you have a "build" command, it has to look at timestamps of source
files.  It doesn't matter whether those time stamps are recorded with
the compilation artifacts, or whether those get the timestamp of when
they were generated.

Either way, so long as the user is not doing something in parallel (like
editing the source, or deleting the compilation artifacts by hand), then
end result will be a consistent build result.

In other words, the job of a "build" tool is to bring everything up to
date, whatever that means for the library model in use.  If the build
tool complains about things being out of date, then it's broken (except
when the user is meddling in parallel as described above).

I claim this is possible to achieve in *any* of the models, without
extra complexity (such as when you talked about compile the same thing
twice).

Of course, implementations are not required to support "build" tools,
and if they don't then the user has to worry about more stuff.
But we were talking about what a build tool would do...

Perhaps Randy or Pascal can explain about non-source-based compilers
better....

**************************************************************

From: Tucker Taft
Sent: Wednesday, February 12, 2003  3:33 PM

Robert A Duff wrote:
> It seems just like spec vs. body.  Now we have limited view vs. spec.

Yes, that is similar.  The "full" name of the unit in the program library
could really be <comp_unit => "P.Q.R", kind => spec/body/limited-view>.

> > > I don't understand the need for this consistency check.
> > > It seems to me that P.Q.R depends upon P.Q.R'Limited,
> > > and you do the normal check at link time, to make sure that
> > > two different versions of P.Q.R'Limited are not included in the same
> > > partition.
> > >
> > > In normal operation, you just do a "build" command, and the only way the
> > > link-time check can fail is if you edit your source code while the build
> > > is running.
> >
> > Again, I was trying to accommodate a model where timestamps
> > are associated with compiled representations, rather than
> > only with source text.  I agree that a purely source-based
> > model can be simpler.
>
> I don't see what difference the library model makes in this regard.  If
> you have a "build" command, it has to look at timestamps of source
> files.  It doesn't matter whether those time stamps are recorded with
> the compilation artifacts, or whether those get the timestamp of when
> they were generated.

I'll let Randy explain whatever problem he was foreseeing.
I may not have it right in any case.

If I remember correctly, Randy implied that
the source file timestamps were useless on Windows, because
they weren't reliable or didn't have enough resolution,
so his compiler generated sequential timestamps that were
associated with the order in which files were submitted to
the compiler.  His build tool determines an order that should
work, and perhaps queries the compiler to determine which
units are considered out of date, and determines a sequence
for just those units.

> Either way, so long as the user is not doing something in parallel (like
> editing the source, or deleting the compilation artifacts by hand), then
> end result will be a consistent build result.
>
> In other words, the job of a "build" tool is to bring everything up to
> date, whatever that means for the library model in use.  If the build
> tool complains about things being out of date, then it's broken (except
> when the user is meddling in parallel as described above).

I think this presumes a build tool that understands the out-of-dateness
rules.  I'm not sure that is how RR's works.

> I claim this is possible to achieve in *any* of the models, without
> extra complexity (such as when you talked about compile the same thing
> twice).
>
> Of course, implementations are not required to support "build" tools,
> and if they don't then the user has to worry about more stuff.
> But we were talking about what a build tool would do...

I wasn't talking about what a build tool would do, but maybe
you were.  I was talking about what the compiler would do when
presented with a request to "fully" compile a unit, when it had
earlier been parsed to produce a limited view.  If the full view
and the limited view share a single sequentially-assigned time stamp,
then you can get yourself into a tangle.  If the full view and
the limited view have distinct time stamps, then things can work out.
But you have no way of knowing whether the limited view and the
full view were created from the same version of the source file
if you don't believe source file timestamps.  The only way would
be to compare the results of the two different ways of processing
the same file.  I was suggesting that you compare the representation
of the limited view with the representation of the full view, and
if they are consistent, then there is no need to change the time
stamp assigned to the limited view when it was first parsed.
However, if they are inconsistent, then you need to recreate the
limited view, and need to assign it the next sequential time stamp
to make everything that depended on it out of date.

> Perhaps Randy or Pascal can explain about non-source-based compilers
> better....

I presume so as well.  And I suspect that Rational and RR have
different timestamp mechanisms.  So I suppose what I really did
was propose an implementation model for a library based on
sequentially assigned time stamps, where the modification date
of the source text is not saved, nor generally believed.

Our compiler, on the other hand, bases everything on the modification
date of the source text, and, as you point out, if you do
that, the only way things can get out of sync is if the file
is edited in the middle of a "build."

**************************************************************

From: Robert A. Duff
Sent: Wednesday, February 12, 2003  4:30 PM

> If I remember correctly, Randy implied that
> the source file timestamps were useless on Windows, because
> they weren't reliable or didn't have enough resolution,

I know of two problems with windows file timestamps, which have caused
trouble for AdaMagic in the past:

It does weird things with daylight-savings time, and with time zones, so
that if you compiled everything in the summer, and then waited until
winter (or vice versa) it would wrongly think everything it out of date
and need recompiling, and similarly that shipping a program library to a
different time zone would cause everything to seem out of date.

Different windows file systems store the timestamps with different
granularity (half second versus 1 second, or something like that), so
that if you copy a directory from one to the other it would perturb some
of the timestamps by, say, half a second.

We found workarounds to both of these problems, but I sympathize with
Randy: it was a pain.

> so his compiler generated sequential timestamps that were
> associated with the order in which files were submitted to
> the compiler.

**************************************************************

From: Robert Dewar
Sent: Wednesday, February 12, 2003  10:27 PM

> It does weird things with daylight-savings time, and with time zones, so
> that if you compiled everything in the summer, and then waited until
> winter (or vice versa) it would wrongly think everything it out of date
> and need recompiling, and similarly that shipping a program library to a
> different time zone would cause everything to seem out of date.

This sounds like something that results from an improper attempt to use GMT
time stamps on windows. That's always a mistake.

**************************************************************

From: Randy Brukardt
Sent: Thursday, February 13, 2003  2:09 PM

> > It does weird things with daylight-savings time, and with time zones, so
> > that if you compiled everything in the summer, and then waited until
> > winter (or vice versa) it would wrongly think everything it out of date
> > and need recompiling, and similarly that shipping a program library to a
> > different time zone would cause everything to seem out of date.
>
> This sounds like something that results from an improper attempt to use
> GMT time stamps on windows. That's always a mistake.

I'm not sure. But I do know that when daylight saving time changes, all of
the file times change an hour. It's not just GMT time stamps, if you look at
files in Explorer, you can see this effect. (This is certainly true on
Windows 2000, I don't recall seeing it on NT or 95.) I hadn't realized that
it happened for timezones as well until Bob mentioned it, although that
makes perfect sense.

Most of the problems we had, however, was that the granularity of source
changes is fairly large, and its possible to have multiple time stamps that
are identical. In the worst cases (such as subunits updated by a tool), this
potentially caused out-of-date things to fail to get recompiled.

Anyway, the build tool uses the source timestamps anyway, but the compiler
uses a timestamp for each generated symbol file created at the creation (or
rewrite) of the file. We couldn't use the timestamps directly because of the
various problems. We then compare those timestamps. (Those wouldn't really
have to be timestamps at all; anything strictly accending without
duplication would work).

**************************************************************

From: Tucker Taft
Sent: Tuesday, February 11, 2003  2:17 PM

I thought I would update the full ARG on the Padua
discussions relating to AI-00217, the mutual-dependence
AI.  This is rightly the job of the minute taker or Pascal, but
I thought some of you might be waiting with bated breath...
So this is a *very* unofficial summary of what we decided.

Well, we *didn't* decide on a single solution.  However,
we did settle on an approach to make the decision, and
we came to a number of decisions that limit the number
of proposals that remain alive, and that answer some
of the tricky semantic questions that all of the
proposals are facing.

We agreed to proceed with a full write up of the
following three proposals:

1) type stubs, limited to a single stub per full type,
   where the package containing the full type must
   have a "with" for the package containing the stub,
   and where a "separate with ..." context clause
   is required if a stub appears inside a package.
   Randy will write this one up.

2) incomplete type to be completed in a child/nested package.
   (aka "type C.T;").  Tuck will write this one up.

3) "limited with" of a package.  Pascal will write this
   one up, presumably starting with Bob's version.

As usual, the writeups should include wording, rationale, pros and cons,
various examples, known gotchas, etc.

In addition, Randy, Tuck, and Pascal will produce at least one
extended example of use using "their" proposal, and then exchange
examples and do the same for the examples developed by the other two.

Finally, all three will develop implementation models for all
three proposals within their respective compiler technologies.
[NOTE: Tuck has an initial draft for "limited with" in the
"AdaMagic" technology, which will be sent out shortly.]

Tuck will include a presentation of these three alternatives
at the upcoming Ada UK meeting on April 8/9 in Swindon,
and hope to get some useful user input.  The extended examples
should be available by this time for inclusion in the presentation.
The implementation models need not be available by this date.

During and after this sequence, the ARG will undergo
its typically efficient process of e-mail analysis ;-),
and no later than the end of the first day of the next
ARG meeting, make a decision.

----------------

As far as specific technical decisions we made:

1) When the full type declaration is visible, including
via a renaming of its enclosing package, the incomplete
type declaration is hidden from all visibility (type C.T
and limited-with proposals), or the type stub becomes equivalent to
the full type (type stub proposal).

2) When a package is visible, including via a renaming, any
"limited view" of the package is hidden from all visibility
(type C.T and limited-with proposals).  By "limited view" we
mean the view of a package or nested package which is made
visible by the limited-with, or the child/nested package used as
a prefix in the "type C.T;" proposal.

3) A "limited with" would not be allowed if a
full view of the package named is visible
via some other means (limited with proposal).  An inherited
limited with, or an implicit limited with of a package mentioned
(but not the specifically "with"ed package itself) in a limited
with clause (e.g. "P.Q" in "limited with P.Q.R"), would be
ignored if the full view of the package is visible via some other means.

4) No use clauses or renamings are permitted for a limited
view of a package (type C.T and limited-with proposals).
[Rationale: Use clauses seem to introduce possible Beaujolais-like
effects, and renamings of limited views just make the
"hidden from all visibility" rules even weirder.]

5) "Limited with" would be available only on "normal" package
declarations; a limited-with of a renaming or package instantiation
would require too much semantic analysis for the average
"dumb" parser.  No such limitation seemed necessary for
the other two proposals, though it wasn't examined in
detail.

Some potentially interesting equivalences were noticed:

* A "limited with" of a child has an effect that is
  very similar to the "type C.T;" proposal.

* A type stub for a type declared in a child has an
  effect very similar to the "type C.T;" proposal.

Because of these equivalences, it is always possible
to have a single place where the access-to-incomplete type
can be declared, and that place could always be the
parent of the unit containing the full type, if that
were desired.

**************************************************************

From: Robert A. Duff
Sent: Tueday, February 11, 2003  4:12 PM

I'll repeat my earlier admonishment: we ought to try to get this done
ASAP.  The above schedule seems reasonable -- let's stick to it.

I do not agree with Robert Dewar's comments last week, saying we should
avoid rushing into a decision that may be technically wrong.  First,
we've been at this for about 8 years -- it's hardly rushing.  Second,
there are several workable solutions on the table.  In particular, the
ARG already approved one workable solution.  We can patch it to address
the (very minor, IMHO) concern that WG9 had, if we like.  Or we can
leave it as is.  Or we can choose one of the other alternatives.  Either
way, there is little danger of passing something that's broken.

It seems silly to dither about this for a long time, just because we
can't decide which of the several good solutions is best.  And no
solution will be "perfect" in every regard.

Of course, if we choose "limited with", we have to work out all the
detailed rules.  But that seems entirely feasible to do in a matter of
months, not years.  If not, we should pick one of the other good
solutions.

**************************************************************

From: Robert Dewar
Sent: Tuesday, February 11, 2003  4:25 PM

> I do not agree with Robert Dewar's comments last week, saying we should
> avoid rushing into a decision that may be technically wrong.

That's rather amazing, look at what you are saying, you are saying you
think it's fine to rush into a decision that may be technically wrong.

Do you really mean that, or, as your supporting argument implies, are you
arguing that we can make a decision that is technically right?

To me, the LIMITED WITH looks very promising. I had thought some consensus
was gathering on that solution, and I find it disappointing that Padua
thought otherwise.

**************************************************************

From: Robert A. Duff
Sent: Tuesday, February 11, 2003  5:00 PM

> Do you really mean that, or, as your supporting argument implies, are you
> arguing that we can make a decision that is technically right?

I'm disagreeing with the assertion that we're "rushing" and I'm
disagreeing with the assertion that there's a serious danger of
making a bad choice.  I claim that we *already* have a technically-good
solution (type stubs).

> To me, the LIMITED WITH looks very promising. I had thought some consensus
> was gathering on that solution, and I find it disappointing that Padua
> thought otherwise.

I also have a mild preference for the "limited with" idea.  But that
proposal is still half baked.  If we can work out the rules in a
reasonable time, and we don't discover any major surprises, then I'll be
happy with that.  But I would also be happy with type stubs, or with
Tucker's "type C.T" idea (which is pretty well baked), and if it takes
two more years to "fully bake" the "limited with" idea, then I prefer
the (nearly) already-worked-out solutions.

If we were searching for perfection, we would still be working on the
Ada 9X project.  ;-)

**************************************************************

From: Robert Dewar
Sent: Tuesday, February 11, 2003  10:41 PM

For me, the ability to reasonably automatically convert existing foreign
language specs is the most important issue here. Perhaps I do not fully
understand, but it seems to me that the limited with proposal is significantly
better from this point of view.

**************************************************************

From: Robert A. Duff
Sent: Tuesday, February 11, 2003  4:50 PM

Tucker wrote:

> 1) When the full type declaration is visible, including
> via a renaming of its enclosing package, the incomplete
> type declaration is hidden from all visibility (type C.T
> and limited-with proposals), or the type stub becomes equivalent to
> the full type (type stub proposal).

All of these proposals seem to have an issue with indirect
with_clauses.  Consider:

    package P is
        type C.T is tagged;
        procedure Proc(X: C.T);
    end P;

    package P.C is
        type T is tagged ...;
    end P.C;

    with P.C;
    package Q is
        Y, Z: P.C.T;
    end Q;

    with P, Q;
    procedure Main is
    begin
        P.Proc(X => Q.Y); -- Legal?
        Y := Z; -- Legal, surely.
    end Main;

Is the above legal?  Should it be?  Why?

The full type decl of P.C.T is not visible in Main.  But we can still
get our hands on that type via Y.  So we've got a situation where the
incomplete type and the full type T are both "there" -- only one is
visible by name, but we've got to ask whether these are the same type.

It seems weird (to me) to make this illegal.  If it's illegal, then how
do we explain why "X := Y" is legal?

Tucker hates "Ripple Effects".  I don't entirely agree; it seems to me
that with_clauses should have been transitive in the first place.

Similar question: Can Main declare a variable of type P.C.T?

If we're worried about renamings, do we also need to worry about
subtypes (which act like renamings of types)?  E.g.:

    package P is
        type C.T is tagged;
        procedure Proc(X: C.T);
    end P;

    package P.C is
        type T is tagged ...;
    end P.C;

    with P.C;
    package Q is
        subtype S is P.C.T;
    end Q;

    with P, Q;
    procedure Main is
        W: S; -- Legal?
    begin
        Proc(X => W); -- Legal?
    end Main;

Here, the full type decl of P.C.T is not visible, but a subtype that
essentially equivalent to that full type is visible.

> 2) When a package is visible, including via a renaming, any
> "limited view" of the package is hidden from all visibility
> (type C.T and limited-with proposals).  By "limited view" we
> mean the view of a package or nested package which is made
> visible by the limited-with, or the child/nested package used as
> a prefix in the "type C.T;" proposal.

    limited with P.C;
    package P is
        procedure Proc(X: C.Nest.T);
    end P;

    package P.C is
        package Nest is
            type T is tagged ...;
        end Nest;
    end P.C;

    with P.C;
    package Q is
        package Nest_XXX renames P.C.Nest;
    end Q;

    limited with P.C;
    with P, Q;
    procedure Main is
        ...
    end Main;

Now in Main, does the above rule mean that we have a limited view of
P.C, but a normal view of P.C.Nest and P.C.Nest.T?

> 3) A "limited with" would not be allowed if a
> full view of the package named is visible
> via some other means (limited with proposal).  An inherited
> limited with, or an implicit limited with of a package mentioned
> (but not the specifically "with"ed package itself) in a limited
> with clause (e.g. "P.Q" in "limited with P.Q.R"), would be
> ignored if the full view of the package is visible via some other means.

Wouldn't it be simpler to use the "ignore" semantics in both cases?
In fact, the previous rule about the full view hiding the limited view
would cover these cases, and imply the "ignore" semantics.

The above rule seems to imply that:

    with X;
    limited with X;
    package ...

is illegal, but:

    limited with X;
    with X;
    package ...

is legal, which seems kind of silly.

> 4) No use clauses or renamings are permitted for a limited
> view of a package (type C.T and limited-with proposals).
> [Rationale: Use clauses seem to introduce possible Beaujolais-like
> effects, and renamings of limited views just make the
> "hidden from all visibility" rules even weirder.]
>
> 5) "Limited with" would be available only on "normal" package
> declarations; a limited-with of a renaming or package instantiation
> would require too much semantic analysis for the average
> "dumb" parser.

I don't think the issue is "dumb parser", per se.  The issue is: if you
say "limited with X;", the compiler has to be able to process X without
looking at any source code outside of X, because that might lead back to
*this* unit via a cycle.  In any case, I agree with this rule, although
I would state the rationale differently.

>...  No such limitation seemed necessary for
> the other two proposals, though it wasn't examined in
> detail.

It seems clear that the other two proposals do not need such a
limitation, because they do not require looking at the other package at
all -- the type stub, or the "type C.T" tells us the name of the type,
and whether it's tagged, which is all we need to know.  This seems like
a minor argument in favor of those other proposals -- they would allow
the full type to come from a renaming or an instance.

**************************************************************

From: Tucker Taft
Sent: Tuesday, February 11, 2003  10:28 PM

>>1) When the full type declaration is visible, including
>>via a renaming of its enclosing package, the incomplete
>>type declaration is hidden from all visibility (type C.T
>>and limited-with proposals), or the type stub becomes equivalent to
>>the full type (type stub proposal).
>
>
> All of these proposals seem to have an issue with indirect
> with_clauses.

I hope not.  We made a big effort to avoid
the so-called "ripple" effect.  This is why we
talk about being visible rather than being in scope.

 > ... Consider:
>
>     package P is
>         type C.T is tagged;
>         procedure Proc(X: C.T);
>     end P;
>
>     package P.C is
>         type T is tagged ...;
>     end P.C;
>
>     with P.C;
>     package Q is
>         Y, Z: P.C.T;
>     end Q;
>
>     with P, Q;
>     procedure Main is
>     begin
>         P.Proc(X => Q.Y); -- Legal?

Yes.  This is thanks to the "extra" matching
rules.

>         Y := Z; -- Legal, surely.

Yes.

>     end Main;
>
> Is the above legal?  Should it be?  Why?

The "extra" matching rules deal with the cases
where the expected type is incomplete and the actual
is complete, and vice-versa.


>
> The full type decl of P.C.T is not visible in Main.  But we can still
> get our hands on that type via Y.  So we've got a situation where the
> incomplete type and the full type T are both "there" -- only one is
> visible by name, but we've got to ask whether these are the same type.
>
> It seems weird (to me) to make this illegal.  If it's illegal, then how
> do we explain why "X := Y" is legal?

It is intended to be legal, thanks to the extra matching rules.


>
> Tucker hates "Ripple Effects".  I don't entirely agree; it seems to me
> that with_clauses should have been transitive in the first place.

There is no ripple effect in any of these examples.  And
they should all be legal.

>
> Similar question: Can Main declare a variable of type P.C.T?

No.  The name "P.C.T" refers to the incomplete type in Main.

>
> If we're worried about renamings, do we also need to worry about
> subtypes (which act like renamings of types)?

No.  Subtypes are treated differently by the proposed
rules.  There seems no implementation problem having a subtype of
the complete type visible while the incomplete type
declaration is also visible.  Whether there is a
danger of user confusion, I don't think so, but of
course that is pretty hard to prove for an arbitrary user.

> E.g.:
>
>     package P is
>         type C.T is tagged;
>         procedure Proc(X: C.T);
>     end P;
>
>     package P.C is
>         type T is tagged ...;
>     end P.C;
>
>     with P.C;
>     package Q is
>         subtype S is P.C.T;
>     end Q;
>
>     with P, Q;
>     procedure Main is
>         W: S; -- Legal?

Yes, "S" is a subtype of a full type, and that
never changes.

>     begin
>         Proc(X => W); -- Legal?

Yes, the extra matching rules allow this.

>     end Main;
>
> Here, the full type decl of P.C.T is not visible, but a subtype that
> essentially equivalent to that full type is visible.

True.

>>2) When a package is visible, including via a renaming, any
>>"limited view" of the package is hidden from all visibility
>>(type C.T and limited-with proposals).  By "limited view" we
>>mean the view of a package or nested package which is made
>>visible by the limited-with, or the child/nested package used as
>>a prefix in the "type C.T;" proposal.
>
>
>     limited with P.C;
>     package P is
>         procedure Proc(X: C.Nest.T);
>     end P;
>
>     package P.C is
>         package Nest is
>             type T is tagged ...;
>         end Nest;
>     end P.C;
>
>     with P.C;
>     package Q is
>         package Nest_XXX renames P.C.Nest;
>     end Q;
>
>     limited with P.C;
>     with P, Q;
>     procedure Main is
>         ...
>     end Main;
>
> Now in Main, does the above rule mean that we have a limited view of
> P.C, but a normal view of P.C.Nest and P.C.Nest.T?

No.  It means you have a limited view of P.C, but
P.C.Nest and P.C.Nest.T are hidden from all visibility.
If you want to refer to P.C.Nest, you have to do so
using Q.Nest_XXX.

Although it may be weird for the subpackage P.C.Nest to
"disappear," this only happens in cases where the limited
with was unnecessary, and a normal "with" would not
have introduced circularities.  We want to be sure this
works, but we don't need to make it completely "pretty."

>3) A "limited with" would not be allowed if a
>>full view of the package named is visible
>>via some other means (limited with proposal).  An inherited
>>limited with, or an implicit limited with of a package mentioned
>>(but not the specifically "with"ed package itself) in a limited
>>with clause (e.g. "P.Q" in "limited with P.Q.R"), would be
>>ignored if the full view of the package is visible via some other means.
>
>
> Wouldn't it be simpler to use the "ignore" semantics in both cases?
> In fact, the previous rule about the full view hiding the limited view
> would cover these cases, and imply the "ignore" semantics.

Yes, disallowing the limited with is simply a "friendly"
arbitrary restriction.  It would be ignored in any case.
But it seems friendly to notify the programmer that a
"limited with" is useless.  This is particularly true when
the reason it is useless is because there is a visible rename.
We would like the compiler to point that out.


> The above rule seems to imply that:
>
>     with X;
>     limited with X;
>     package ...
>
> is illegal, but:
>
>     limited with X;
>     with X;
>     package ...
>
> is legal, which seems kind of silly.

No, that was not the intent.  The legality check
on the limited with would be after fully processing
the context clause (and the parent name and the
inherited with's).  It would not be dependent
on which came first.

>>...  No such limitation seemed necessary for
>>the other two proposals, though it wasn't examined in
>>detail.
>
>
> It seems clear that the other two proposals do not need such a
> limitation, because they do not require looking at the other package at
> all -- the type stub, or the "type C.T" tells us the name of the type,
> and whether it's tagged, which is all we need to know.  This seems like
> a minor argument in favor of those other proposals -- they would allow
> the full type to come from a renaming or an instance.

True.

**************************************************************

From: Robert A. Duff
Sent: Wednesday, February 12, 2003  2:10 PM

Tuck says:

> I hope not.  We made a big effort to avoid
> the so-called "ripple" effect.  This is why we
> talk about being visible rather than being in scope.

Could you please explain to me (again) precisely what a ripple effect
is, and why you consider them to be evil?  Sorry to be dense...

> >>3) A "limited with" would not be allowed if a
> >>full view of the package named is visible
> >>via some other means (limited with proposal).  An inherited
> >>limited with, or an implicit limited with of a package mentioned
> >>(but not the specifically "with"ed package itself) in a limited
> >>with clause (e.g. "P.Q" in "limited with P.Q.R"), would be
> >>ignored if the full view of the package is visible via some other means.
> >
> > Wouldn't it be simpler to use the "ignore" semantics in both cases?
> > In fact, the previous rule about the full view hiding the limited view
> > would cover these cases, and imply the "ignore" semantics.
>
> Yes, disallowing the limited with is simply a "friendly"
> arbitrary restriction.  It would be ignored in any case.
> But it seems friendly to notify the programmer that a
> "limited with" is useless.  This is particularly true when
> the reason it is useless is because there is a visible rename.
> We would like the compiler to point that out.

This is a fairly minor point, but I still suggest going with the simpler
rule.  The RM is not in the business of good error messages; leave that
to implementers.

It's like the use-clause thing -- if you have two homographs called X
they cancel each other out.  The straightforward implementation of that
would cause the compiler to say "no directly visible X" or something,
which is an extremely unhelpful message.  Good compilers go to some
extra trouble to point out that there are actually *two* X's, each of
which is "almost" directly visible.

Maybe when you see the actual RM wording, you'll agree...  ;-)

**************************************************************

From: Tucker Taft
Sent: Wednesday, February 12, 2003  3:18 PM
Robert A Duff wrote:
>
> Tuck says:
>
> > I hope not.  We made a big effort to avoid
> > the so-called "ripple" effect.  This is why we
> > talk about being visible rather than being in scope.
>
> Could you please explain to me (again) precisely what a ripple effect
> is, and why you consider them to be evil?  Sorry to be dense...

A ripple effect is when adding or removing a "with" clause in some
unit on which the current unit depends semantically (potentially
quite indirectly) has a significant effect on the legality of the
current unit.  For example, at one point in Ada 83, to use 'Address, the
current unit had to have a semantic dependence on System.  This
involves a ripple effect, since adding or removing a "with" of
System in some "distant" unit could affect legality of the unit
containing the 'Address.

> > >>3) A "limited with" would not be allowed if a
> > >>full view of the package named is visible
> > >>via some other means (limited with proposal).  An inherited
> > >>limited with, or an implicit limited with of a package mentioned
> > >>(but not the specifically "with"ed package itself) in a limited
> > >>with clause (e.g. "P.Q" in "limited with P.Q.R"), would be
> > >>ignored if the full view of the package is visible via some other means.
> > >
> > >
> > > Wouldn't it be simpler to use the "ignore" semantics in both cases?
> > > In fact, the previous rule about the full view hiding the limited view
> > > would cover these cases, and imply the "ignore" semantics.
> >
> > Yes, disallowing the limited with is simply a "friendly"
> > arbitrary restriction.  It would be ignored in any case.
> > But it seems friendly to notify the programmer that a
> > "limited with" is useless.  This is particularly true when
> > the reason it is useless is because there is a visible rename.
> > We would like the compiler to point that out.
>
> This is a fairly minor point, but I still suggest going with the simpler
> rule.  The RM is not in the business of good error messages; leave that
> to implementers.

It seems very simple to disallow these useless "limited withs," so I would
prefer to make them illegal, and my sense was that the rest of the ARG
in Padua felt similarly.  But they can presumably answer for themselves.

> ...
> Maybe when you see the actual RM wording, you'll agree...  ;-)

With whom?

**************************************************************

From: Tucker Taft
Sent: Tuesday, February 11, 2003  2:21 PM

Here is some initial analysis of implementation issues associated
with "limited with."  I tried to make most of the analysis
technology independent, but of course that is nearly impossible,
so it may apply less or more to technologies other than AdaMagic.
-Tuck

-----------------
Thoughts on the implementation of "limited with" in AdaMagic

          $Revision: 1.14 $ $Date: 2004/06/25 01:30:52 $

The "limited with" clause makes a "limited view"
of a package visible in the compilation unit
that has the limited-with clause, as well as all of its
descendants.  However, the limited view is
hidden from all visibility (as are all the incomplete
types and nested package limited views within it) under
certain circumstances.  After much discussion at
the Padua ARG meeting, the best rule seemed
to be that if the full view of the package is
visible, including via one or more renamings,
then the limited view is hidden from all visibility.
[This was considerd necessary so as to avoid having two different
views of the same package usable at the same point
in the source text, which was felt to create both
implementation complexity and possible user confusion.]

SEPARATE "LIMITED" COMPILATION UNIT

Our model for implementing a limited view of a library package
will be that it has its "own" compilation-unit name,
constructed from its "real" name but with suffix "'Limited".
Hence P.Q.R'Limited is the limited view of P.Q.R.
[Note that "P.Q.R.Limited" could also be used as the
name, since "limited" is a reserved word, and would
never be the real name of a package.  However, it is
very important that in any case P.Q.R.Limited or
P.Q.R'Limited not depend semantically on P.Q.R itself.]

In the representation of the library, it is important
that P.Q.R'Limited can exist before P.Q.R has been
fully compiled, and can retain some continuing existence
while P.Q.R is being fully compiled, and thereafter.
It is also important to realize that the packages
nested within P.Q.R'Limited are themselves limited
views.  Hence, P.Q.R'Limited.NP always denotes a limited view
of a package, if P.Q.R.NP denotes a nested package spec.  Similarly,
P.Q.R'Limited.T denotes an incomplete type declaration for T,
if P.Q.R.T denotes a full type declaration.

SEPARATE TIMESTAMP FOR LIMITED VIEW

P.Q.R'Limited has its own timestamp, for the purposes
of out-of-dateness checking.  It is important that this
*not* change when a "full" compile of P.Q.R is started,
because that would make the units that P.Q.R depends
on that themselves depend on P.Q.R'Limited, automatically
appear out of date.

Once the full compile of P.Q.R is complete, a check for
consistency between P.Q.R'Limited and P.Q.R should be
performed.  If the views are no longer consistent, due to
some change in P.Q.R, a new version of P.Q.R'Limited should
be constructed, and its timestamp changed.  This will effectively
put all the dependents of P.Q.R'Limited out of date,
and may in fact put P.Q.R itself out of date.  If so,
then P.Q.R will have to be compiled again.  Ideally, this
will be performed immediately, since the source code for
P.Q.R is readily available at that time.  However, depending on the
compilation mechanism, it may be first necessary to compile
other dependents of P.Q.R'Limited, so they are back in
an up-to-date state before re-attempting the full
compilation of P.Q.R.  This might be left until link time,
or to a separate "make" tool.

Of course if P.Q.R'Limited is initially created by simply parsing
the source text for P.Q.R, the timestamp of P.Q.R'Limited would be
set at parse time.

For those compilation systems that retain the date of modification
of the source file as the time stamp of the corresponding units
in the program library, some of this becomes simpler.
Nevertheless, it seems a helpful, and in some cases
necessary, "optimization" to *not* change the timestamp of
the limited view every time the source text timestamp changes.
Because the check for consistency between a full view and
a limited view should be relatively easy to perform, there
seems no need to change the timestamp on the limited view
so long as it remains consistent with the full view, even
if the full view corresponds to source text with a different
timestamp.  This means that units that depend only on the
P.Q.R'Limited are isolated from "minor" changes to P.Q.R,
and are only affected when a type or nested package is
added or removed from P.Q.R.   In other words, you get
a "poor man's" version of incremental compilation as a side
effect of implementing limited views of packages.

HIDING THE LIMITED VIEW

As mentioned above, a limited view of a package can be hidden from
all visibility.  The anticipated rule is that the limited
view is hidden from all visibility if the full view is visible
either "normally" or via a rename.  This implies that if
P.Q.R is visible, including via a rename, then P.Q.R'Limited
is not visible.  Furthermore, even when P.Q.R'Limited
is visible, it is possible that P.Q.R'Limited.NP might
not be visible, because P.Q.R.NP might be visible via
a renaming.

A relatively simple implementation of this requirement
would seem to be to build up a set representing those (external) packages
that have a full view visible, including via a rename, within the
current compilation unit.  (By "external" we mean it is from some
other compilation unit.)  This set will not grow once the
context clause has been fully processed, since it is not
possible to locally declare a renaming of a package that is not
already visible.

This set of visible packages would probably be represented
using a hash table of some sort.  It would start out empty
when starting to compile a new compilation unit, and would
grow each time a (real, not limited) "with" for a package or a package
renaming is processed.  Each of these "with"ed packages
would be scanned for enclosed renamings, recursively.
[Alternatively, the representation of every package could
include a set of all enclosed renamings (directly or indirectly).]
The "with"ed package, plus all of the packages for which it
contains a renaming, would be unioned into this set.

Later, whenever a limited view of a package is encountered
within a name, a check is made to see whether a corresponding
full view exists in this set.  If so, the limited view may
not be named, and so the name is illegal.

Another anticipated rule is that a limited with clause is
illegal if the package is already visible, including
via a rename.  This would seem to be friendlier than allowing
the programmer to put a "limited with" on a compilation unit and
then not be able to use it at all.  However, there will still
be "limited with"s inherited from ancestor units, and these
will be essentially ignored if the corresponding package
is fully visible some other way.  Also, in a limited with
clause like "limited with P.Q.R;" presumably implicit
limited withs for P and P.Q are provided, but such an implicit
limtied with again must be ignored if the package mentioned
is already fully visible via some other means.

To enforce the above rule regarding disallowing and/or ignoring
limited withs, it may be necessary to do a second pass over
the context clause, after the full set of visible packages has
been built up.  In many cases, a second pass is already necessary
in Ada 95 to correctly deal with "with"s of private units, since it is
not known whether those with's are legal until the name of the
compilation unit being compiled is known, so it may be possible
to combine these error-checking passes.

WHEN IS THE INCOMPLETE TYPE COMPLETE?

Our current plan is to disallow "use" clauses and renamings
for limited views.  Given that, the only way to name one
of the incomplete types inside a limited view is by naming
the enclosing package, so there is no need to have a separate mechanism
for determining whether the incomplete type declaration is visible.
It is visible exactly when the enclosing limited view can be named.

However, there are access types that might have been declared
using the incomplete type, and it is important to know when a
dereference of the access type is considered "complete."
The basic rule would be that when the full type is visible,
including via a rename of the enclosing package, (or equivalently,
when the incomplete type declaration is hidden from all visibility),
the access type may be dereferenced to produce the full type.
This implies that any time such an access type is dereferenced,
a check should be made whether the designated type's enclosing limited
view(s) are all still visible.  If they are all still visible,
then the incomplete type is still visible, and the dereference
produces an incomplete object.  If any one of them is hidden from
all visibility, then the full type is visible, and the dereference
produces an object of a "full" type.  Some optimization of this
check is probably possible.

Even when a dereference of the access type is considered
incomplete, it may still be legal in a context where a full
type would normally be required, so long as the expected type
is the full type, or the full type is "nearby."  One of the
versions of the type-stub proposal laid out the rules for
this matching between incomplete and full type, and those
rules seem still relevant for all of the other proposals as well.

Here is a copy of those rules (from AI-00217-04/04):

    A dereference (implicit or explicit) of a value of an access type whose
    designated type D is incomplete is allowed only in the following
    contexts:

    * in a place where the completion of D is available (see above);

    * in a context where the expected type is E and
       o E covers the completion of D,
       o E is tagged and covers D,
       o E covers D'Class or its completion, or
       o E'Class covers D or its completion;

    * as the target of an assignment_statement where the type of the value
      being assigned is V, and V or V'Class is the completion of D.

    In these contexts, the incomplete type is defined to be the same
    type as its completion, and its first subtype statically matches the
    first subtype of its completion.

The first bullet would be replaced by "where the full type declaration
for D is visible" for the limited-with proposal (and for the
"type C.T;" proposal).

**************************************************************

From: Pascal Leroy
Sent: Thursday, March  6, 2003  4:56 AM

(I am discussing the issue below in the context of limited with, but I believe
that the other incarnations of 217 all have similar problems if they use
visibility to determine when the completion can be used.)

A while back, in a message about implementation of limited withs in AdaMagic,
Tuck wrote:

> As mentioned above, a limited view of a package can be hidden from
> all visibility.  The anticipated rule is that the limited
> view is hidden from all visibility if the full view is visible
> either "normally" or via a rename.  This implies that if
> P.Q.R is visible, including via a rename, then P.Q.R'Limited
> is not visible.  Furthermore, even when P.Q.R'Limited
> is visible, it is possible that P.Q.R'Limited.NP might
> not be visible, because P.Q.R.NP might be visible via
> a renaming.

From a user's perspective, the notion that you have a limited view of P.Q.R but
a full view of its nested package P.Q.R.NP sounds very confusing to me.
Especially considering that the reason why you have a full view of NP is that
someone somewhere declares a renaming of NP.

From an implementer's perspective, my head aches when I try to see how we could
implement this.  In fact, the above description seems distinctly at odds with
what was discussed during the meeting.  The minutes say:

"Randy says that two views of the same package make the implementation much
harder. Our implementation only expects a single view with a few well-defined
places of visibility change. This is scattered all about. Pascal expresses
agreement with Randy's assessment."

And then the minutes present rules intended to avoid the "two views" issue
(although arguably the problem discussed by Tucker was not addressed, at least
as far as I remember).

The implementation difficulties for us are worse than I originally thought
because of incremental compilation.  Say that you compile a unit containing the
name P.Q.R.NP.Var, and that name is legal because of the existence of some
renaming of NP. Then that renaming is removed.  During incremental recompilation
the name P.Q.R.NP.Var must be obsolesced, even though it doesn't reference the
renaming that is being removed.  That's hard.  Not as hard as a full-fledged
ripple effect maybe, but still hard.

I understand that there is some similarity with name clashes introduced by use
clauses, but I'd rather not add more complexity to code that is already very
hairy.  Also note that from a user's perspective name clashes are probably quite
rare, but visibility changes due to the introduction/removal of renamings might
be much more frequent, and would certainly result in much puzzlement.

This leads me to two questions:

1 - Why did we decide to go for visibility anyway?  What would be wrong with
using the "availability" defined in 217-05?
2 - If we have to stick to visibility, couldn't we define the rules in such a
way that there is only one view visible?  I'd say that the situation described
by Tucker above should be illegal, and that if (1) you have a limited with of
P.Q.R and (2) some renaming gives you a full view of P.Q.R.NP then your code is
illegal.  You need to add a "with P.Q.R" to hide the limited view of P.Q.R from
all visibility.

Comments?

**************************************************************

From: Tucker Taft
Sent: Thursday, March  6, 2003  8:17 AM

Pascal Leroy wrote:
...
> From a user's perspective, the notion that you have a limited view of P.Q.R but
> a full view of its nested package P.Q.R.NP sounds very confusing to me.
> Especially considering that the reason why you have a full view of NP is that
> someone somewhere declares a renaming of NP.

It's not quite that bad.  Remember that you don't have
a full view of NP using the name "P.Q.R.NP".  You have
to use the name introduced via the renaming.  What
you *lose* is the ability to refer to the limited view
of NP via the name "P.Q.R.NP."

...
> The implementation difficulties for us are worse than I originally thought
> because of incremental compilation.  Say that you compile a unit containing the
> name P.Q.R.NP.Var, and that name is legal because of the existence of some
> renaming of NP.

This is not what I intended to propose.  Var would never be visible
via the name "P.Q.R.NP.Var" if you have a limited view of
P.Q.R, since objects are not included in the limited view.
The presence of a renaming only *removes* something from
the limited view, it never adds something.

> ... Then that renaming is removed.  During incremental recompilation
> the name P.Q.R.NP.Var must be obsolesced, even though it doesn't reference the
> renaming that is being removed.  That's hard.  Not as hard as a full-fledged
> ripple effect maybe, but still hard.

Can you consider this again with hopefully a clearer explanation
of what was intended?


...
> This leads me to two questions:
>
> 1 - Why did we decide to go for visibility anyway?  What would be wrong with
> using the "availability" defined in 217-05?

I think if you study the availability rules, you will find
they are essentially the same as the proposed visibility
rules, but just couched in different terms.  If you find
a significant difference, I would like to know what it is.

> 2 - If we have to stick to visibility, couldn't we define the rules in such a
> way that there is only one view visible?  I'd say that the situation described
> by Tucker above should be illegal, and that if (1) you have a limited with of
> P.Q.R and (2) some renaming gives you a full view of P.Q.R.NP then your code is
> illegal.  You need to add a "with P.Q.R" to hide the limited view of P.Q.R from
> all visibility.

I'm not sure what you mean here, since we started off with
different interpretations of the proposed rule.  Could you
restate this, and contrast it with what I have explained
above to be the intent, namely that a rename causes things
to be removed from a limited view, rather than being added.

**************************************************************

From: Pascal Leroy
Sent: Friday, March  7, 2003  10:27 AM

> This is not what I intended to propose.  Var would never be visible
> via the name "P.Q.R.NP.Var" if you have a limited view of
> P.Q.R, since objects are not included in the limited view.
> The presence of a renaming only *removes* something from
> the limited view, it never adds something.

Ok, I was confused, but I still have essentially the same problems.  For
clarity, let's look at some piece of code:

    package P.Q.R is
        package NP is
            type T is new Boolean;
        end NP;
    end P.Q.R;

    with P.Q.R;
    package A is
        package Ren renames P.Q.R.NP;
    end A;

    limited with P.Q.R;
    -- with A;
    procedure Main is
        type Acc is access P.Q.R.NP.T;
    begin
        null;
    end Main;

If I understand you correctly, this code is legal.  However, if the "with A" in
Main is uncommented, the declaration of type Acc becomes illegal, because
P.Q.R.NP is hidden from all visibility.

From a user's perspective, I still find this rule mysterious, but maybe it's not
going to happen frequently in practice.

From an implementer's perspective I still have to have two views around:
P.Q.R'Limited for all of P.Q.R except P.Q.R.NP, and P.Q.R'Full for P.Q.R.NP (but
only when it is accessed through the renaming).  It means that there are a
zillion places in my compiler where I need to ask the question: which view do I
see?  This is feasible, but I'm sure I won't get it right the first time (or the
second time).

Now looking at incremental compilation again, you have not solved my problem.
Say that we had the "with A" all along, but initially A didn't include the
renaming.  Then someone adds that bloody renaming.  The name P.Q.R.NP.T in Main
must be obsolesced, even though it doesn't reference A.  Let me say it once
more: that's hard.

> I think if you study the availability rules, you will find
> they are essentially the same as the proposed visibility
> rules, but just couched in different terms.  If you find
> a significant difference, I would like to know what it is.

It's hard to tell for sure because the availability rules in 217-05 are
formulated in terms of type stubs.  But if you try to adapt them for "limited
withs", it would seem that you'd have to say that the completion of T is
available (1) within the extended scope of the completion of T and (2) within
the scope of a (nonlimited) with clause for the package that declares T. I fail
to see how this would cause a "with A" to hide P.Q.R.NP from all visibility. In
fact I fail to see how the presence of "with A" would have any bearing on the
visibility of entities exported by P.Q.R.

It seems to me that if we used availability, the completion of T would not be
available in Main (regardless of whether there is a "with A" or not) and
therefore we would have only one view of P.Q.R, the limited one.

> I'm not sure what you mean here, since we started off with
> different interpretations of the proposed rule.  Could you
> restate this...

Forget it, the scheme that I had in mind would introduce ripple effects.

**************************************************************

From: Tucker Taft
Sent: Friday, March  7, 2003  12:12 PM

> If I understand you correctly, this code is legal.  However, if the "with A"
> in Main is uncommented, the declaration of type Acc becomes illegal, because
> P.Q.R.NP is hidden from all visibility.

Right.

> From a user's perspective, I still find this rule mysterious, but maybe it's
> not going to happen frequently in practice.

It shouldn't, because it only happens if a piece of code
has a limited-with on something on which it depends
semantically.

...
> It's hard to tell for sure because the availability rules in 217-05 are
> formulated in terms of type stubs.  But if you try to adapt them for "limited
> withs", it would seem that you'd have to say that the completion of T is
> available (1) within the extended scope of the completion of T

Unless we change the definition of scope, this would include all semantic
dependents of the package where the completion is declared (see 8.2(10)).
So this would create ripple effects and would be even worse,
as far as incremental recompilation, I presume.

I really think you need to address 8.3(19) which says within the
*scope* of the completion, the first declaration is hidden from
all visibility.  I have proposed changing this to say where the completion
is visible (including via a renaming), the first declaration is hidden
from all visibility.  We can fiddle with these words further,
since they really only affect this new case.

> ... and (2) within
> the scope of a (nonlimited) with clause for the package that declares T.  I fail
> to see how this would cause a "with A" to hide P.Q.R.NP from all visibility.  In
> fact I fail to see how the presence of "with A" would have any bearing on the
> visibility of entities exported by P.Q.R.
>
> It seems to me that if we used availability, the completion of T would not be
> available in Main (regardless of whether there is a "with A" or not) and
> therefore we would have only one view of P.Q.R, the limited one.

I think you are perhaps muddying things by talking about "availability"
since that is a term that was introduced in an AI and never fully
explored.

Let's focus on this renaming thing.  I was trying to address
the concern that you didn't want to have two views of the same package
at the same place.  I presumed that referred to sub-packages as well.
If a renaming does not remove NP from the limited view of P.Q.R, then
you can see the limited view of P.Q.R.NP through that name,
and the full view of the same package via the renaming.  I can live
with that, but I thought we were trying to avoid that.

Let's decide that question, and then construct rules that accomplish
what we want.  We can call them "visibility" or "availability" or
"supercalifragility" rules or whatever once we decide what the rules
should be.

**************************************************************

From: Pascal Leroy
Sent: Friday, March  7, 2003  2:58 PM

> Let's focus on this renaming thing.  I was trying to address
> the concern that you didn't want to have two views of the same package
> at the same place.  I presumed that referred to sub-packages as well.
> If a renaming does not remove NP from the limited view of P.Q.R, then
> you can see the limited view of P.Q.R.NP through that name,
> and the full view of the same package via the renaming.  I can live
> with that, but I thought we were trying to avoid that.

I actually had a much more stringent requirement in mind: ideally, in the
course of compiling a compilation unit, there would be only one view of each
and every (compilation) unit in its closure.  If we manage to achieve this,
it would simplify the implementation considerably.  So for instance when
compiling Main in my example, you would either need to open P.Q.R'Full or
P.Q.R'Limited, but not both.  And you would probably know that early in the
compilation process, e.g. after processing the context clauses.

Regarding the incremental compilation issue, the legality of a name should
not be affected by adding/removing random declarations to random units (in
particular, package renamings).

Unsure how/if we can come up with rules that satisfy these requirements.

>We can call them "visibility" or "availability" or "supercalifragility"...

:-) :-)

**************************************************************

From: Randy Brukardt
Sent: Saturday, March 22, 2003 12:36 AM


This example shows two ways that I would use alternative #6 of AI-217
in the Claw Builder. See the similar write-up on alternative #5 for details
about the Builder and why it is important to be able to use this facility in
it.

[Editor's note: Here are those details.]

The Claw Builder is a real, existing program which could use this facility.
The problem is that some objects need references to other types of objects,
and these needs are circular. For instance, (some) types of window objects
include menu objects. And some types of menu objects include actions that
open a window.

The current Claw Builder solves this problem by using names rather than
access objects to connect the objects in some cases. This is usually done
only where necessary to break circularities. For instance, menu objects name
the windows they are to open, rather than linking to them. Using names
causes several problems:
   -- Accesses to the linked object is much slower, as they have to be
      looked up by name before use;
   -- If the user renames the linked object, we have to walk the entire
      project to insure any names are updated;
   -- If the user copies the linked object and then renames the copy (the
      required behavior), we have to be careful NOT to walk the project
      and update names -- harming code reuse.
   -- We can't have overloaded names (not a problem for windows, but can
      happen in other cases).

A root window object is an abstract object with a fairly large set of
operations. Each concrete object has to provide implementations for many of
these operations (some it can inherit). All of these operations are
dispatching. Typically, a user of the operation would apply it to a list of
windows using an iterator generic, with the operation dispatching to the
correct implementation. For the purposes of this discussion, we'll look at
just a few: Show, Hide, Display_Name.

The existing package looks something like:

   with CBuild_Menu;
   package CBuild_Root is
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with private;
      type Any_Window_Access_Type is access all Root_Window_Type'Class;

      procedure Show (Window : in out Root_Window_Type) is abstract;
      procedure Hide (Window : in out Root_Window_Type) is abstract;
      function Display_Name (Window : in Root_Window_Type)
          return String is abstract;
      ...
   private
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with record
          ... CBuild_Menu.Something ...
      end record;
   end CBuild_Root;

The menu package looks something like (greatly simplified):

  with CBuild_Id_Type;
  package CBuild_Menu is
    type Menu_Item is record
       Name : String (1..20);
       Action : Action_Type;
       Dialog : CBuild_Id_Type; -- The name of a dialog window.
           -- If Action=Open_Dialog.
    end record;
    procedure Simulate_Action (Item : in Menu_Item);
  end CBuild_Menu;

  with CBuild_Root, CBuild_Lists;
  package body CBuild_Menu is
    procedure Simulate_Action (Item : in Menu_Item) is
    begin
       if Item.Action = No_Action then
           null;
       elsif Item.Action = Open_Dialog then
           CBuild_Root.Show (
                 CBuild_Lists.Lookup(Item.Dialog,
                     CBuild_Data.Top_Level_Window_List));
       ... -- Other actions.
       end if;
    end Simulate_Action;
  end CBuild_Menu;

[End details copied from alternative #5.]

In order to directly use a Any_Window_Access_Type instead of the name in
CBuild_Menu, we would need restructure the CBuild_Root package. We'd need to
construct an abstract of this package for use in circular definitions. This
is necessary to have a single access type, which avoids the need to use a
conversion on nearly every use. Using alternative #7 (and assuming that
AI-326 is approved), this could look like:

   limited with CBuild_Root;
   package CBuild_Root_Abstract is
      type Any_Window_Access_Type is access all
          CBuild_Root.Root_Window_Type'Class;
   end CBuild_Root_Abstract;

This abstract would be used only when you need to declare instances (usually
components) of the type in locations where the reference would be circular.

Since we don't want an extra access type, the real package would be
modified to:

   with CBuild_Root_Abstract;
   with CBuild_Menu;
   package CBuild_Root is
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with private;
      subtype Any_Window_Access_Type is
CBuild_Root_Abstract.Any_Window_Access_Type;

      procedure Show (Window : in out Root_Window_Type) is abstract;
      procedure Hide (Window : in out Root_Window_Type) is abstract;
      function Display_Name (Window : in Root_Window_Type)
          return String is abstract;
      ...
   private
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with record
          ... CBuild_Menu.Something ...
      end record;
   end CBuild_Root;

The client package (in this case, for menus) would look something like:

  with CBuild_Root_Abstract;
  package CBuild_Menu is
    type Menu_Item is record
       Name : String (1..20);
       Action : Action_Type;
       Dialog : CBuild_Root_Abstract.Any_Window_Access_Type;
           -- If Action=Open_Dialog.
    end record;
    procedure Simulate_Action (Item : in Menu_Item);
  end CBuild_Menu;

  with CBuild_Root;
  package body CBuild_Menu is
    procedure Simulate_Action (Item : in Menu_Item) is
    begin
       if Item.Action = No_Action then
           null;
       elsif Item.Action = Open_Dialog then
           CBuild_Root.Show (Item.Dialog.all);
       ... -- Other actions.
       end if;
    end Simulate_Action;
  end CBuild_Menu;

As I've said twice before, I don't much like this solution, because it's not
clear when to use the abstract package, and when to use the main package. It
would be preferable to keep an understandable separation between them.

One way to do this is create a "client" package and a "creator" package. The
"client" package  would be used by ordinary client packages of the
abstraction. The "creator" package would be used only by packages that need
to create new extensions of the type or create objects of the type. The
"creator" package would, of course, be similar to the existing package. The
"client" package would provide enough operations that regular clients could
use it only.

Since the "client" package would be used most frequently, I've given it the
existing name.

   limited with CBuild_Root_Definition;
   package CBuild_Root is
      type Any_Window_Access_Type is access all
          CBuild_Root_Definition.Root_Window_Type'Class;

      procedure Show (Window : in out
          CBuild_Root_Definition.Root_Window_Type'Class);
      procedure Hide (Window : in out
          CBuild_Root_Definition.Root_Window_Type'Class);
      function Display_Name (Window : in
          CBuild_Root_Definition.Root_Window_Type'Class)
          return String;
      ...
   end CBuild_Root;

One annoyance here is the need to give the fully expanded name of the
incomplete type, over and over. That's because use clauses are prohibited on
the limited withed package. But I'll live. :-)

The "creator" package would be as in the previous example (except for the
name):

   with CBuild_Root;
   with CBuild_Menu;
   package CBuild_Root_Definition is
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with private;
      subtype Any_Window_Access_Type is CBuild_Root.Any_Window_Access_Type;

      procedure Show (Window : in out Root_Window_Type) is abstract;
      procedure Hide (Window : in out Root_Window_Type) is abstract;
      function Display_Name (Window : in Root_Window_Type)
          return String is abstract;
      ...
   private
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with record
          ... CBuild_Menu.Something ...
      end record;
   end CBuild_Root_Definition;

The body of CBuild_Root would need visibility on the completing type so that
it
could implement the procedures:

   with CBuild_Root_Definition;
   package body CBuild_Root is
      procedure Show (Window : in out
          CBuild_Root_Definition.Root_Window_Type'Class) is
      begin
          CBuild_Root_Definition.Show (Window);
      end Show;
      procedure Hide (Window : in out
          CBuild_Root_Definition.Root_Window_Type'Class) is
      begin
          CBuild_Root_Definition.Hide (Window);
      end Hide;
      function Display_Name (Window : in
          CBuild_Root_Definition.Root_Window_Type'Class)
          return String is
      begin
          return CBuild_Root_Definition.Display_Name (Window);
      end Display_Name;
      ...
   end CBuild_Root;

With this structure, the clients with CBuild_Root, and the concrete object
packages with (or are children of) CBuild_Root_Definition. The two packages
have clearly defined roles.

With these packages, the use of the types in the menu package is the same
for all three alternatives:

  with CBuild_Root;
  package CBuild_Menu is
    type Menu_Item is record
       Name : String (1..20);
       Action : Action_Type;
       Dialog : CBuild_Root.Any_Window_Access_Type;
            -- If Action=Open_Dialog.
    end record;
    procedure Simulate_Action (Item : in Menu_Item);
  end CBuild_Menu;

  package body CBuild_Menu is
    procedure Simulate_Action (Item : in Menu_Item) is
    begin
       if Item.Action = No_Action then
           null;
       elsif Item.Action = Open_Dialog then
           CBuild_Root.Show (Item.Dialog.all);
       ... -- Other actions.
       end if;
    end Simulate_Action;
  end CBuild_Menu;

---

Of course, the best way to avoid the issue of deciding when to use one of
the two packages is to avoid having the second one in the first place. The
primary reason we need this package is because of access type proliferation
problems. If the solution in AI-230 is adopted, we don't have that problem.
In that case, we can put the stub directly in the menus package.

In that case, the CBuild_Root package need not be modified at all:

   with CBuild_Menu;
   package CBuild_Root is
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with private;
      type Any_Window_Access_Type is access all Root_Window_Type'Class;

      procedure Show (Window : in out Root_Window_Type) is abstract;
      procedure Hide (Window : in out Root_Window_Type) is abstract;
      function Display_Name (Window : in Root_Window_Type)
          return String is abstract;
      ...
   private
      type Root_Window_Type is abstract new
	    Ada.Finalization.Limited_Controlled with record
          ... CBuild_Menu.Something ...
      end record;
   end CBuild_Root;

Then, the menu package would declare the stub and an anonymous access type:

  limited with CBuild_Root;
  package CBuild_Menu is
    type Menu_Item is record
       Name : String (1..20);
       Action : Action_Type;
       Dialog : access CBuild_Root.Root_Window_Type'Class;
           -- If Action=Open_Dialog.
    end record;
    procedure Simulate_Action (Item : in Menu_Item);
  end CBuild_Menu;

  with CBuild_Root;
  package body CBuild_Menu is
    procedure Simulate_Action (Item : in Menu_Item) is
    begin
       if Item.Action = No_Action then
           null;
       elsif Item.Action = Open_Dialog then
           CBuild_Root.Show (Item.Dialog.all);
       ... -- Other actions.
       end if;
    end Simulate_Action;
  end CBuild_Menu;

This solution clearly perturbs the existing program the least, so it is
preferred if it is available.

It's interesting to note that alternatives #5 and #6 look essentially the
same on this program. The structure of the program is the same, presuming
the same assumptions about the language features employed. The advantage of
alternative #5 is to be able to give a different, visible name to the stub
(which simplifies the rules and probably the implementation, but which I
didn't take advantage of in this moderately sized example). The advantage of
#6 is not to have to declare the stub at all.

Alternative #7 perturbs the structure of the program further, and can't
really handle the last example at all. That makes it a distant third in my
view. Alternative #6 probably has the edge on ease of use, but I'd like to
see a worked-out version whose visibility/availability rules really work
before I could even give it a partial endorsement. (And it always will be
much more expensive to implement.)

That concludes my look at implementing the Claw Builder using these
alternatives. One more piece of homework done. Yeah!!!

**************************************************************

From: Pascal Leroy
Sent: Friday, March 21, 2003  8:31 AM

> That means, in fact, that visibility IS the right model here, as long as
> it is clear that it is the visibility of the original completing
> declaration that we are talking about, and not the visibility of some subtype
> name that happens to be declared somewhere else.

The more I read chapter 8 (and the mail messages that Tuck and you
wrote) the more confused I am.

Consider the example:

    package P is
        type T is new Boolean;
    end P;

    with P;
    package Q is
    end Q;

    limited with P;
    with Q;
    package R is
        X : P.T; -- Legal?
    end R;

The question is: is the limited (or incomplete if you prefer) view of
P.T hidden from all visibility in R?  If I read Tuck's proposed change
to 8.3(19) it seems that the answer is a resounding Yes.

The reason is that 8.3(14) says that "a declaration is visible within
its scope, except when hidden from all visibility."  R is clearly part
of the scope of P.T, because Q is a semantic dependent of P and R is a
semantic dependent of Q.  And none of the "hidden from all visibility"
rules apply to P.T here (see 8.3(15-20)).  So the completion of P.T is
visible in R, and the incomplete view of P.T is hidden from all
visibility.

None of this is new, but in Ada 95 (where you don't have "limited with")
P is hidden from all visibility within R because of 8.3(20), so even
though P.T is visible in R, it cannot be named (wonderful language!).
However the "limited with" clause clearly makes (the limited view of) P
visible in R, and so P.T can be named, and the declaration of X is
legal.

But of course there is an awful ripple effect here, because removing the
"with P" in Q causes R to become illegal.  Yuck.

In essence, I guess I am saying that visibility has the same
transitivity effect as scope as far as I can tell.

**************************************************************

From: Tucker Taft
Sent: Friday, March 21, 2003  9:28 AM

> The question is: is the limited (or incomplete if you prefer) view of
> P.T hidden from all visibility in R?  If I read Tuck's proposed change
> to 8.3(19) it seems that the answer is a resounding Yes.

That was certainly not my intent.  I have suggested that 8.3(19)
be changed so that the hidden-from-all-visibility only happens
where the completion is *visible* as opposed to throughout
the scope of the completion.  In the above, the completion is
*not* visible inside R, so the incomplete declaration is
not hidden.  But as I read on...

> The reason is that 8.3(14) says that "a declaration is visible within
> its scope, except when hidden from all visibility."  R is clearly part
> of the scope of P.T, because Q is a semantic dependent of P and R is a
> semantic dependent of Q.  And none of the "hidden from all visibility"
> rules apply to P.T here (see 8.3(15-20)).  So the completion of P.T is
> visible in R, and the incomplete view of P.T is hidden from all
> visibility.

Hmmmm....  I see your point.  I was presuming that if there is
no way to name the enclosing package, then all the declarations
in the visible part of the package are similarly hidden from
all visibility.  The RM doesn't back up that view.  It looks
like in addition to changing 8.3(19) we would have to also
change 8.3(14) roughly as follows:

  A declaration is visible within its immediate scope except where hidden
  from all visibility.  A declaration that occurs immediately within the
  visible part of an enclosing visible declaration, or that is a public
  child of a visible declaration, is visible except where
  hidden from all visibility.  A declaration is hidden from all visibility
  as follows:


>
> None of this is new, but in Ada 95 (where you don't have "limited with")
> P is hidden from all visibility within R because of 8.3(20), so even
> though P.T is visible in R, it cannot be named (wonderful language!).
> However the "limited with" clause clearly makes (the limited view of) P
> visible in R, and so P.T can be named, and the declaration of X is
> legal.
>
> But of course there is an awful ripple effect here, because removing the
> "with P" in Q causes R to become illegal.  Yuck.
>
> In essence, I guess I am saying that visibility has the same
> transitivity effect as scope as far as I can tell.

Good point.  As suggested above, 8.3(14) would also have to be changed
to make the change to 8.3(19) useful.

**************************************************************

From: Pascal Leroy
Sent: Friday, March 21, 2003  3:28 PM

> Hmmmm....  I see your point.  I was presuming that if there is
> no way to name the enclosing package, then all the declarations
> in the visible part of the package are similarly hidden from
> all visibility.  The RM doesn't back up that view.  It looks
> like in addition to changing 8.3(19) we would have to also
> change 8.3(14) roughly as follows:
>
> A declaration is visible within its immediate scope except
> where hidden from all visibility.  A declaration that occurs immediately
> within the visible part of an enclosing visible declaration, or that
> is a public child of a visible declaration, is visible except where
> hidden from all visibility.  A declaration is hidden from
> all visibility as follows:

The second sentence doesn't seem to solve the problem.  It says in essence
"A declaration ... is visible except where hidden from all visibility".
That's not saying where it's visible (I presume that you don't want to say
that it's visible to the entire universe, even outside its scope).  I would
have expected a qualification like "is visible <at some places> except where
hidden from all visibility."

I'm not sure what to put in the <>, though.  Maybe "is visible at the places
where the enclosing declaration or parent unit is visible, except ..."?

**************************************************************

From: Tucker Taft
Sent: Friday, March 21, 2003  4:02 PM

We can just fall back on "scope" I think.
I.e.:
   Outside its immediate scope, but within its (extended) scope, a declaration that is not
   hidden from all visibility is visible if the declaration of the
   immediately enclosing declarative region, or a renaming thereof, is visible.

But this may be overkill, if we adopt one of the more restrictive versions
of 8.3(19) that I have suggested.  E.g., if we say that:

   a declaration is hidden from all visibility within the immediate scope
   of its completion, and within the extended scope of the completion where
   the declaration of the (completion's) enclosing library unit, or a library
   unit renaming thereof, is visible.

Or something like that...

**************************************************************

From: Randy Brukardt
Sent: Monday, March 24, 2003  3:31 PM

BTW, Pascal, how is AI-217-06 coming?

I've just been thinking that the 'two views' problem is impossible to avoid
without serious ripple effects. If that's true, of course, 'limited with' is
dead in the water, because it will be even harder to implement than
previously thought (I'd say impossible, but of course it's always possible
to implement anything self-consistent.)

The problem is that you have a 'view' of a package in your symboltable any
time the package is in the semantic closure. Thus, any rule that attempts to
avoid 'two views' issue is going to cause ripple effects.

For example:

      package P is
          type Root is abstract tagged ...;
      end P;

      with P;
      package Q is
          type Win is new Root with ...
      end Q;

      with Q;
      package R is
          Main : Q.Win;
      end R;

      limited with P;
      with Q, R;
      package S is
          type Any_Root is access all P.Root; -- Legal?
          Child : Q.Win := R.Main;
      end S;

Clearly, we have a full view of P in our semantic closure, so all of the
information about type Root will be loaded (but not visible). That's clearly
necessary so we can handle the initialization of Child. (In Janus/Ada, the
entire symboltable for the package is loaded in this case, but I suppose one
could arrange to load only a partial view - but that view necessarily has
quite a bit more information that the 'limited view' does.)

The rules that we've discussed for fixing this problem is that the limited
with would be either illegal or ignored in some case. In order to avoid
having two views, the rule clearly has to be effective any time the package
P is in the semantic closure (as in this example).

In that case, either the 'limited with P' is illegal (but that seems
confusing to the user, as there is no sign of any P in package S), or the
'limited with P' is ignored, meaning the 'P.Root' is illegal, since there is
no P visible (which is even more confusing to the user).

Moreover, if maintenance changes Q to not depend on P, then of course S
becomes legal: a classic ripple effect. Of course, the ripple effect is bad
if it occurs in the other direction. Presume that a fielded system was a
package Q that doesn't depend on P. S is legal. Then, if maintenance adds a
'with P' to package Q, S becomes illegal. (And S would be illegal even if
Child wasn't declared and package Q wasn't withed by S.)

Ripple effects like this can't be tolerated. Thus, it appears to me that we
cannot legislate away the 'two views' problem. That makes the implementation
cost of this proposal far more than we thought in Padua (and that was higher
than the other proposals to begin with).

**************************************************************

From: Pascal Leroy
Sent: Wednesday, March 26, 2003  8:49 AM

> I've just been thinking that the 'two views' problem is impossible to
> avoid without serious ripple effects. If that's true, of course, 'limited
> with' is dead in the water...

The first sentence may actually be true, but I fail to see the logical
connection with the second sentence.  Certainly the "type C.T" proposal
has exactly the same problem.  The type stubs proposal is a bit
different because the stub and the completion have different names, but
you still have to have both views in your hand and decide which one to
use.  Here is an example:

        limited with P;
        package P_Stub is
            type Root is tagged separate of P.Root;
        end P_Stub;

        with P_Stub;
        package P is
            type Root is tagged ...;
        end P;

        with P;
        package Q is
            type Arr is array (1..10) of P.Root;
        end Q;

        with Q;
        package R is
            Pain : Q.Arr;
        end R;

        with P_Stub;
        with Q, R;
        package S is
            X : P_Stub.Root; -- Illegal, completion not available.
            Y : Boolean := Q.Arr (1) = Q.Arr (2); -- Legal
        end S;

So on each construct you have to be extra-cautious to determine if you
are allowed to peek at the completion or if you must stop at the stub.
(There is a similar problem in the current language with private types.)

>       package P is
>           type Root is abstract tagged ...;
>       end P;
>
>       with P;
>       package Q is
>           type Win is new Q.Root with ...
>       end Q;
>
>       with Q;
>       package R is
>           Main : Q.Win;
>       end R;
>
>       limited with P;
>       with Q, R;
>       package S is
>           type Any_Root is access all P.Root; -- Legal?
>           Child : Q.Win := R.Main;
>       end S;

I have actually played with a different mechanism to solve this
conundrum.  I am very far from having RM words at this point, but the
idea would be that P.Root would become incomplete again in the scope of
the "limited with P".  This means that simply by looking at the context
clause (and the ones you inherits) you could decide which view of P
needs to be made available.  There would not be ripple effects, because
it would all be based on the scope of with clauses.

With this mechanism, the declaration of Any_Root would of course be
legal, but the declaration of Child would not, because Q.Win would be a
type with an incomplete component.  This might seem a bit odd, but it's
probably OK for the user: if you want to declare a variable of type
Q.Win, then surely you can write a "with P" so the "limited with P"
appears pointless.

I realize that this is all half-baked...

**************************************************************

From: Tucker Taft
Sent: Wednesday, March 26, 2003  9:46 AM

> I realize that this is all half-baked...

You should also include in every example a visible
non-library-unit renaming of P, just to be sure you
have a good answer for that.

**************************************************************

From: Randy Brukardt
Sent: Wednesday, March 26, 2003  1:29 PM

> The first sentence may actually be true, but I fail to see the logical
> connection with the second sentence.  Certainly the "type C.T" proposal
> has exactly the same problem.

type C.T does not meet the SigAda criteria IMO, and I'm thinking about it as
little as possible. I'd motion to kill it, but I'll have to wait until June
to do that.

...
> So on each construct you have to be extra-cautious to determine if you
> are allowed to peek at the completion or if you must stop at the stub.
> (There is a similar problem in the current language with private types.)

Of course, every Ada compiler has to be prepared to handle two views of the
same type. I'm concerned about two views of the same (original) fully
expanded name, and especially two views of the CONTENTS of the same name.

In Janus/Ada, we handle visibility of names by two mechanisms: standard
scoping, and a visibility bit. The second was added to handle things that
are invisible but still are in scope: units in the semantic closure,
inherited private record components, etc. Package renames are implemented by
a pointer to the original package node.

But neither of these allow a name to be both visible by one path and
invisible by another path. If we have a path that allows us access to all
the package contents, then any path that will get us to the package node is
going to have access to all the package contents.

Similarly, the limited with proposal doesn't bring discriminants in. These
of course are part of the contents of the type. If the type has
discriminants, we don't have a way to prevent accessing them (or knowing
about their existence) if some other view supposedly doesn't have them.
(Some such uses would be illegal for other reasons, but this certainly comes
up if we allow Tucker's deferencing rules; but I'd expect that it would come
up in other ways as well.)

> I have actually played with a different mechanism to solve this
> conundrum.  I am very far from having RM words at this point, but the
> idea would be that P.Root would become incomplete again in the scope of
> the "limited with P".  This means that simply by looking at the context
> clause (and the ones you inherits) you could decide which view of P
> needs to be made available.  There would not be ripple effects, because
> it would all be based on the scope of with clauses.

There would still be ripple effects in children, but since those happen in
the existing language, presumably we can ignore that. (Removing a with from
the parent can make children illegal).

> With this mechanism, the declaration of Any_Root would of course be
> legal, but the declaration of Child would not, because Q.Win would be a
> type with an incomplete component.  This might seem a bit odd, but it's
> probably OK for the user: if you want to declare a variable of type
> Q.Win, then surely you can write a "with P" so the "limited with P"
> appears pointless.

Well, that's counter to the principle of least visibility (that is, only use
the least visibility that you have to get the job done). That effectively
makes limited with look like a hack, because you can only use it in very
limited circumstances, and it possibly would break other, unrelated code. Of
course, it IS a hack; I'm certain that an elegant solution to this problem
doesn't exist.

In any case, this sounds like no real proposal is going to be on the table
soon, so I'm going to give this whole subject lower priority for now.

**************************************************************

From: Pascal Leroy
Sent: Thursday, March 27, 2003  4:54 AM

Well, I admit that it may be a bit surprising to the user, but not more
than ripple effects or that an unused package renaming in an auxiliary
unit affecting the visibility.  I contend that it's not going to happen
often in practice.  And it's probably not worse (and less frequent) than
name clashes in use clauses or reemergence in generics, which when you
think of it are odd too (only, we've got used to them).

>       package P is
>           type Root is abstract tagged ...;
>       end P;
>
>       with P;
>       package Q is
>           type Win is new Root with ...
>       end Q;
>
>       with Q;
>       package R is
>           Main : Q.Win;
>       end R;
>
>       limited with P;
>       with Q, R;
>       package S is
>           type Any_Root is access all P.Root; -- Legal?
>           Child : Q.Win := R.Main;
>       end S;

Look at your original example again.  Why the hell do you write "limited
with P" in S?  I can see two reasons:

1 - S and P are part of a cycle.  Then P will need to have a "limited
with S" and I'm not sure why you wouldn't write "with P" in S.  At any
rate, this looks like cheesy OO design to me.  You'll have to give a
more convincing example to demonstrate that there is a usability problem
here.

2 - You want to document/ensure that S only uses the limited view of P.
Then I am going to argue that my model is exactly what you want, because
it effectively hides the full view of P.

**************************************************************

From: Pascal Leroy
Sent: Thursday, March 27, 2003  5:18 AM

> I would encourage
> Pascal to produce something soon, even if it lacks
> detailed wording, so long as it captures his intent
> in a way that others can review it.

OK, you'll find below an attempt at capturing my ideas on visibility in
RM wording.  I'm pretty sure there will be adjustments needed (I never
seem to get wording right) but it should give enough detail for others
(in particular, Tuck and Randy) to review.

This approach is supposed to:

1 - Solve the two-view problem: at the end of the context clause the
compiler knows which view it must grab.
2 - Solve the ripple effect problem: it's all phrased in terms of the
scope of with clauses, so distant changes in visibility/renamings don't
affect the legality of a unit.

Assuming that these rules work, I would be strongly in favor of using
them for the "type C.T" proposal, as it has the same problems.  The
current rules of AI 217-07 are nearly impossible to implement for an
incremental compiler.  (Well, I would be strongly in favor of killing
AI 217-07 altogether, but that's a different discussion.)

Let me look specifically at two examples to explain the consequences of
the rules.  First, an example from the minutes of the Padua meeting:

    package Q is
        type T is ...;
        Z : T;
    end Q;

    limited with Q;
    package P is
        type Acc is access Q.T;
        X : Acc;
    end P;

    with Q;
    package A is
        package MyQ renames Q;
    end A;

    limited with Q;
    with P; with A;
    package R is
        P.X.all; -- Illegal, causes freezing of the incomplete type Q.T.
        A.MyQ.Z -- Illegal, full view of Q hidden from all visibility.
        Q.Z -- Illegal, full view of Q hidden from all visibility.
    end R;

And then the example that Randy circulated recently:

      package P is
          type Root is abstract tagged ...;
      end P;

      with P;
      package Q is
          type Win is new Root with ...
      end Q;

      with Q;
      package R is
          Main : Q.Win;
      end R;

      limited with P;
      with Q, R;
      package S is
          type Any_Root is access all P.Root; -- Legal.
          Child : Q.Win := R.Main; -- Illegal, freezing P.Root.
      end S;

The rules will be found below, each paragraph of wording following by a
[bracketed] paragraph explaining the purpose of the rule.

--

- Add after 10.1.2(5):

A with_clause is *more nested than* another with_clause if they are both
part of the same context_clause, or if the scope of the first
with_clause is entirely included in the scope of the second with_clause.

[We need to define the nesting of with_clauses to phrase a number of
rules that describe the interaction between nonlimited_ and
limited_with_clauses appearing on different units. Note that a
with_clause is more nested than itself (not that it matters much, but it
make some of the wording simpler).]

- Replace 10.1.2(6) by:

A library_item is *explicitly mentioned* in a with_clause if it is
denoted by a library_unit_name in the with_clause. A library_item is
mentioned in a nonlimited_with_clause if it is explicitly mentioned or
if it is denoted by a prefix in the with_clause. A library_item is
mentioned in a limited_with_clause if it is explicitly mentioned or if
it is denoted by a prefix in the with_clause, and that with_clause is
not more nested than a nonlimited_with_clause that mentions the same
library_item.

[For nonlimited_with_clauses, this doesn't change the current RM. For
limited_with_clause, it says that if for instance the specification of Q
has a "with P" and the body of Q has a "limited with P.R.S" then P is
not mentioned in the limited_with_clause. So the limited_with_clause has
no effect for P; in particular it doesn't revert to the limited view of
P. On the other hand, it makes the limited views of P.R and P.R.S
visible. This is mostly a methodological restriction, as we have
(pathological) cases where the limited view hides the full view.
However, we want to make such cases infrequent, in part to avoid
confusing users, and in part to account for the fact that the checks
associated with "reverting" to a limited view may be a bit costly.]

- Add after 10.1.2(8):

A limited_with_clause which explicitly mentions a library_item shall not
be more nested than a nonlimited_with_clause which mentions the same
library_item.

[Thus, if for instance the specification of Q has a "with P.R", it shall
not have a "limited with P" or a "limited P.R". The same applies to the
body of Q, its subunits, etc. On the other hand, a "limited with P.R.S"
is fine. Again this is to avoid reverting to the limited view as much as
possible.]

- Add after 8.3(20):

The full view of a library_item is hidden from all visibility within the
scope of a limited_with_clause that mentions it. The limited view of a
library_item is hidden from all visibility within the declarative region
of the library_item, and within the scope of a nonlimited_with_clause
that mentions it.

[The first sentence is the rule that prevents ripple effects and the
two-view problem. If you are is the scope of a "limited with P", the
full view of P is hidden from all visibility, even if P would otherwise
be visible indirectly (e.g. by a renaming in an auxiliary package). So
the limited_with_clause effectively reverts to the limited view of P,
and the types it declares are incomplete, and therefore subject to the
limitations described in 3.10.1. From an implementation standpoint, it
means that compilers can decide early (i.e., after looking at the
context_clauses of a unit and of its ancestry) what view of P needs to
be made available, and they never have to juggle with both views. Note
that I expect this rule to kick in only in rather pathological
situations: if you are at a place where the full view of P is visible,
it is unclear why you would want to write "limited with P", except
perhaps to document the fact that at this point you only use the limited
view.]

- Replace 3.11.1(8) by:

A type is completely defined at a place that is after its full type
definition (if it has one) and after all of its subcomponent types are
completely defined. Furthermore, if the type is declared within the
visible part of a library_item, it is *not* completely defined within
the scope of a limited_with_clause that mentions the enclosing
library_item. A construct that causes freezing of a type shall occur at
a place where the type is completely defined (see 13.14 and 7.3).

[Within the scope of a "limited with P", not only is P.T an incomplete
type, but any type that has a part of type P.T is incompletely defined.
Such types could exist because they would be declared in auxiliary units
where the full view of P is visible. This rule makes it possible to
piggyback on the freezing rules: within the scope of a "limited with P",
any construct that would cause freezing of P.T is illegal. Surely
compilers already have mechanisms to implement the freezing rules, and
for instance on every object_declaration they have to verify that the
type doesn't have incompletely defined subcomponents. It is hoped that
it would be relatively straightforward to modify these mechanisms to
implement this rule. One difference though is that currently compilers
never have to look at the parent part because 13.14(7) ensures that it
is frozen. To implement the new rule, the parent part would have to be
inspected. If performance is a concern, this part of the check could
only be performed in the scope of a limited_with_clause.]

- Replace 13.14(17) by:

A construct that causes freezing of a type shall occur at a place
where the type is completely defined.

[A simple copy of the last sentence of 3.11.1(8). This rule is redundant
anyway. Note that within the scope of a "limited with P" the type P.T is
incompletely defined, but it is still frozen as explained in the second
sentence of 13.14(2) (I don't want to mess with the freezing rules).
What we have here is a construct that causes freezing occurring after
the freezing point, and such a construct is illegal if the type has
reverted to "incompletely defined".]

**************************************************************

From: Tucker Taft
Sent: Thursday, March 27, 2003  5:49 AM

Looks good to me!  Thanks for making the effort
to provide this sooner rather than later.
I'll see if I can figure out how an approach like
this could apply to the incomplete child type
proposal, and I presume Randy will do the same
for the type stub proposal.

**************************************************************

From: Tucker Taft
Sent: Thursday, March 27, 2003  9:11 AM

I have interspersed some detailed comments with your
proposed wording.  But first, to summarize...

My major comment is that we should restrict ourselves to
talking about what is in the limited view of a package,
namely it contains limited views of nested packages, and
incomplete type declarations.  It seems unnecessary to go and
render incomplete other uses of a type declared in
the package, presuming those uses appeared in
places where a full view of the package existed
(e.g. objects of the type, or subtypes of the
type, or types derived from subtypes of the type, or ...).

This is mostly to simplify implementations, though I
think it also reduces the number of wording changes.  Right
now, the freezing rules never need to be of concern
when handling statements.  They only apply to expressions
and names that appear in declarations.  With the change you
are proposing, freezing checks have to be performed
in all statements as well, and that seems like overkill.

I think the very *good* part about your proposal is
that it implies that any renaming of a package, when
within the limited view of the package, is considered
limited.  I think that is the big improvement over
earlier proposals, and is probably the most intuitive
for the user.

If we focus just on the limited package view and what it contains,
and not worry about visible entities that are somehow
based on information gleaned from a full view of the package,
then that keeps the implementation effort focused
on package names, and whether they denote a full view
or an incomplete view.  That is a nice place to focus,
because there is already work needed when dealing with
package names and renames, to be sure to include children,
decide whether the private part is visible, etc.  This might
be a relatively small increment on that.  I fear if you
start burdening every object/subtype name usage with worries about limited
views, you will significantly increase the implementation burden.

...
> - Replace 10.1.2(6) by:
>
> A library_item is *explicitly mentioned* in a with_clause if it is
> denoted by a library_unit_name in the with_clause. A library_item is
> mentioned in a nonlimited_with_clause if it is explicitly mentioned or
> if it is denoted by a prefix in the with_clause. A library_item is
> mentioned in a limited_with_clause if it is explicitly mentioned or if
> it is denoted by a prefix in the with_clause, and that with_clause is
> not more nested than a nonlimited_with_clause that mentions the same
> library_item.
>
> [For nonlimited_with_clauses, this doesn't change the current RM. For
> limited_with_clause, it says that if for instance the specification of Q
> has a "with P" and the body of Q has a "limited with P.R.S" then P is
> not mentioned in the limited_with_clause. So the limited_with_clause has
> no effect for P; in particular it doesn't revert to the limited view of
> P. On the other hand, it makes the limited views of P.R and P.R.S
> visible. This is mostly a methodological restriction, as we have
> (pathological) cases where the limited view hides the full view.
> However, we want to make such cases infrequent, in part to avoid
> confusing users, and in part to account for the fact that the checks
> associated with "reverting" to a limited view may be a bit costly.]


I think we might be able to simplify this by talking about the
scope of a limited view of a package, and say that it is the scope of
a limited with clause that mentions it minus the scope of any
more nested (nonlimited) with clause that mentions it, or
any renaming of it.  I presume we are still disallowing a
limited with mentioning a library-unit renaming.

> - Add after 10.1.2(8):
>
> A limited_with_clause which explicitly mentions a library_item shall not
> be more nested than a nonlimited_with_clause which mentions the same
> library_item.

I think this is approximately equivalent to saying that a limited
with clause is illegal if it has no effect (because it is immediately
canceled by some outer or same-context-clause nonlimited with).
Withs that "skip over" via renamings one of the enclosing packages
mentioned in the limited with make this only an approximate equivalence.

By the way, I think we may want to disallow a limited with of
a package inside a "use" clause for a renaming of the package
(or nested package thereof).  Or we need to make the "use"
lose its effect.  In any case, I don't think we want to allow
(new) "use" clauses of limited views of packages.

...
> - Add after 8.3(20):
>
> The full view of a library_item is hidden from all visibility within the
> scope of a limited_with_clause that mentions it. The limited view of a
> library_item is hidden from all visibility within the declarative region
> of the library_item, and within the scope of a nonlimited_with_clause
> that mentions it.

I think you want to talk about library packages here rather than
library "items", and make it clear that a non-limited with
of a renaming of a library package also hides the limited view.
Rather than talking about "hiding" the limited view, we could
talk about its scope.  We might also want to talk about the
scope of the full view instead of using hiding for that.
Essentially the scope of the full view is as defined now,
except it does not include places that are within the scope
of the limited view.  The scope of the limited view is
defined in terms of the scope of limited/non-limited with clauses
that mention it (or a renaming thereof).

...
> - Replace 3.11.1(8) by:
>
> A type is completely defined at a place that is after its full type
> definition (if it has one) and after all of its subcomponent types are
> completely defined. Furthermore, if the type is declared within the
> visible part of a library_item, it is *not* completely defined within
> the scope of a limited_with_clause that mentions the enclosing
> library_item. A construct that causes freezing of a type shall occur at
> a place where the type is completely defined (see 13.14 and 7.3).

This is where I think you are getting more complicated than
necessary.  Rather than talking about whether a type is completely
defined, I would talk only about what P.T denotes, when
P is a limited view of a package.  I think you are asking
for a *much* more complicated implementation effort if
you are expecting every use of a type, including in subprogram
bodies, to check to see whether it should be considered incomplete.
Why not just say that within the limited view of a package P,
"P.T", or "Q.T" where Q is any renaming of P, denotes
an incomplete view of T.  Other subtypes declared using
full views of T, or objects declared using full views of
T, are not affected.  And of course, a limited view of
a package only contains nested packages and incomplete
type declarations, nothing else.

...
> - Replace 13.14(17) by:
>
> A construct that causes freezing of a type shall occur at a place
> where the type is completely defined.

Again, I don't think we need to muck with the freezing rules.

**************************************************************

From: Pascal Leroy
Sent: Thursday, March 27, 2003  10:55 AM

> My major comment is that we should restrict ourselves to
> talking about what is in the limited view of a package,
> namely it contains limited views of nested packages, and
> incomplete type declarations.  It seems unnecessary to go and
> render incomplete other uses of a type declared in
> the package, presuming those uses appeared in
> places where a full view of the package existed
> (e.g. objects of the type, or subtypes of the
> type, or types derived from subtypes of the type,
> or ...).

I am confused here.  The problem I am trying to solve is the two-view
problem, i.e. the situation where the compiler has to juggle with the
limited and the full view of the unit.  My fear is that, if that if we
don't have the "revert to incomplete" rule, the compiler will end up
wandering in the symbol table for the full view of the unit, and we
won't have solved the problem.

Or maybe this won't happen, i.e. the compiler will never try to look at
the full view of the unit?  Hmmm, not sure if all the rules of the
language combine to prevent this problem from happening...

This being said, I must admit that the notion of checking the freezing
rules in statement parts is unappealing.

**************************************************************

From: Tucker Taft
Sent: Thursday, March 27, 2003  2:13 PM

If you make all renames of the package show the limited view,
and you disallow "use"s, then I don't see why you would be wandering
in the package's full-view symbol table at all.  But your
mileage may vary...

> This being said, I must admit that the notion of checking the freezing
> rules in statement parts is unappealing.

And I think its worse than that.  Once we freeze something, we
mark it frozen.  We would have to ignore that flag and, pretty
much on every use, have to check whether the entity is declared
somewhere inside a package that has its limited view in scope.
Uggh.

**************************************************************

From: Randy Brukardt
Sent: Thursday, March 27, 2003  6:11 PM

I don't see that as particularly bad: it just means more work for the
Check_Freezing subprogram. (Or whatever it's called). And if there are
places where that's not called, they'd have to be added. I doubt that is
anywhere near as much work as implementing limited views themselves.

(But, of course, a better idea is welcome...)

**************************************************************

From: Pascal Leroy
Sent: Friday, March 28, 2003  4:48 AM

My opinion exactly.  And I doubt that is anywhere near as much work as
juggling with two views and having to decide on each and every construct
of the language which view you are allowed to access.

**************************************************************

From: Pascal Leroy
Sent: Thursday, March 27, 2003  3:32 PM

> If you make all renames of the package show the limited view,
> and you disallow "use"s, then I don't see why you would be wandering
> in the package's full-view symbol table at all.  But your
> mileage may vary...

Consider the following example:

    package P is
        type T is tagged;
        function "=" (L, R : T) return Boolean;
    end P;

    with P;
    package Q is
        type A is array (1..10) of T;
        X, Y : A;
    end Q;

    with Q;
    limited with P;
    procedure Main is
    begin
        if Q.X = Q.Y then
            ...
        end if;
    end Main;

One code generation strategy for the equality in Main is to generate a loop
calling P."=" for each component.  I fail to see how this can be done
without accessing the full view of P.  In particular, a clever compiler
might just want to use a block comparison if T has no user-defined "=", so
it will surely need to look at the full view of P.

Also, from the user standpoint, I think that operating on objects (or types)
which have incomplete parts is counter-intuitive and can lead to all sorts
of surprises.

**************************************************************

From: Tucker Taft
Sent: Thursday, March 27, 2003  4:58 PM

I think we weren't understanding one another.  I certainly
agree you need to get to the full view of the types declared
in P, and all of their "basic" operations, because there
might be subtypes, objects, derived types, etc. of them around.

I just think you don't need to get to the full view of the
package itself.  That is, while within the limited view of P,
you don't need to do a lookup in the full symbol table of P.

Perhaps what you are saying is that you do a *lookup* in P to
find the primitive "=" on tagged T needed for creating a composite
"="?  I guess that surprises me.  With this proposal, I think
you will have to eliminate any such "implicit" lookups if you
do them, and simply store on the type node a reference to its
primitive "=" if it is tagged.

The explicit operator lookups are fine, since the package cannot
be used (and presumably, nor can any types declared in the package)
while in the limited view of the package.  Hence the explicit
operator lookups will never find anything.

> Also, from the user standpoint, I think that operating on objects (or types)
> which have incomplete parts is counter-intuitive and can lead to all sorts
> of surprises.

I agree.

I don't think this is the way they should or would think about it.
The parts don't become limited, just because we are in the limited
view of a package where a part's type might have been defined.

The key point is that anything already extracted out of the full view of
the package is still fine.  You just can't extract anything
more out of the package, while in its limited view.

**************************************************************

From: Randy Brukardt
Sent: Thursday, March 27, 2003  6:20 PM

Right. You're going to have to have the full view in your symboltable, like
it or not. I know that Janus/Ada will fail spectacularly if a package in the
semantic closure isn't loaded. (We had a number of bugs like that in the
past.) So, there is no option to avoid loading the full views of everything
in the semantic closure.

But, we can't allow the package view to be full to some users and limited to
others. And we can't just use the semantic dependence because of the ripple
effects. So I think Tucker's proposal is about right. If the package was in
the semantic closure, we'd use a subprogram to go through it and turn off
the visibility bits of everything that is not a package or a type.
That means that there would be only one lookup view. The other checking
would have to be done at the point of use (there wouldn't be anyway to mark
it) - presumably the code that determines if we have enough visibility to
proceed from the incomplete view to the complete view would have to muck
around in a list of with clauses. (I don't think that such a list exists
currently.) If the view is limited, we don't allow moving to the complete
view.

But I'll have to think about this more to be sure that this is doable.

**************************************************************

From: Robert Eachus
Sent: Thursday, March 27, 2003  5:06 PM

I have been reading this ongoing discussion with stunned disbelief.  Not
that the details are complex, or that some solution is required.
It is just that for many of these problems, the simple solution is to
make the complicating factors illegal.

For example, the issue of renaming packages which contains potentially
incomplete types.  The simple solution is to make it illegal to rename a
package if the context clause doesn't contain the completion of any
incomplete types in that package.

The problem we are trying to solve could be solved by allowing
implementations to require both the unit with the incomplete type and
its completion to be compiled together.  I think that is a bit
draconian, but I also think that some of the problem cases being
discussed are a result of trying to make the rules for these types too
orthogonal.

I can live with some pretty severe restrictions to make such
declarations available.  Not renaming packages with an incomplete view
of a type is pretty mild.

**************************************************************

From: Randy Brukardt
Sent: Thursday, March 27, 2003  6:29 PM

> It is just that for many of these problems, the simple solution is to
> make the complicating factors illegal.

No, we can't do that, because that is precisely the ripple effect that we're
trying to avoid: adding or removing a with clause on a distant unit changing
the legality of a unit.

> For example, the issue of renaming packages which contains potentially
> incomplete types. The simple solution is to make it illegal to rename a
> package if the context clause doesn't contain the completion of any
> incomplete types in that package.

That's not the issue at all. Look at the examples that Pascal posted more
closely. The package which was renamed was a full view. The incomplete view
is coming from the local package; the full view is coming from the semantic
closure. The point of that example is that visibility isn't the right model,
because with a renaming, we can have visibility onto the full view -- and
thus even using a visibility model causes ripple effects.

> The problem we are trying to solve could be solved by allowing
> implementations to require both the unit with the incomplete type and
> its completion to be compiled together.  I think that is a bit
> draconian, but I also think that some of the problem cases being
> discussed are a result of trying to make the rules for these
> types too orthogonal.

That's similar to what alternative 05 does for type stubs, but is much
stronger. And I don't think it helps, but it makes it a lot harder to use
these things (particularly if you want to avoid extra packages). Besides,
"allowing" implementations to do something is not likely to be helpful if
the implementation has no way to do anything with that (that's certainly
true of Janus/Ada, which can only process one unit at a time, and I think it
is true of Gnat as well).

**************************************************************

From: Robert Eachus
Sent: Thursday, March 27, 2003  8:44 PM

I think you are taking what I was saying a little too literally.  If it
takes subtle analysis do decide when a renaming needs to be outlawed
because it causes visibility clashes, fine.  All the user needs to know
is that a renaming of a package with an incomplete type may be illegal
in some contexts.  That won't bother many, if any, users.  (Most package
renamings I see are either library renamings or abbreviations of
predefined language units.)

**************************************************************

From: Tucker Taft
Sent: Thursday, March 27, 2003  9:21 PM

Robert,
    You are definitely confused about the issue.
We have already agreed to disallow renaming
of limited views.  What we are dealing with
now are renamings that were done in a place which
only saw the full view, and is totally unaware that
a "limited with" even exists.  The problem is
that a unit that has a limited-with might also
"with", directly or indirectly, something that
already has a rename of the full view.  This means
that it can get at this renaming.  What should
it see if it tries to use the renaming.  Should it
see the full view, which was what existed when the
renaming was declared, or should it see the
limited view, which is all it can see by naming
the package directly?

In Pascal's proposal, you will see the limited view,
even though when the renaming was declared, it was
a renaming of the full view.  I think Pascal's
proposal is the right answer here.  It gibes with
what happens when you go through a renaming of
a library unit with children.  The children you
see are based on those that are visible at the
point of reference to the renaming, rather than
at the point where the renaming was declared.

**************************************************************

From: Robert Dewar
Sent: Thursday, March 27, 2003  10:12 PM

> That's similar to what alternative 05 does for type stubs, but is much
> stronger. And I don't think it helps, but it makes it a lot harder to use
> these things (particularly if you want to avoid extra packages). Besides,
> "allowing" implementations to do something is not likely to be helpful if
> the implementation has no way to do anything with that (that's certainly
> true of Janus/Ada, which can only process one unit at a time, and I think it
> is true of Gnat as well).

Actually it is not true of GNAT at all, it is perfectly reasonable for GNAT to
process multiple units in the sense of requiring some units to be around when
another unit is compiled.

**************************************************************

From: Robert Eachus
Sent: Thursday, March 27, 2003  11:09 PM

Exactly the situation I thought I was addressing. ;-)  Pascal's approach
of seeing the limited view is troubling, but I think we can outlaw the
case without much pain.  Yes, it means making it illegal for a unit with
a limited view of a type to also have a full view of the type.

The problem case would seem to be a package that created many types
completed in several other units.  This would say that in that case you
couldn't use the completed versions of some types in the bodies of
packages that completed other types.  This would occasionally require
multiple child packages, some of which completed types, others that
created objects of those types.  Shrug.

If we are trying to compare several approaches on the assumption that
they will cause minimal implementation pain, I think we should be
willing to accept such limitations and see where it leads.  If one of
the three approaches looks much better given such limitations, great.
Then we can choose that approach, knowing that it is a workable solution
and see how much we can extend it before the implementation difficulties
become too great.  That is what I was driving at.

The current approach seems to be to have full implementation designs for
the most general version of each approach, and I think that that way
lies madness.

In this sense I think Robert Dewar's recent message is a breath of fresh
air.  GNAT might or might not "take advantage" of some permitted
implementation freedom.  That doesn't mean we should reject the
potential restriction ahead of time.  If allowing the restriction now
makes a particular approach acceptable, we can either argue the politics
later, or accept the restriction for now, and look into removing it
later.  An implementation restriction on a feature that currently
doesn't exist is not a hardship for anyone.

**************************************************************

From: Tucker Taft
Sent: Thursday, March 27, 2003  9:37 PM

Well I haven't seen a real answer to my suggestion:
Being in the limited view of a package should have no
effect on objects or subtypes declared using the
full view.  The only effect should be on what you
get if you (explicitly) look up P.T, or renaming-of-P.T, and what
you get is an incomplete view of T.  But there might
be full views of T already embedded in objects or
subtypes or derived types or composite types, and
none of those are affected.  If implicit lookups
in P are needed to support these preexisting
objects/subtypes/derived types, then those implicit
lookups should use the full view, but these should
be rare, and could probably be handled in some other
way (e.g., hanging the primitive "=" directly off
the type, rather than having to look it up each
time it is needed to implement the equality of
an enclosing composite object).

I think this eliminates any need to futz/muck/mess
with freezing rules, and localizes the changes to
explicit package lookups.

This is reminiscent of what we already do for
library packages.  We have the original view
of a package, and then we construct an "augmented"
view when a package is "with"ed, and in the augment
view we add children that get "with"ed.  The original
view is preserved in a read-only state.  The
augmented view is built up incrementally during
context clause processing.

I could see also having a "limited" view for a package.
It could be constructed based on the full view if available,
or by doing a "loose parse" of the source of the package.
This limited view would be used for all explicit lookups
in the package when the limited view is in scope
due to mention in a limited with clause (and not
"canceled" due to mention in an inner non-limited "with").

**************************************************************

From: Pascal Leroy
Sent: Friday, March 28, 2003  4:47 AM

First, let me make clear that from my perspective, the Ripple Effect
Problem is way more severe than the Two-View Problem.  The former is a
no-no for me, as it kills incremental compilation.  The latter is a
nuisance, but if I absolutely have to, I suppose I can live with it.

So my objection to your position, Tucker, is that, if you don't have the
"revert to incomplete" rule, the language design looks botched.  For
instance:

    package P is
        type T is ...;
    end P;

    with P;
    package Q is
        type T is
            record
                C : P.T;
            end record;
    end Q;

    limited with P;
    with Q;
    package Main is
        X : P.T; -- Illegal
        Y : Q.T; -- Legal
    end Main;

The fact that the declaration of X is illegal but the declaration of Y
is legal looks like a wart to me.  Try to explain this to real users!
It would seem much more natural to say that they are both illegal.

As I understand it, you main objection is one of implementation
complexity (I don't buy arguments based on the number of words in the RM
that have to change).  Since all incarnations of AI 217 will require a
lot of work anyway, I cannot get too excited.  It seems to me that we
should concentrate on finding a nice and clean language mechanism, even
if that's at the expense of some implementation complexity.

> I think we weren't understanding one another.  I certainly
> agree you need to get to the full view of the types declared
> in P, and all of their "basic" operations, because there
> might be subtypes, objects, derived types, etc. of them around.
>
> I just think you don't need to get to the full view of the
> package itself.  That is, while within the limited view of P,
> you don't need to do a lookup in the full symbol table of P.
>
> Perhaps what you are saying is that you do a *lookup* in P to
> find the primitive "=" on tagged T needed for creating a composite
> "="?  I guess that surprises me.

After checking the code, it turns out that we don't in this case.  But
there are other situations where we do.  For instance:

    package P is
        type T is ...;
        function F return T;
    end P;

    with P;
    package Q is
        subtype S is P.T;
    end Q;

    limited with P;
    with Q;
    package Main is
        type NT is new Q.S;
    end Main;

In order to build the inherited subprograms for NT, we scan the
declarative part where T is declared, i.e. the full view of P.  And we
are certainly not going to change this, it would be way too disruptive.
If we had to implement the rules that you propose, we would have to live
with the Two-View Problem.  But again, my objection is mostly one of
language design.

> And I think its worse than that.  Once we freeze something, we
> mark it frozen.  We would have to ignore that flag and, pretty
> much on every use, have to check whether the entity is declared
> somewhere inside a package that has its limited view in scope.

I realize that implementation complexity depends a lot on how things are
structured.  However I am a bit surprised by your claim, because for us
changing the freezing rules to deal with type "reverting to incomplete"
would literally be an afternoon project.  Let me give a few
implementation notes to make the discussion more concrete.

When a type gets frozen we do mark it with a bit, Sm_Frozen, in the
Diana tree.  It turns out that we run the freezing checks on statement
parts as well as declarative parts, because some constructs in statement
parts declare anonymous subtypes that need to get frozen (don't ask).
Most of the time this processing ends very quickly because it finds that
Sm_Frozen is set.  But I realize that this is an idiosyncrasy of our
implementation, and that most compilers probably don't look at statement
parts.

To implements the "revert to incomplete" rule, I would probably have a
global bit indicating whether or not I am in the scope of a limited
with.  (I could be more clever and only set this bit when I am in the
scope of a "limited with P" and P is in the closure through some other
path, but I guess that I would be too lazy to do that.)  During the
freezing checks, this global bit would cause the setting of Sm_Frozen to
be ignored, so we would effectively traverse the subcomponents, etc. of
each type.  Once a type has been found to be OK (i.e., to not have any
incomplete subcomponent) I would mark it with a temporary bit so as to
avoid redoing the traversal over and over again.  This means that every
type would be analyzed (at most) once, so the performance impact should
be minimum.  Obviously there would be no performance impact at all when
you are not in the scope of a limited with.

This doesn't sound like rocket science to me.  Could you maybe explain
your implementation concerns in a bit more detail?

**************************************************************

From: Pascal Leroy
Sent: Friday, March 28, 2003  4:54 AM

> Well I haven't seen a real answer to my suggestion:
> Being in the limited view of a package should have no
> effect on objects or subtypes declared using the
> full view.  The only effect should be on what you
> get if you (explicitly) look up P.T, or renaming-of-P.T, and what
> you get is an incomplete view of T.  But there might
> be full views of T already embedded in objects or
> subtypes or derived types or composite types, and
> none of those are affected.

If you want my "real answer" to this it is that this approach really
looks like a hack and is going to puzzle users.

**************************************************************

From: Tucker Taft
Sent: Friday, March 28, 2003  8:28 AM

 > So my objection to your position, Tucker, is that, if you don't have the
 > "revert to incomplete" rule, the language design looks botched.

I don't agree, but in any case, we should agree we
are talking about corner cases here, because this only
occurs when you are in the scope of a limited view, but
the full view is in your semantic closure.  Hence, I believe
as few words and as little implementation complexity should be
devoted to this area as possible, subject to eliminating
ripple effects.  I believe the key thing is to limit ourselves
to talking about limited views of *packages*, and not get
hung up on imposing limited views on anything else.
Packages are already a bit funny because of public/private
children and visible/private parts.

 > For
 > instance:
 >
 >     package P is
 >         type T is ...;
 >     end P;
 >
 >     with P;
 >     package Q is
 >         type T is
 >             record
 >                 C : P.T;
 >             end record;
 >     end Q;
 >
 >     limited with P;
 >     with Q;
 >     package Main is
 >         X : P.T; -- Illegal
 >         Y : Q.T; -- Legal
 >     end Main;
 >
 > The fact that the declaration of X is illegal but the declaration of Y
 > is legal looks like a wart to me.

It doesn't look like a wart to me, because the types used to
implement Q.T are the business of Q, and should not be
relevant to a user of Q.  Suppose Q.T were implemented in
terms of U.T, which is implemented in terms of V.T, which
is implemented in terms of W.T, which just happens to have
a teeny little component in one of its many variant parts that
happens to be of a type that is derived from P.T?

Or suppose we change Q to:

    with P;
    package Q is
        type R is private;
        ...
    private
        type R is record
            C : P.T;
        end record;
    end Q;

Clearly with your defender-of-privacy hat on,
you wouldn't want the characteristics of R to
be affected when a user of Q happens to have
a limited view of P.

 > ...  Try to explain this to real users!

The key is that the compilation unit has a limited view of *P*,
so it can only see certain things in P, and it sees only
some features of those via an expanded name involving P
(or one of its renamings).  It doesn't affect any other
compilation unit, nor any names that don't involve P or
one of its renamings.

 > It would seem much more natural to say that they are both illegal.

I don't agree, and I really think this way lays big trouble.
Suppose you instantiate a generic, which instantiates a
generic, which declares an object of a derived type of P.T.
I really think you are recreating a ripple effect, where
some distant package's decision to use something from
P affects whether you can instantiate it.


 >
 > As I understand it, you main objection is one of implementation
 > complexity (I don't buy arguments based on the number of words in the RM
 > that have to change).

I agree that number of words is not a great guage, but it
is relevant, and especially when you are affecting the
legality of essentially every construct in the language.

Would it be legal to call a subprogram that had a default
expression that happened to use a 'Access attribute of an
object of type P.T?  Or how about use a record type where one of
the components had a default expression that referenced the
identity attribute of an exception declared in P?

 > ...  Since all incarnations of AI 217 will require a
 > lot of work anyway, I cannot get too excited.  It seems to me that we
 > should concentrate on finding a nice and clean language mechanism, even
 > if that's at the expense of some implementation complexity.

Well, I believe your proposal is much cleaner if you focus
strictly on what you can see when you explicitly try
to refer to components of a limited view of a package
via an expanded name.

You are opening an unnecessary can of very nasty worms
if you start changing the characteristics of things
in other compilation units, other than renamings of
this same package.

...
 > In order to build the inherited subprograms for NT, we scan the
 > declarative part where T is declared, i.e. the full view of P.  And we
 > are certainly not going to change this, it would be way too disruptive.
 > If we had to implement the rules that you propose, we would have to live
 > with the Two-View Problem.  But again, my objection is mostly one of
 > language design.

I guess we have a list of primitives hanging off every type, so
we never have to scan the package itself, but I agree, deriving
is probably tricky.  But again, what happens if the type you
are deriving from is a private extension of some ancestor of
P.T, but when you get into the private part, you discover its
immediate parent is P.T itself.  I really think this shouldn't
affect legality of the new type derivation.

 >>And I think its worse than that.  Once we freeze something, we
 >>mark it frozen.  We would have to ignore that flag and, pretty
 >>much on every use, have to check whether the entity is declared
 >>somewhere inside a package that has its limited view in scope.
 >
 >
 > I realize that implementation complexity depends a lot on how things are
 > structured.  However I am a bit surprised by your claim, because for us
 > changing the freezing rules to deal with type "reverting to incomplete"
 > would literally be an afternoon project.  Let me give a few
 > implementation notes to make the discussion more concrete.
 > ...
 > This doesn't sound like rocket science to me.  Could you maybe explain
 > your implementation concerns in a bit more detail?

I don't want to focus the argument on our implementation
complexity, because I think the argument can be better
focused on areas like privacy and ripple effects which
I know are near-and-dear to your heart.

But in general, I think making this proposal depend
subtly on the freezing rules, and what is
frozen by what sorts of uses, is a mistake, and
completely unnecessary to achieve all of your objectives.

As I said in my first paragraph, we are talking about
corner cases, and they don't deserve the effort that
will be involved in defining exactly what is legal when using
entities that in some vague way depend on the full
view of P.  Renamings of P are clear, everything else
just gets muddier and muddier in my view.

**************************************************************

From: Pascal Leroy
Sent: Friday, March 28, 2003  8:48 AM

> I don't agree, but in any case, we should agree we
> are talking about corner cases here, because this only
> occurs when you are in the scope of a limited view, but
> the full view is in your semantic closure.

I definitely agree that this is a corner case.  I still think it's
important to come up with a model that is not too surprising to users,
and consistent with the rest of the language.  But if we can't, I'm not
going to throw the baby with the bathwater.

> Or suppose we change Q to:
>
>     with P;
>     package Q is
>         type R is private;
>         ...
>     private
>         type R is record
>             C : P.T;
>         end record;
>     end Q;
>
> Clearly with your defender-of-privacy hat on,
> you wouldn't want the characteristics of R to
> be affected when a user of Q happens to have
> a limited view of P.

That's a rather compelling argument, and I need to give it more thought
over the week-end.  Of course, my immediate answer would be to say that
you have to assume-the-worst for private types, but I'm not sure I like
that, because it would disallow perfectly reasonable usages of perfectly
good private types.

> I don't want to focus the argument on our implementation
> complexity, because I think the argument can be better
> focused on areas like privacy and ripple effects which
> I know are near-and-dear to your heart.

Right, language design, not implementation complexity, should be the
focus here.

**************************************************************

From: Randy Brukardt
Sent: Friday, March 28, 2003  7:24 PM

> I guess we have a list of primitives hanging off every type, so
> we never have to scan the package itself, but I agree, deriving
> is probably tricky.  But again, what happens if the type you
> are deriving from is a private extension of some ancestor of
> P.T, but when you get into the private part, you discover its
> immediate parent is P.T itself.  I really think this shouldn't
> affect legality of the new type derivation.

Yuck! This example gives me heartburn, precisely because we do exactly what
Rational does. It MIGHT work anyway (because I think that the visibility
bits are ignored for the purposes of derivation - but I'm not at all sure
about that) -- that's a very fragile area, and I hate the idea of having to
muck with it because of some completely unrelated feature.

I guess what worries me here is that we're going to end up with two views in
all cases except those that cause ripple effects -- which is hardly going to
help implementation or usability.

But I don't see a showstopper here. And I certainly agree that this is a
corner case; having it perfect isn't critical. (The same can be said about
dereferencing rules, BTW, because the only cases that we'd be worrying about
are the same ones as here: where we're in the semantic closure of P but only
have a limited view. For type stubs, that's where we're in the semantic
closure of the completing type, but only have visibility on the incomplete
type. Neither sounds real likely.)

**************************************************************

From: Dan Eilers
Sent: Friday, March 28, 2003 11:10 AM

> The fact that the declaration of X is illegal but the declaration of Y
> is legal looks like a wart to me.  Try to explain this to real users!
> It would seem much more natural to say that they are both illegal.

This looks easy to explain to real users by citing the rule that
adding a "with" clause to a unit can't make anything in the unit
illegal that was previously legal.  The rule is just being extended
to "limited with" clauses.

**************************************************************

From: Tucker Taft
Sent: Friday, March 28, 2003 11:42 AM

I agree with this principle, and we should try to preserve it, though it isn't
quite true for "with" clauses, and it won't be true with limited with if you
already refer to a renaming of the package (which is admittedly a real corner
case).  But I agree that adding a limited with for package P should not affect
the legality of anything in the compilation unit other than existing references
to package P or renamings thereof (or nested packages thereof).

It isn't quite true for "with" clauses because adding a "with" clause can hide
something that was "use" visible.  E.g.:

   package P1 is
       type P2 is (tbd);
   end P1;

   with P1; use P1;
   with P2;  -- Makes decl of X illegal
   procedure Main is
       X : P2;  -- becomes illegal
   begin null end;

But if the name "P2" doesn't appear in the compilation unit, then adding
a "with" clause for P2 cannot make anything illegal.

**************************************************************

From: Robert I. Eachus
Sent: Friday, March 28, 2003  7:37 PM

> I agree with this principle, and we should try to preserve it, though
> it isn't quite true for "with" clauses, and it won't be true with limited
> with if you already refer to a renaming of the package (which is admittedly
> a real corner case).  But I agree that adding a limited with for package P should
> not affect the legality of anything in the compilation unit other
> than existing references to package P or renamings thereof (or nested packages thereof).

I can't believe I can see this so clearly, and the everyone else keeps
looking at the trees.  Let me try to explain my logic, and see if you
can punch holes in it.  Something is wrong, and I suspect that I am just
not communicating my point clearly enough....

Thesis:  It should be illegal for a context to include a  "limited with
P;" clause if the remainder of the context semantically depends on P.

In other words, if "limited with P;" doesn't add visibility while
preventing a semantic dependence on P, someone is making a mistake.

The whole idea of "limited with" is that a unit with a "limited with P;"
does not semantically depend on P.  If a unit semantically depends on P
for some other direct or indirect reason, then a "limited with P;" is at
best a waste of breath.  I argue that we should just make it illegal.
There is an error somewhere.  We don't know where.  But trying to make
complex rules and add lots of compiler support for something which is
not just a corner case but ALWAYS an error is silly.

The same type of logic holds in the alternative proposals.  In every
case we should remember what the goal is, and not try to add bells and
whistles that are of no possible use.  In the child package approach, we
want to create child packages that do not semantically depend on their
parents.  Cases where a child both does and does not semantically depend
on the parent (directly or indirectly) are both logically impossible and
silly.

**************************************************************

From: Randy Brukardt
Sent: Friday, March 28, 2003  8:01 PM

> In other words, if "limited with P;" doesn't add visibility while
> preventing a semantic dependence on P, someone is making a mistake.

Yes, of course. But we can't prevent that.

> The whole idea of "limited with" is that a unit with a "limited with P;"
> does not semantically depend on P.  If a unit semantically depends on P
> for some other direct or indirect reason, then a "limited with P;" is at
> best a waste of breath.  I argue that we should just make it illegal.
> There is an error somewhere.  We don't know where.  But trying to make
> complex rules and add lots of compiler support for something which is
> not just a corner case but ALWAYS an error is silly.

We tried that. But that is a severe ripple effect, and such an effect kills
incremental compilation -- which means that Rational cannot support such a
feature in their Ada compiler. We can't be putting vendors out of the Ada
business in this Amendment! We don't have enough as it is.

In case you don't see the problem, consider the program:

    package P is
        type T is tagged ...
        type Index is ...
    end P;

    package Q is
        procedure Proc ...
    end Q;

    with Q;
    limited with P;
    package R is
        type Acc is access all P.T'Class;
    end R;

We all agree this is legal.

Now, imagine that during maintenance, someone decides that they need a
version of Proc that takes an Index. So, they change Q to:

    with P;
    package Q is
        procedure Proc ...
        procedure Proc (I : in P.Index; ...);
    end Q;

Now, by your rule, R is now illegal. That's the ripple effect; for a
compiler like Rational's, every compilation unit that might semantically
depend on Q has to be checked. In real systems, that's pretty much
everything.

Note that Q doesn't depend on P.T at all, it just happens to depend on
something that's coincidentally in P.

That tends to suggest that Tucker is right in the discussion he and Pascal
are having. Indeed, it seems that there would be a ripple effect with
Pascal's rule: if someone adds an Index component to some type in Q, that
would make uses of that type illegal in R. I can't imagine that's what
Pascal wants.

    package Q1 is
       type Priv is private;
    private
       type Priv is record
          A : Integer;
       end record;
    end Q1;

    with Q1;
    limited with P;
    package R1 is
        type Acc is access all P.T'Class;
        Obj : Q1.Priv;
    end R1;

All is OK here. Now, if a maintenance programmer changes Q1 to get better
type checking:

    with P;
    package Q1 is
       type Priv is private;
    private
       type Priv is record
          A : P.Index;
       end record;
    end Q1;

    with Q1;
    limited with P;
    package R1 is
        type Acc is access all P.T'Class;
        Obj : Q1.Priv; -- Illegal by Pascal's proposed rules.
    end R1;

That looks like a second-hand ripple effect to me. It's especially annoying
because P.Index is never used as an incomplete type. (The big possible
annoyance with limited with is the lack of control over what becomes
incomplete. Here we see that it can be too much.)

**************************************************************

From: Robert I. Eachus
Sent: Friday, March 28, 2003  8:42 PM

> We tried that. But that is a severe ripple effect, and such an effect kills
> incremental compilation -- which means that Rational cannot support such a
> feature in their Ada compiler. We can't be putting vendors out of the Ada
> business in this Amendment! We don't have enough as it is.

Now I see what you are worried about, but I still don't see it as
specific to this feature.  If instead, package Q was changed so that
Proc required a new parameter, then any unit that depended on Q might
become illegal.  Sure this new rule would means that modifying any unit
that R depends on by adding a with P will make R illegal, and that would
mean adding a new check.  But I just don't see any better solution.

In reality, as I said we will see very few cases where a unit with a
limited with will be in some metric "distant" from the unit it does't
depend on. ;-)  If we have to come up with some restrictive rule that
makes it easier for Rational, lets do that.  But I think that trying to
make the dual path work is a waste of effort.

Let me take your example and extend it a bit...

     with R; -- added
     package P is
         type T is tagged record
                X: R.Acc; -- added
                ...
            end record;
         type Index is ...
     end P;

     package Q is
         procedure Proc ...
     end Q;

     with Q;
     limited with P;
     package R is
         type Acc is access all P.T'Class;
     end R;

Still legal right?  Now add your with P to package Q:

     with P;
     package Q is
         procedure Proc ...
         procedure Proc (I : in P.Index; ...);
     end Q;

and there is NO EASY WAY to make the program legal.  It is not a minor
detail, it is a major design problem.  And this will be the normal case.
  The only reason to have limited with is to break cycles.  If a
"limited with P;" can be added to Q, then you are okay.  So my
understanding is that this example is a red herring.  The real ripple
effect is that you can't add a with P; you have to add a limited with P;
to Q if you want things to compile.

**************************************************************

From: Robert I. Eachus
Sent: Friday, March 28, 2003  11:52 PM

I did some more thinking about this example and mapped it to the
"classic" example using employees and offices.  The surprise is that
adding a dual path does not make the program illegal--at least without
my proposed rule--but the "harmless" maintenance change does even more
violence to the program structure:

limited with Employees;
package Offices is
   type Office is tagged private;
   type Office_Pointer is access all Office'Class;
   ...
private
   type Office is tagged record
     ...
      Occupant: Employee_Pointer;
     end record;
end Offices;

limited with Offices;
with Perks;
package Employees is
   type Employee is tagged private;
private
   type Employee is tagged record
     ...
     Assigned_Office: Office_Pointer;
     Bonus_Package: Perks.Goodies;
   end record;
end Employees;

with Offices;
package Perks is
   type Goodies is tagged private;
private
   type Goodies is tagged record
     ...
     Corner_Office: Office_Pointer;
   end record;
end Perks;

Now in my proposal, a cowboy maintainer when he added the Corner_Office
to the list of possible perks could go ahead and change the limited with
Offices; to with Offices; and assuming that doing so did not cause any
naming conflicts, the program would now compile.  The careful maintainer
would add a subclass, say Executive, to Employee and put it in a
separate package that would not need a with or limited with for Offices.

Without my rule, the limited with on Employees is now a lie.  When
another cowboy maintainer comes along and wants to add a with clause to
Offices, he will need a fairly sophisticated tool--hopefully a part of
the compiler or linker error messages--to tell him what has gone wrong.

So as I see it, with my rule (you can't have a limited with Foo; on a
unit that already depends on the Foo) USERS should be happy.  Without it
you may find the same objections as to earlier proposals, that the
context clause does not accurately describe the dependencies.  It may be
that the proposal needs to be further modified to make it easier for
Rational to do incremental compilation, but I just don't see it.  It is
just one more constraint that has to be checked when a unit is modified.

**************************************************************

From: Pascal Leroy
Sent: Monday, March 31, 2003  4:24 AM

> That looks like a second-hand ripple effect to me. It's especially
> annoying because P.Index is never used as an incomplete type.

I agree that there's a ripple effect here.  However it's not too bad
from my perspective: if I have to disable incremental compilation in the
child units and body when a limited_with is added to a spec, that's a
nuisance, but I can live with it: it's not going to have a huge impact.
Compare this with having to disable incremental compilation in the
entire universe when any with_clause or renaming changes (that would not
be acceptable).

This being said, it would be better if the language were free of ripple
effects.

But to me the compelling argument is the breach of privacy: since I
don't think that assuming-the-worst is acceptable, I guess I will have
to agree (somewhat reluctantly) with Tucker's position.

> (The big possible
> annoyance with limited with is the lack of control over what becomes
> incomplete. Here we see that it can be too much.)

Right, but we are not going to give control on a declaration basis.  I
think the right answer here is that you can use child units to structure
your code appropriately: put the declarations that don't need to become
incomplete in the parent, and in a child put exclusively those types
that need to become incomplete.  Of course, you probably don't want to
do that on existing code, as it could entail major restructuring.  But
on new code, this is a perfectly sensible way to organize things.

> However, any solution that requires writing two things is going to be
> inferior to a solution that requires writing one. So, I suspect that
> alternative 6 ("limited with") is likely to carry the day based on
> better usability, even if it gives some implementers heartburn.

Randy has seen the light.  ;-)

**************************************************************

From: Tucker Taft
Sent: Monday, March 31, 2003  1:29 PM

I think we all would say "limited with" has a nice "feel" about it.
The big issue is implementability.  Eliminating the affects on
other compilation units resolves my main concern.  I think there might
still be some issue for folks with fat and thin pointers (e.g. GNAT), but they
don't seem worried, so why should I be?  We will have to implement
a pretty big chunk of code to build up the limited view from a
"quick-and-dirty" parse of a package.  I think "type T.C;" could
be significantly easier to implement.  But if the consensus is
to go with "limited with," I am "on board."

**************************************************************

From: Robert Dewar
Sent: Monday, March 31, 2003  2:24 PM

Well GNAT allows the use of thin or fat pointers, although I am not sure
this is a problem in any case.

**************************************************************

From: Randy Brukardt
Sent: Monday, March 31, 2003  5:39 PM

> > (The big possible
> > annoyance with limited with is the lack of control over what becomes
> > incomplete. Here we see that it can be too much.)

> Right, but we are not going to give control on a declaration basis.

Every proposal other than alternative 6 (limited with) does have control on
a declaration basis. Moreover, the lack of such control has been an
annoyance in Ada since it was invented (this isn't a new problem with
'limited with'; it's a problem with all with clauses).

> I think the right answer here is that you can use child units to structure
> your code appropriately: put the declarations that don't need to become
> incomplete in the parent, and in a child put exclusively those types
> that need to become incomplete.  Of course, you probably don't want to
> do that on existing code, as it could entail major restructuring.  But
> on new code, this is a perfectly sensible way to organize things.

"Can" is certainly true, but "desirable" is in the eye of the beholder.
Here, you're taking the other side (Tuck's side) of the same argument that
has typically been used against type C.T;. I don't think proposals should
have to rely on "structuring" in order to work; we don't want to impose a
structure on users of Ada - they should be allowed to use their own design.

In particular, I prefer to keep the ancillary types with the main O-O type.
Most O-O types have some types (typically elementary) that are used to help
define the operations for the main type - such as discriminants, lengths,
etc. I strongly disagree with Robert Eachus' contention that such types
indicate that something is wrong with the design.

For instance, Claw.Image_List includes:

package Claw.Image_List is
    type Root_Image_List_Type is abstract new Ada.Finalization.Controlled
...

    type Color_Kind_Type is (
        Default_Color_Scheme,
        Color_4_Bit,
        Color_8_Bit,
        Color_16_Bit,
        Color_24_Bit,
        Color_32_Bit,
        Color_Device_Dependent);

    type Image_List_Index_Type is new Claw.Natural_Int;

    type Overlay_Index_Type is new Claw.Natural_Int range 0 .. 15;
    NO_OVERLAY : constant Overlay_Index_Type := 0;

    function Get_Image_Count (List : in Root_Image_List_Type)
        return Claw.Image_List.Image_List_Index_Type;
        -- Returns the number of images in List.

    procedure Set_Image_Count (List : in Root_Image_List_Type;
                               New_Count : in Claw.Image_List.Image_List_Index_Type);
        -- Sets the number of images in List.

    ...

end Claw.Image_List;

I prefer to keep these extra types with the root type, as they can only be
used with that type. Putting them in a separate package just means more
times that you have to go searching for the declaration (if you use a lot of
use clauses) or more noise in the text (if you use fully expanded names).

> > However, any solution that requires writing two things is going to be
> > inferior to a solution that requires writing one. So, I suspect that
> > alternative 6 ("limited with") is likely to carry the day based on
> > better usability, even if it gives some implementers heartburn.
>
> Randy has seen the light.  ;-)

I don't think that's quite the right description. It's more that the Ada
community as a whole is not likely to give too much weight to the
implementability of a feature, and that whatever is nasty about 'limited
with' is not going to visible to average users until it was been implemented
in compilers for a while. So I thinks users will prefer 'limited with', and
there isn't going to be a obvious 'show stopper' for it.

On the other hand, unless there is some major revelation when I do the
implementation analysis (either way) or a customer with $$$, I don't see RRS
being able to afford the implementation costs for 'limited with'. A
simplified version of alternative 4 will meet our immediate needs with what
appears to be far less work.

But, our implementation needs should not stand in the way of what probably
will be judged to be the best solution.

Tucker said:
> I think we all would say "limited with" has a nice "feel" about it.
> The big issue is implementability.  Eliminating the affects on
> other compilation units resolves my main concern.  I think there might
> still be some issue for folks with fat and thin pointers (e.g. GNAT), but they
> don't seem worried, so why should I be?  We will have to implement
> a pretty big chunk of code to build up the limited view from a
> "quick-and-dirty" parse of a package.  I think "type T.C;" could
> be significantly easier to implement.  But if the consensus is
> to go with "limited with," I am "on board."

Which is pretty much where I stand (with the exception of the remark about
"type T.C"). I don't see any reason why 'limited with' couldn't be
implemented - the amount of work involved is unlikely to be of interest to
users.

**************************************************************

From: Randy Brukardt
Sent: Monday, March 31, 2003  6:31 PM

> Now in my proposal, a cowboy maintainer when he added the Corner_Office
> to the list of possible perks could go ahead and change the limited with
> Offices; to with Offices; and assuming that doing so did not cause any
> naming conflicts, the program would now compile.  The careful maintainer
> would add a subclass, say Executive, to Employee and put it in a
> separate package that would not need a with or limited with for Offices.

You didn't show the original program, which is important, because it appears
to be illegal. If Perks originally had a 'limited with Offices' in it (or
Perks doesn't exist at all), "Assigned_Office: Office_Pointer;" in Employees
is illegal, because Office_Pointer is incomplete.

In any case, this is not the case that I'm interested in. I was interested
in the case where one of the related elementary types that declared with
these types gets used. See my answer to Pascal for a real-life example of
what I'm talking about.

Or, to use your example (made legal assuming AI-230):

limited with Employees;
package Offices is
   type Office is tagged private;
   type Window_Count is range ...;
   ... -- Operations.
private
   type Office is tagged record
     ...
      Occupant : access Employees.Employee'Class;
      Windows : Window_Count;
     end record;
end Offices;

-- Employees remains the same.

package Perks is
   type Goodies is tagged private;
   ... -- Operations.
private
   type Goodies is tagged record
       Dental_Insurance : Boolean := False;
       Disability_Insurance : Boolean := False;
   end record;
end Perks;

Now, imagine that we got a call on Friday that we have to add support for a
minimum number of windows as a Perk by the end of the quarter (today is a
particularly good day for this example :-).

The obvious fix which preserves strong typing is:

with Offices;
package Perks is
   type Goodies is tagged private;
   ... -- Operations.
private
   type Goodies is tagged record
       Dental_Insurance : Boolean := False;
       Disability_Insurance : Boolean := False;
       Minimum_Windows : Offices.Window_Count := 0;
   end record;
end Perks;

One could argue that the type Window_Count should be moved somewhere else,
but that is unlikely for two reasons: 1) The number of windows is a property
of an Office - it isn't very meaningful by itself; 2) The time allotted for
the change means that major restructuring has to be avoided if at all
possible.

This is often how I have to do maintenance: when a customer has a critical
bug, you have to find a way to fix it, design be damned. If the fix is a
real mess, I'll note that and try to schedule some time to redo the code.
But that often doesn't happen for months or even years, because that sort of
cleanup rarely helps the customers or attracts new ones.

> Without my rule, the limited with on Employees is now a lie. ...

Not really. Employees is essentially doing a 'with type
Employees.Employee;'. The only problem is that 'Window_Type' is getting
dragged along as well (a type we don't even use in Employees). That, of
course, is a standard problem with with_clauses in Ada, it certainly isn't
new here. But we don't want rules that make that clearly obvious.

> So as I see it, with my rule (you can't have a limited with Foo; on a
> unit that already depends on the Foo) USERS should be happy. Without it
> you may find the same objections as to earlier proposals, that the
> context clause does not accurately describe the dependencies.

Ada with_clauses are too coarse to accurately describe dependencies (unless
you restrict yourself to one declaration per package -- which I doubt many
people do). I don't see that we can do better (or should try to do better)
with 'limited with'.

In any case, this maintenance change doesn't suddenly make the original
design invalid, and it's unlikely to confuse the programmer. But your rule
(or Pascal's, for that matter) might make it a lot harder to fix a problem
and preserve strong typing. In the example above, if various units become
illegal from adding the 'with Office;', the programmer (who is under severe
time constraints) is likely to toss the strong typing and just make the
number of windows an Integer. I don't think we want language rules which
encourage that.

> It may be that the proposal needs to be further modified to make it easier for
> Rational to do incremental compilation, but I just don't see it.  It is
> just one more constraint that has to be checked when a unit is modified.

I'd be interested to know Pascal's opinion on this proposal. It seems that
he's said that it would be unacceptable, but I'm not certain. Robert's rule
would require rechecking all possible dependents [at least to see if they
have a limited with] when a with_clause is added, even those that don't
directly with the unit. I believe that currently only units that have a with
for the changed unit need to be checked, so that potentially would be a
significant change.

**************************************************************

From: Pascal Leroy
Sent: Tuesday, April 1, 2003  3:09 AM

This is just another argument against my "revert to incomplete" rule and
in favor of Tuck's approach.  But then I think I have come to agree that
Tuck is right anyway because of the interaction with private types.

I actually think that in this example the quick-and-dirty fix is to use
Natural for component Minimum_Windows, and to come back later to do more
extensive restructuring.  The reason is that we are getting pretty close
to a situation where a cycle that involved two packages (Employees and
Offices) must be extended to involve a third package (Perks).  This is
particularly true if you take operations into account.  For instance, in
order to assign offices to employees, the package Offices may have to
export an operation like:

    procedure Check_Consistency (Of_Offices : Office;
                                 With_Perks : Perks.Goodies);

which would force limited_withs all over the place.  This is telling me
that this maintenance operation might actually expose shortcomings of
the original design, and therefore require extensive rearchitecturing.

> Ada with_clauses are too coarse to accurately describe dependencies (unless
> you restrict yourself to one declaration per package -- which I doubt many
> people do). I don't see that we can do better (or should try to do better)
> with 'limited with'.

Ada dependencies are at the level of units (well, declarative parts).
That has been the case from day 1 and I actually think it would be
counter-productive to offer finer granularity for type circularity
(among other things Ada is for programming in the large, and I don't see
it useful to micro-manage visibility in real-life programs).

...
> I'd be interested to know Pascal's opinion on this proposal. It seems that
> he's said that it would be unacceptable, but I'm not certain. Robert's rule
> would require rechecking all possible dependents [at least to see if they
> have a limited with] when a with_clause is added, even those that don't
> directly with the unit. I believe that currently only units that have a
> with for the changed unit need to be checked, so that potentially would be a
> significant change.

I didn't reply to Bob E.'s proposal because I don't see how it differs
from the classic ripple effect that we have been trying to avoid since
we started discussing 217.  Adding a with_clause to a unit would require
rechecking all the dependents of that unit to see which ones have a
limited_with_clause, and in these units locate the constructs that need
to become illegal (remember that we manage dependencies at the
declaration level, not at the unit level).

Not only would this entail a considerable implementation effort, it
would essentially negate incremental compilation.  The basic tenet of
incremental compilation is that the impact of a change is proportional
to the size of the change, not to the size of the whole program.  We go
to considerable lengths to ensure this property.  The analysis needed to
incrementally detect a dual path would be proportional to the size of
the whole program, so it's not acceptable.

**************************************************************

From: Robert I. Eachus
Sent: Tuesday, April 1, 2003  12:59 PM

Ah, but there is a better way to implement my proposal.  I think we can
agree that the cost of supporting limited withs should be zero if there
are no limited withs, and the cost of adding a with clause for a unit
that does not have any limited withs should be small.  To do this, keep
a database of units with limited withs containing the closure of all
dependencies for that unit.

If a limited with is added to a unit, there is work to be done.  If a
with is added to a unit that is in the closure database, you can
immediately check if it causes an illegality in another unit.  If not
there is work to be done to extend the closure database with the
closures of the current unit.  (This can be done at the point the
context clause and unit name have been processed.)

Also with this approach, units with limited withs but few or no
dependencies on non-predefined library units have little or no impact on
compilation times of other units.  I hope that this will be the common
case for units with limited withs.  There will be many units with units
with limited withs in their closure, but that is not a problem, it is
just the closures of units with limited withs themselves that should be
kept small.

Also note my exclusion of predefined units from the closures.  In a
heirarchy of units, only levels containing limited withs need this
closure database.

Oh, as a sort of humorous aside, there isn't much work for a large
program with lots of limited withs and few real withs.  That is a
program structure we might want to visit once we have a set of agreed
rules for limited withs--it may make sense.

**************************************************************

From: Pascal Leroy
Sent: Thursday, April 3, 2003  2:31 AM

> If a limited with is added to a unit, there is work to be done.  If a
> with is added to a unit that is in the closure database, you can
> immediately check if it causes an illegality in another unit.

There are too many occurrences of the words "with" and "unit" to be
sure, but I don't think this works.  The pseudo-code for handling the
situation where "with P" is added to unit Q should look like:

1:    for each unit U that has a limited with loop
2:        for each unit V which is mentioned by U in a limited_with loop
3:            if U depends (transitively) on Q and
4:                P depends (transitively) on V then
5:                    error;
6:            end if;
7:        end loop;
8:    end loop;

First, notice that the outer loop (line #1) has to process all the units
that have a limited_with.  If you assume that a constant proportion of
the units in the program will have limited_with_clauses, this loop runs
in O(N).  (I'll admit that the proportionality constant may be quite
small.)

More importantly, to answer the questions at lines #3 and #4 you need to
traverse the dependencies of units U and P.  The check at line #4 is
particularly annoying because (1) it has to be in the innermost loop and
(2) if you want to use a database to speed it up, this database has to
contain the transitive dependencies of each unit in the program, and
that means that the size of the database is in O(N**2).

We have customers with tens of thousands of units.  An algorithm with a
running time in O(N) is unappealing for such large-scale programs.  A
database of size O(N**2) is just ridiculous.

**************************************************************

From: Robert I. Eachus
Sent: Thursday, April 3, 2003  6:52 AM

I'll use with P; ... package Q is ... to identify the units involved.

No when you add the with clause, you probe the database, and the initial
probe takes time on the order of log(N).  If P is not in the database done.

If P is in the database, it may take time proportional to the number of
units with limited withs that depend on Q to verify whether or not the
unit mentioned in the with is included in these dependences, but I
expect that number to be small.  Notice that there is a lot of
cancelling out that goes on.  A unit has to be in the database only if
it depends on a unit with a limited with for X, and does not depend on a
unit with a direct dependence on X.  This is key.  In our canonical
example, any unit that depends on both Offices and Employees does not
need to be in the database.  In effect the limited withs cancel out.

If P does appear in the database then you need to update the database to
add the (limited with) closure of P to the (limited with) closure of Q.
  This is a join which will take time proportional to the sum of the
size of the two closures.  In my experience, when the ordinary closures
get large, the overall project is in trouble, independent of limited
withs and compiler issues.  If you think of that added with as a bomb
with effect proportional to the size of the closure you can see what
happens.      When the average number of side effects of a maintenance
change is greater than one, any maintenance change has global consequences.

But ignoring that for the moment, the worst case for this step is O(N).
  And if it is O(N), the time has nothing to do with the limited with.
You have to compute the closure of Q to insure that it is not circular.
  The size of the joined closures in the database will be much smaller
than the one you have to compute anyway, unless all units have limited
withs.

> More importantly, to answer the questions at lines #3 and #4 you need to
> traverse the dependencies of units U and P.  The check at line #4 is
> particularly annoying because (1) it has to be in the innermost loop and
> (2) if you want to use a database to speed it up, this database has to
> contain the transitive dependencies of each unit in the program, and
> that means that the size of the database is in O(N**2).
>
> We have customers with tens of thousands of units.  An algorithm with a
> running time in O(N) is unappealing for such large-scale programs.  A
> database of size O(N**2) is just ridiculous.

As I said above, well before the database gets that large, the structure
of the with clauses will be a maintenance nightmare.  But remember that
constant of proportionality.  Only units that depend on a unit that has
a limited with, and do not have a direct dependence on the same unit
need to be in the database.  If 1 in 100 units has a limited with, the
size of the database would be worst case 1/100 O(N**2).  But even that
is worst case.  Units with limited withs may be nearer than other units
to the root of the dependency tree, so lets say that it is ten times
more likely that a unit will depend on a unit with a limited with.  Now
the constant of proportionality is 1/1000.  And for programs that don't
use limited withs, the constant is zero.

And that I think has to be considered the final answer.  If there are no
units in the library with limited withs, the added effort is at worst
O(1)--checking to see that there are no limited withs to worry about.
If a unit does not depend on a unit with limited withs in a library with
limited withs, the time to determine that is O(logN), where N is the
number of units with limited withs.  As for a program where a large
fraction of the units have limited withs, and 100% of the units depend
on units with limited withs, that structure is going to be costly.  But
no more costly than a structure where those withs are replaced by withs.
  (Of course, the structure with the withs would be highly circular, but
that is beside the point.  We only have to worry about the compile time
effects of such a program if limited withs are such a wonderful feature
that "everyone" uses them.)  I fail to see that as an argument against
the limited with approach.

**************************************************************

From: Pascal Leroy
Sent: Thursday, April 3, 2003  9:57 AM

> Notice that there is a lot of
> canceling out that goes on.  A unit has to be in the database only if
> it depends on a unit with a limited with for X, and does not depend on a
> unit with a direct dependence on X.

Now you have completely lost me.  Aren't you proposing that this case be
illegal?  In a previous message you wrote 'It should be illegal for a
context to include a  "limited with P;" clause if the remainder of the
context semantically depends on P.'  So the case you mention is not
canceling, it's just illegal.

> But remember that constant of proportionality.
> Only units that depend on a unit that has
> a limited with, and do not have a direct dependence on the same unit
> need to be in the database.  If 1 in 100 units has a limited with, the
> size of the database would be worst case 1/100 O(N**2).

You seem to imply that only the units having limited_with_clauses need to be
in the database, and I don't understand this.  How do you handle the
following:

    package P is ... end P;

    with P;
    package Q is ... end Q;

    -- with Q; -- Uncomment this.
    package R is ... end R;

    limited with P;
    with R;
    package S is ... end S;

My understanding is that you are proposing that uncommenting the "with Q" on
R would make unit S illegal.  If that's the case, I don't see how you can
detect this case without storing (or recomputing) the dependencies of Q.
But then this means that _each_and_every_unit_ needs to be in your database,
so the proportionality constant is more like 1.  (In essence, I am saying
that you need pretty much the whole graph to detect the addition of a second
path, and from an information theory standpoint it doesn't sound too
surprising.)

**************************************************************

From: Robert I. Eachus
Sent: Thursday, April 3, 2003  5:03 PM

> Now you have completely lost me.  Aren't you proposing that this case be
> illegal?  In a previous message you wrote 'It should be illegal for a
> context to include a  "limited with P;" clause if the remainder of the
> context semantically depends on P.'  So the case you mention is not
> canceling, it's just illegal.

The difference is direct and indirect dependence.  All I am proposing is
that a unit call it Y that depends on a unit X cannot have a limited
with for X.  You obviously want the body of a unit whose declaration has
a limited for X to be able to have a regular with for X.  The point is
that the body of Y, if it has a direct with of X is back in the normal
Ada universe.  Even though it has a dependence on X and a dependence on
its declaration which has a limited with for X, you can drop it from the
database.

> You seem to imply that only the units having limited_with_clauses need to be
> in the database, and I don't understand this.  How do you handle the
> following:
>
>     package P is ... end P;
>
>     with P;
>     package Q is ... end Q;
>
>     -- with Q; -- Uncomment this.
>     package R is ... end R;
>
>     limited with P;
>     with R;
>     package S is ... end S;
>
> My understanding is that you are proposing that uncommenting the "with Q" on
> R would make unit S illegal.  If that's the case, I don't see how you can
> detect this case without storing (or recomputing) the dependencies of Q.

That is the tricky part.  The database needs to contain each unit that
depends on a units with a limited with Foo, and as mentioned above, does
not also depend directly or indirectly on Foo.  In this case, when first
compiling everything without the commented with, the compilation of S
puts two entries in the database: S --> P, and R --> P.  Since R is in
the database, recompiling R requires adding its new closure to the
database:  Q --> P, P --> P (Illegal.)

Only units with limited withs, and units that depend on units with
limited withs (with the exception explained above) need to be in the
database.  If a unit declaration has a huge number of with clauses,
and/or a large list of direct dependences the project (and your
compiler) have major maintenance problems unrelated to limited withs.
But for "well structured" Ada programs, bodies often have large
closures, declarations (I hope the only place we allow limited withs) don't.

In fact, if I were into defining quality measures for Ada code, I would
probably use the sum of the size of closures of package and generic
specifications divided by the number of such units as one measure.  If
you want to add in the size of the closures of bodies, there should be a
weighting factor that made body closures have one tenth the weight or
some such multiplier.

> But then this means that _each_and_every_unit_ needs to be in your database,
> so the proportionality constant is more like 1.  (In essence, I am saying
> that you need pretty much the whole graph to detect the addition of a second
> path, and from an information theory standpoint it doesn't sound too
> surprising.)

As I have said, if a large project with hundreds or thousands of units
gets to this point, it is already a disaster.  I and others on this list
can probably tell you horror stories like this.  My favorite was one Air
Force project where they recompiled the world every night.  (First clue
of a problem...)

They asked the Air Force for advise on a better compiler, because with
their current compiler, they were taking over 8 hours to do this and
were only 20% of the way into the project.  The call got directed to me,
and I went over the code structure with the developer.  I explained to
the developers how to structure an Ada program so that key library
packages didn't have to be recompiled so often.  (Moving with clauses to
the body where possible, replacing constants in package specs with
inlined functions and so on.)

About a month later I checked back to see how things were going.  They
were now doing reference builds once a week--good sign--and recompiling
the world took less than an hour.  But the key news was that error rates
were way down and code development was ahead of schedule. ;-)

This is why I take 'problems' of slow compilation with a large grain of
salt.  If the problem can be traced to bad program structure, I'd rather
find out about it as soon as possible.

**************************************************************

From: Pascal Leroy
Sent: Thursday, April 3, 2003  6:20 AM

Attached is an attempt at updating AI 217-06 based on the recent
discussion of the visibility rules.  I am not sure that we really need
to outlaw a use_clause for a limited view of a package.  It seems to me
that the Beaujolais-ish effect that Tuck originally discovered in this
area was due to an improper interaction between limited_with_clauses,
nonlimited_with_clauses, and visibility.  Tuck, what do you think?

[Editor's note: This is version /02 of the AI.]

**************************************************************

From: Tucker Taft
Sent: Thursday, April 3, 2003  10:50 AM

I think a nice simplification of the implementation is that
it is *only* expanded names that are affected by a "limited with"
clause, where the prefix is a package with a limited view.
Adding "use" clauses to the mix breaks that, and seems like an
unnecessary complexity.  Also, I believe we are requiring the
limited "with" itself not mention renamings.  This means that
it is kept as simple and straightforward as possible.  I might
go further and disallow uses of renamings of limited with packages,
rather than just saying they are limited as well.  This would really
make sure that all uses are clear and unambiguous.

But as far as whether the Beaujolais-ish effect is present,
I think it is still there if we allow "with P" inside "limited with P.Q;"
and we allow "use P" after a limited with of P.Q.

**************************************************************

From: Pascal Leroy
Sent: Thursday, May 15, 2003  3:29 AM

Attached is a document explaining how we would implement "limited with"
in our compiler.  This was one of my action items from the last meeting
(I know, I'm late...).

---

Preliminary Remarks

Because this document is mostly an internal design document, it refers to details of the
implementation of our compiler that are probably incomprehensible to the uninitiated.
For the benefit of the ARG readers, I am starting by providing some general information
on the terminology used below.

Our compilation process is decomposed into three phases: parsing, semantic analysis and
coding (coding is not particularly relevant to the implementation of limited with clauses).
At the end of the parsing phase the unit is said to be in the _source_ state, and at the end of
the semantic analysis it is said to be in the _analyzed_ state.  A unit in the source state has
a Diana tree with lexical information but no semantic information.  A unit in the analyzed
state has a Diana tree with all the static semantic information.

The _program_library_ is a cache gathering some of the information found in Diana trees.
The program library is used to determine which units need to be recompiled without
having to open numerous Diana trees.

A _decl_#_ is a hash code computed for every declaration in a unit, which describes the
static semantics of that declaration.  Decl #s serve two purposes:

1 - They are used to build cross-unit reference: a cross-unit reference has the form <unit
id, decl #>.
2 - They are used to implement incremental compilation: if a declaration is edited, and its
decl # changes, then all the declarations that depend on it must be recompiled.

The decl #s existing in a unit are stored in the program library and constitute the _exports_
of the unit.  Currently, decl #s are computed at the end of semantic analysis.

An _id_table_ is an AVL tree associated with a declarative part that maps an identifier (in
the lexical sense) to one or more Diana nodes.  Id tables are used for name resolution.

Most of the semantic information related to Diana trees is stored permanently.  However,
there exists information that is never referenced across compilation units, but needs to be
recorded on a Diana tree during the course of a single compilation.  This information is
stored in _temporary_attributes_ of the tree.

I think that's all for the terminology.


Implementation Model

In the course of parsing a library package specification, we construct limited decl #s in
the parser reduction actions.  Limited decl #s are only built for (visible) types and
package declarations.  They are only based on (1) the type or package name and (2) the
type's tagged-ness.  These decl #s constitute the limited exports of the unit, and they are
recorded as such in the program library.  (Meaning that we probably need to run a
minimal exports collection phase after parsing.)

When a unit is analyzed, full decl #s are allocated for each declaration (as they are today).
These full decl #s don't replace the limited ones, because there may be clients which
reference the limited decl #s.  Therefore, we may have two decl #s for a given
declaration.  This is not a problem when following a pointer, but note that when building
a reference we must be cautious to use the limited or full decl # depending on whether we
have visibility on the limited or full view of a unit.

In recompilation terms, consider the situation where a declaration is edited in such a way
that its full decl # changes, but not its limited decl #.  Only those clients that referenced
the full decl # need to be recompiled.  This ensures recompilation minimization even in
the presence of limited withs.

We cannot find ourselves in a situation where the full view of a unit is somehow
inconsistent with the limited view of the same unit, because as soon as a declaration is
edited in the Ada editor, it is reparsed, its limited decl # is recomputed, and its full decl #
is dropped on the floor.  The normal incremental compilation mechanisms then kick in to
ensure that the proper clients get obsolesced.

I don't foresee any issue with timestamps and the like: after parsing (and limited decl
numbering) has taken place, a unit looks pretty much as if it had been compiled and then
edited, so again incremental compilation should save the day.

Also as part of parsing, id tables must be built for each declarative part that is part of the
limited view.  These limited id tables only contain the names of types and packages.
They are used for name resolution in clients which only see the limited view of the unit.
Note that they are not overwritten when the unit is analyzed, because it is possible to
reference an analyzed unit through a limited with.  So for some Diana nodes we'll have
two id tables.

At the beginning of the semantic analysis of a unit, the context clauses of that unit and of
its ancestors are processed.  If a limited with clause is encountered before a nonlimited
with clause, we records this fact, so that name resolution is later performed using the
limited id table (we might need a temporary attribute to facilitate this, like we do for
private with).

When the name of a type is resolved through a limited id table, a temporary attribute is
set on that type to record the fact that its type spec is effectively incomplete, even though
it might look like a full type spec in the Diana tree.  This attribute must be consulted
when checking for premature usages of incomplete types.  (We already have such an
attribute for checking premature usages of private types, so I expect that this part will
more-or-less fall out naturally.)

Note that supposedly we do not have to bother with this attribute for types declared in
other packages (e.g., an access-to-incomplete type declared in an intermediate package).
Say that such a type was compiled against the limited view of a unit.  If we see the full
view of that unit, following Diana pointers through the intermediate unit will leads us to
the full type definition, which is what we need.  If we only see the limited view of the
original unit, then we should never need to follow any Diana pointer through the
intermediate unit, because the corresponding source constructs ought to be illegal.  Of
course, I fully expect that there will be cases where we do too much look-through, but
these should be fixed, as they probably correspond to know-too-much bugs.

There are some contexts (e.g., use clauses) where the name of a unit referenced through a
limited with is illegal.  Presumably this is a legality rule, not a name resolution rule, so it
is checked pretty much like the rules related to private with clauses, i.e., at the point
where we decorate a name, after name resolution has been performed.

The compilation ordering algorithm must be modified to take limited with relationships
into account.  Before analyzing a unit, its full closure (i.e., the closure obtained by
following both limited with clauses and nonlimited with clauses) must be in the source
state.  Its normal closure (i.e., the one obtained by following only the nonlimited with
clauses) must be in the analyzed state.  There is no prerequisite for promoting a unit to
the source state (this is good, it means that you don't do compilation ordering when
hitting the Syntax button).

When a unit is demoted to source, its full decl #s, full id tables and full export lists are
deleted, but the limited decl #s, limited id tables and limited export lists must be retained.

The coding phase should be unaffected by all this.  Before a unit is coded, the
compilation ordering algorithm must ensure that its full closure is analyzed.  This
guarantees that coding never lands in an unsemanticized unit.

**************************************************************

From: Tucker Taft
Sent: Thursday, May 15, 2003  8:37 AM

Thanks for doing this.  I guess I must be late too (I forgot about
this part of the assignment).  Perhaps Randy can do his usual
excellent job of reminding me of the remaining items on my todo
list for the upcoming ARG meeting ;-).

As you may have noticed, the AdaUK folks showed a surprisingly
strong preference for the "type C.T" approach, followed by
the "limited with."  Since I presented the material, part of
this might be that my bias showed through, though I tried to
be impartial (other observers there, such as John Barnes, might
be able to comment on my success), and frankly, I like the
"limited with" proposal from a user point of view.

Some of the reasons cited for preferring the type C.T approach were
that it seemed to be a smaller change that was easier to understand,
and that it didn't introduce any new kinds of module interdependence.
It also seemed to be more "structured" to some members of the audience.

For what that's worth...

**************************************************************

From: Randy Brukardt
Sent: Tuesday, June 17, 2003  2:12 AM

Here is the implementation report I promised at the last meeting for all
three live AI-217
proposals. I know that these are really late, but at least you'll have three
days in which to read them. (Of course, if you're at Ada Europe already,
that probably is really about 3 minutes. :-(

[Editor's note: The report can be found in the !appendix to AI-217-07.]

**************************************************************

From: Ed Schonberg
Sent: Thursday, July 10, 2003  12:14 PM

At the last ARG meeting, the following example was discussed:

                     package P is
                        type T is record ...
                     end P;
-----------------------------------------------------------------
limited with P;                | with P;
package Q is                   | package R is
    type Acc_Lim is access P.T;|   procedure Proc (X : P.T);
    X : Acc_Lim;               | end R;
end Q;                         |
-----------------------------------------------------------------
                      with Q, R;
                      limited with P;      --   (1)
                      procedure Main is
                      begin
                         R.Proc (Q.X.all); -- Should be legal
                      end;
-----------------------------------------------------------------

It was agreed that the call was legal, because the context of the call
provided the full view of type T, and the explicit dereference, although
it yielded the limited view, was allowed to "be aware of" the full view.
Furthermore, the presence or absence of the limited with_clause  (1) did
not affect the legality of the call.

On the other hand, there was some agreement that, in the same context,
with the Limited_With present,

                         R.Proc (Q.T'(Q.X.all));

was illegal, because within S, Q.T can only denote the limited view, and
an incomplete type cannot be the prefix of a qualified expression.

THE rule would seem to be that a direct use of the type name only sees the
limited view, but uses of the type that come from the semantic closure are
legal if the non-limited view is available (although the rules will probably
not mention the term semantic closure).

Consider now the following variation on the previous example:

                     package P is
                        type T is record ...
                     end P;
-----------------------------------------------------------------------------
limited with P;                | limited with P;            | with P;
package Q is                   | package R is               | package S is
    type Acc_Lim is access P.T;|   procedure Proc (X : P.T);|    ...
    X : Acc_Lim;               | end R;                     | end S;
end Q;                         |                            |
-----------------------------------------------------------------------------
                      with Q, R;
                      with S;          --  (2)
                      limited with P;
                      procedure Main is
                      begin
                         R.Proc (Q.X.all); --  legal?
                      end;

Without the with_clause (2), this should be illegal, because the full view
of the type is nowhere available (both Q and R have limited_withs on P).
However, once the with_clause on S is present, the semantic closure does
make the full view of P available. Following the previous example, this
would be legal. Either this is counterintuitive (and unpermissibly alcoholic)
or else the previous attempt at stating the rule is incorrect.

Guidance is welcome. The query comes in connection with a prototype implemen-
tation of Limited_With that Javier Miranda and I have been constructing within
GNAT. The implementation so far has been straightforward, and provides the
desired functionality for all straightforward examples tested so far.  We
hope that the precise semantics of pathological cases is less of a thicket than
it appears to us right now!

**************************************************************

From: Gary Dismukes
Sent: Thursday, July 10, 2003  1:21 PM

> Furthermore, the presence or absence of the limited with_clause  (1) did
> not affect the legality of the call.

Right, the call is legal because the expected type of the parameter
is the full view (that's the "context of the call").

> On the other hand, there was some agreement that, in the same context,
> with the Limited_With present,
>
>                          R.Proc (Q.T'(Q.X.all));
>
> was illegal, because within S, Q.T can only denote the limited view, and
> an incomplete type cannot be the prefix of a qualified expression.

My understanding as well (though you mean P.T, not Q.T in the above).

> THE rule would seem to be that a direct use of the type name only sees the
> limited view, but uses of the type that come from the semantic closure are
> legal if the non-limited view is available (although the rules will probably
> not mention the term semantic closure).

I believe that the model we discussed was based on whether the full view
is available via an expected type, not based on whether the full view
is in the semantic closure.  If it were based on the semantic closure
then ripple effects could occur, which we're trying to avoid.

> Consider now the following variation on the previous example:
> ...
>                       with Q, R;
>                       with S;          --  (2)
>                       limited with P;
>                       procedure Main is
>                       begin
>                          R.Proc (Q.X.all); --  legal?
>                       end;
>
> Without the with_clause (2), this should be illegal, because the full view
> of the type is nowhere available (both Q and R have limited_withs on P).

Right.

> However, once the with_clause on S is present, the semantic closure does
> make the full view of P available. Following the previous example, this
> would be legal. Either this is counterintuitive (and unpermissibly alcoholic)
> or else the previous attempt at stating the rule is incorrect.

No, the semantic closure shouldn't be used here, only the expected type
should be taken into account.  At least that was my understanding of
the model discussed at the meeting.

**************************************************************

From: Tucker Taft
Sent: Thursday, July 10, 2003  1:21 PM

> THE rule would seem to be that a direct use of the type name only sees the
> limited view, but uses of the type that come from the semantic closure are
> legal if the non-limited view is available (although the rules will probably
> not mention the term semantic closure).

I think the rule is rather that a dereference that delivers an
incomplete type is allowed so long as it is used in a context
where the corresponding full type is a) immediately in scope, or
b) declared within a (non-limited) with'ed package, or c) provided
by the current complete context of overloading (e.g. as the expected
type in this example).

Mere presence in the "semantic closure" is *never* sufficient.

> Consider now the following variation on the previous example:
....[See it above - ED]

The declaration of procedure Proc would only be legal if T were a *tagged*
type, since only tagged incomplete types may be used as formal parameter types.
Presuming the example is changed to have T tagged, then...

>                       with Q, R;
>                       with S;          --  (2)
>                       limited with P;
>                       procedure Main is
>                       begin
>                          R.Proc (Q.X.all); --  legal?
>                       end;

The legality is unaffected by the presence of the "with" of S,
since it doesn't directly contain the declaration of the full type
(the semantic closure is irrelevant).

Since none of the 3 possibilities I gave above are satisfied, this should
be considered illegal.  However, I forget whether we allowed for
a special case for a dereference producing a *tagged* incomplete type.
We didn't want to allow that for dynamically tagged controlling parameters,
because those require fetching the tag for dispatching or a tag check.
I'm not sure what we decided about the case of non-controlling or
statically-tagged parameters.  If we allow it, then that becomes
a case (d) to be added to the cases (a)..(c) given above.

...

The key message is presence in the semantic closure is never sufficient.
All rules must be based on what is directly with'ed, or in the immediate
scope, or in the overloading context.

**************************************************************

From: Randy Brukardt
Sent: Thursday, July 10, 2003  3:16 PM

> Since none of the 3 possibilities I gave above are satisfied, this should
> be considered illegal.  However, I forget whether we allowed for
> a special case for a dereference producing a *tagged* incomplete type.
> We didn't want to allow that for dynamically tagged controlling parameters,
> because those require fetching the tag for dispatching or a tag check.
> I'm not sure what we decided about the case of non-controlling or
> statically-tagged parameters.  If we allow it, then that becomes
> a case (d) to be added to the cases (a)..(c) given above.

It could be valuable to allow this for class-wide parameters (where there is
no real dereference, since tagged types are pass-by-reference). The same is
true of statically bound calls. The important thing is to avoid any
dereference where there is access to the object itself (as in accessing the
tag or discriminants), because we don't know enough about the components to
do anything. It's hard to say how that rule should be written, though.

**************************************************************

From: Ed Schonberg
Sent: Thursday, July 10, 2003  2:56 PM

The first two cases are clear. The third one (the overloading context) presents
serious implementation problems, at least for us. Our implementation assumes
that at any given time in the analysis there is only one view of the type that
is available. That is either the limited view Tl or the non-limited view Tn.
When we obtain the expected type from the context, for example the formal of a
subprogram, there is no indication of the visibility that the type may have
had at the time the subprogram declaration was analyzed. It's just T, whose
attributes are either those of Tl or Tn. If the current view is Tl, it is
possible to indicate that Tn is available (if it showed up in the analysis
of the closure) but it is never the case that two constituents of an expression
have different views of the same type. To go back to the first example:

                                 package P is
                                   type T is tagged record...
                                 end P;

limited with P;   --  (1)                    with P;   --  (2)
package Q is                                 package R is
   type Acc_Lim is access P.T;                  procedure Proc (X : P.T);
   X : Acc_Lim;                              end R;
end Q;

                              with Q, R;
                              limited with P;
                              procedure Main is
                              begin
                                 R.Proc (Q.X.all);
                              end Main;

By the stated rule this is legal, but it would be illegal if (1) and (2)
were exchanged (because the context provided by the call would then be
the limited view, right?).  However, when we analyze Main, these two cases
are indistinguishable (in our implementation of course, but I thought that
Janus and Rational had a similar view of things). Therefore, the proposed
rule does not seem to be implementable for us, without major changes in
data structures (or a sudden blinding flash of inspiration!).

**************************************************************

From: Tucker Taft
Sent: Thursday, July 10, 2003  3:30 PM

Perhaps you can treat the incomplete types made available in a limited view
of a package as truly distinct types that just happen to match
a particular full type during overload resolution.  Clearly
you must have rules that allow two different types to match
under certain circumstances.  For example, T'Class and T are clearly
different but they match in certain contexts.  Similarly, certain named
access types can be used in contexts where the expected type
is an anonymous access type.

Incomplete types can only be used in a few places (e.g. as designated types,
and if tagged, as formal parameters).  These places seem well enough defined
that having a special matching rule wouldn't be too bad, presuming you
can manage to treat them as truly distinct types in most cases.

**************************************************************

From: Robert I. Eachus
Sent: Friday, July 11, 2003 11:46 AM

I hate it when this happens--but it is just the way my mind works. I
think/hope that the original case has been addressed. But there is
another case that looks troublesome:

                     package P is
                        type T is tagged record ...
                        procedure Proc (X: T);
                     end P;
--------------------------------------------------------------------------------------------
limited with P;                       | with P;                        |  with P,R;
package Q is                          | package R is                   |  package body Q is
    type Acc_Lim is access P.T'Class; |   type Child is new P.T;       |  begin
    X : Acc_Lim;                      |   procedure Proc (X : Child);  |    X := new Child;
end Q;                                | end R;                         |  end Q;
--------------------------------------------------------------------------------------------
                      with P, Q;
                      limited with R;
                      procedure Main is
                      begin
                        Proc (Q.X.all); --  which Proc is called?
                      end;

I think that the "right" answer has to be that R.Proc gets called via
dispatching.  There is a possibility if I had put the call elsewhere that
Program_Error could occur, but that is nothing new.  I don't think there are
any cases where type T could be private and only visible through a limited
with, and the access type is visible through a with.

***************************************************************

From: Tucker Taft
Sent: Friday, July 11, 2003 11:46 AM

I don't know what the issue is here.  package body Q is illegal because
it isn't needed by its spec, but presumably you add a pragma Elaborate_Body
or some sort of dummy procedure which will cause it to be allowed,
it seems clear that by the time you get to Main, Q.X points at an object
with tag R.Child (you need to add a "use" for R in the body of Q as well).
In general, whether you use limited with or with has no run-time effect,
except perhaps elaboration order, so what exactly is your question?

***************************************************************

From: Robert I. Eachus
Sent: Saturday, July 12, 2003  6:32 PM

>I don't know what the issue is here.  package body Q is illegal because
>it isn't needed by its spec, but presumably you add a pragma Elaborate_Body
>or some sort of dummy procedure which will cause it to be allowed,

Oops, consider it done.

>it seems clear that by the time you get to Main, Q.X points at an object
>with tag R.Child (you need to add a "use" for R in the body of Q as well).

Just to make sure we are on the same wavelength, assume I change the body of Q to:
with P,R;
package body Q is
begin
X := new R.Child;
end Q;

to fix that.

>In general, whether you use limited with or with has no run-time effect,
>except perhaps elaboration order, so what exactly is your question?

Sorry if that wasn't clear. It is possible for a limited with to result
in a (run-time) elaboration order that results in additional ABE checks
being necessary. I think I have convinced myself that these are already
language defined checks, but that get optimized away currently.

***************************************************************

From: Ed Schonberg
Sent: Sunday, September 28, 2003  4:04 PM

!standard 10.01.02   (03)                           03-04-03  AI95-00217-06/02
!standard 10.01.02   (04)
!standard 10.01.02   (08)
!standard J.10       (00)
!class amendment 03-02-04
!status work item 03-02-04
!status received 03-02-04
!priority Medium
!difficulty Hard
!subject Limited With Clauses

This is a summary of our implementation of Limited_With clauses in GNAT.
In a nutshell: after a few false starts, the implementation proved to be
straightforward, thanks to a critical suggestion from Tucker. The changes
to various phases of the compiler were very localized, and amounted to no
more than a few hundred lines of code. We reused some of the machinery of our
implementation of now-discarded With_Type clauses. The current prototype
implementation is complete, except for one exception mentioned below.

Overview
--------

GNAT is a source-based compiler. If unit U1 has a limited_with clause that
mentions unit U2, then U2 must be present in the environment. U2 is parsed,
and its abstract syntax tree is constructed. The tree is not analyzed, but
undergoes a simple processing that identifies type declarations in the
visible part of the unit (and the visible part of nested packages). These
type declarations are used to created shadow entities in the tree, that
receive the semantic annotations that describe incomplete types. These
shadow entities are made visible while the limited_with_clause is effective.
In a context where a regular with_clause on U2 is effective, the shadow
entities are invisible. Most of the implementation code handles the
installation and removal of the shadow entities and their counterparts,
within the data structures that implement the visibility rules.

There are small changes to the basic type-checking routine, to some binder
routines, and to the parser. These changes amount to less than 50 lines of code.

Semantics.
----------

The semantic complications introduced by this construct arise from cases
of multiple access paths: in a given context, an entity may be available
through a limited_with_clause, as well as through a regular_with clause,
a renaming, direct visibility in a child, etc. The proposed legality rules
for limited_with_clauses make some of these cases illegal, and otherwise try
to minimize the disruption that the addition of such a clause might have
on an existing program.

At first sight, the limited_view of a type is analogous to the partial
view of a private type, and a similar implementation model suggests itself.
In GNAT, for entities with multiple views, it is always the case that there
is a single tree node (a defining_occurrence) that holds the current view of
the entity. In the case of a private type, the occurrence in the
private declaration (that is to say, the specific node in the abstract
syntax tree that is the defining_identifier of the declaration) is used to
denote the entity. Its contents are swapped with the semantic information of
the full view when needed (i.e. when compiling the private part, the package
body, or the private part or body of a child unit), but at all times it is the
same node in the syntax tree that embodies the identity of the type. Our first
implementation attempt therefore treated the limited_view in the same way:
the semantic information attached to the entity was either incomplete or
complete, but the same node of the AST was used in all cases.

However, this model cannot handle correctly cases of multiple access paths,
because it assumes that at a given point, only one of the possible views is
available. However, there are cases when two components of a construct
denote different views of the same type.

The most common usage of limited_with_clauses involves multiple
access paths: if a specification for package P has a limited_with_clause in
order to import some type T from package Q, it is usually the case that the
body of P will have subprogram bodies that manipulate T, and thus the body of
P will have a regular with_clause on Q. Thus, in the body of P, the type T is
available both through the limited_with_clause on the library_unit, and the
regular with_clause on the body. In such a case the desired semantics are
obvious: the non-limited view of T is available.
In other cases, the semantics are more subtle, as in the following example:

Consider a package A that declares a type TA, a package P that has a limited
view of A, and a package Q that has a non-limited view of A. Both P and Q
declare acces types whose designated type is TA. In context where both P
and Q are visible (but A might not be), objects designated with these access
types must be compatible, even though one of them denotes the limited view of
TA and the other the non-limited view.

As a consequence, we must treat the limited_view of a type as an entity in
its own right, and add special type resolution rules to make the limited
and non-limited views compatible. We introduce shadow entities to hold the
limited_views, and links between the shadow entity and its partner, to
simplify type-checking. It turns out that these shadow entities (which
collectively correspond to the "abstract" of the package)  make the
implementation simpler that the one-entity model with a swapping mechanism
between views. They also make it easier to maintain necessary invariants
between the semantic analyzer and the back-end (that is to say the tree
transducer that uses the Ada AST to build the GCC tree).

Implementation details.
-----------------------

The changes to compiler internals are only of interest to GNAT maintainers.
It may be worth listing the following:

a) There were small changes in the contents of AST nodes for with_clauses,
for package declarations, and for type declarations.

b) The package that handles compilation units and the processing of
context clauses was obviously the most affected. The following subprograms
were added:

Build_Limited_View: to create an incomplete (shadow) entity.

Install_Limited_Withed_Unit: to make the incomplete entities visible

Remove_Limited_With_Clause: to remove the incomplete entities from visibility

Expand_Limited_With_Clause: to generate limited_with clauses on the ancestors
of a unit that is mentioned explicitly in a limited_with clause (as long as
those ancestors are not mentioned in an effective regular with_clause).

In addition, the following existing routines had to be modified to make use of
the above (names should be self-explanatory): Analyze_Compilation_Unit,
Analyze_With_Clause, Install_Context_Clauses, and Remove_Context_Clauses.

c) The package that handles type declarations was modified to retrieve the
non-limited view (when available) of the designated type of an access type
that denotes the limited view.

d) There were a few modifications to the binder, to prevent limited_with
clauses from creating semantic dependencies (that's the whole point, after
all!).

e) Finally, there were small modifications to error checking and reporting:
half-a-dozen error messages were added, and some error conditions disabled
when dealing with limited views.

To test the implementation on the examples proposed in various versions of
this AI, we also modified the processing of subprogram declarations, to allow
tagged incomplete formal parameters. All the examples compiled properly
(or were rejected correctly).

Pending.
--------
The single gap in the implementation is the rule that requires the closure of
a partition to include the body of a unit that is mentioned in a limited_with
clause, when there is no regular with_clause on that unit elsewhere in the
partition. This is a bind-time rule that is simple enough to implement: we
did not have time to include yet, and suspect that partitions that require it
will be execeedingly rare.

***************************************************************

From: Tucker Taft
Sent: Sunday, September 28, 2003  4:26 PM

Cool!  Glad I made a useful suggestion.
Which one was it?

***************************************************************

From: Ed Schonberg
Sent: Sunday, September 28, 2003  8:17 PM

That the limited views be full-blown new entities, rather than
transient attributes of existing ones. As usual, there is no
design problem that can't be solved with an additional level of
indirection. In any case, thanks again for sending us on the right
path!

***************************************************************

From: Randy Brukardt
Sent: Tuesday, September 30, 2003  3:36 PM

> As a consequence, we must treat the limited_view of a type as an entity in
> its own right, and add special type resolution rules to make the limited
> and non-limited views compatible. We introduce shadow entities to hold the
> limited_views, and links between the shadow entity and its partner, to
> simplify type-checking. It turns out that these shadow entities (which
> collectively correspond to the "abstract" of the package)  make the
> implementation simpler that the one-entity model with a swapping mechanism
> between views. They also make it easier to maintain necessary invariants
> between the semantic analyzer and the back-end (that is to say the tree
> transducer that uses the Ada AST to build the GCC tree).

Janus/Ada has always used a model like this for incomplete and private
types. However, type matching is done independently of visibility (with the
exception of a nasty hack for private types in parent units).

Could you explain in more detail the matching rules needed here? I'm trying
to determine if they can be implemented in Janus/Ada without doing great
violence to the existing type mechanisms.

***************************************************************

From: Ed Schonberg
Sent: Tuesday, September 30, 2003  5:23 PM

>  Janus/Ada has always used a model like this for incomplete and private
>  types. However, type matching is done independently of visibility (with the
>  exception of a nasty hack for private types in parent units).

"nasty hack" and "implementation of private types" are closely related in
my experience!

>  Could you explain in more detail the matching rules needed here? I'm trying
>  to determine if they can be implemented in Janus/Ada without doing great
>  violence to the existing type mechanisms.

The basic type-checking routine for T1 and T2 checks whether T1 and T2 have
the same base type, or one of them is a universal type and the other a type
of the class, etc. In this case, T1 and T2 are two distinct entities: one
is a limited view which appears as an incomplete type, the other is some
(partial or full view) of some other type, so they will not type-check.
We have to recognize that these two entities are actually views of the same
type, and that is done by means of a link between the two, created when the
limited views are created. I don't know whether this implementation artifact
could be used in Janus/Ada, but in our case it falls out of the data structure.
This link cannot be the same one that relates the two views of a single type,
but it is needed precisely to exchange limited and non-limited views when
installing/removing parent units, etc.

***************************************************************

From: Randy Brukardt
Sent: Tuesday, September 30, 2003  5:39 PM

I see. In our case, an incomplete type is just a view of the completion. Of
course, there is a link between the two nodes to represent that. (Private
types are implemented the same way). What we do for private types is
'ignore' the existence of the link when the full type cannot be used (the
visible part of a child, for instance). That's a nasty hack, because it
doesn't scale well (OK, it doesn't scale at all - it only works for sure in
the innermost nested program unit).

The main reason for the links existing always is so that any stub or spec
has the same view of the real types as the units that it withs. Otherwise,
there are code generation issues (you might use the wrong parameter passing
mechanism, for one example). (Code generation never depends on visibility.)

I don't want to have to walk the type table adding or removing links once
the context clause is processed, and I'd rather not have to add a new
mechanism for adding links during context clause processing (that is very
tricky to do right, even without visibility issues).

Humm. Lots to think about here. Well, thanks for your input.

***************************************************************




Questions? Ask the ACAA Technical Agent